clickhouse-benchmark

Documentation for clickhouse-benchmark

Connects to a ClickHouse server and repeatedly sends specified queries.

Syntax

$ clickhouse-benchmark --query ["single query"] [keys]

or

$ echo "single query" | clickhouse-benchmark [keys]

or

$ clickhouse-benchmark [keys] <<< "single query"

If you want to send a set of queries, create a text file and place each query on the individual string in this file. For example:

SELECT * FROM system.numbers LIMIT 10000000;
SELECT 1;

Then pass this file to a standard input of clickhouse-benchmark:

clickhouse-benchmark [keys] < queries_file;

Command-line options

• --query=QUERY — Query to execute. If this parameter is not passed, clickhouse-benchmark will read queries from standard input.
• --query_id=ID — Query Id.
• --query_id_prefix=ID_PREFIX — Query Id Prefix.
• -c N, --concurrency=N — Number of queries that clickhouse-benchmark sends simultaneously. Default value: 1.
• -C N, --max_concurrency=N — Gradually increases number of parallel queries up to specified value, making one report for every concurrency level.
• --precise — Enables precise per-interval reporting with weighted metrics.
• -d N, --delay=N — Interval in seconds between intermediate reports (to disable reports set 0). Default value: 1.
• -h HOST, --host=HOST — Server host. Default value: localhost. For the comparison mode you can use multiple -h keys.
• -i N, --iterations=N — Total number of queries. Default value: 0 (repeat forever).
• -r, --randomize — Random order of queries execution if there is more than one input query.
• -s, --secure — Using TLS connection.
• -t N, --timelimit=N — Time limit in seconds. clickhouse-benchmark stops sending queries when the specified time limit is reached. Default value: 0 (time limit disabled).
• --port=N — Server port. Default value: 9000. For the comparison mode you can use multiple --port keys.
• --confidence=N — Level of confidence for T-test. Possible values: 0 (80%), 1 (90%), 2 (95%), 3 (98%), 4 (99%), 5 (99.5%). Default value: 5. In the comparison mode clickhouse-benchmark performs the Independent two-sample Student’s t-test to determine whether the two distributions aren’t different with the selected level of confidence.
• --cumulative — Printing cumulative data instead of data per interval.
• --database=DATABASE_NAME — ClickHouse database name. Default value: default.
• --user=USERNAME — ClickHouse user name. Default value: default.
• --password=PSWD — ClickHouse user password. Default value: empty string.
• --stacktrace — Stack traces output. When the key is set, clickhouse-benchmark outputs stack traces of exceptions.
• --stage=WORD — Query processing stage at server. ClickHouse stops query processing and returns an answer to clickhouse-benchmark at the specified stage. Possible values: complete, fetch_columns, with_mergeable_state. Default value: complete.
• --roundrobin — Instead of comparing queries for different --host/--port just pick one random --host/--port for every query and send query to it.
• --reconnect=N — Control reconnection behaviour. Possible values 0 (never reconnect), 1 (reconnect for every query), or N (reconnect after every N queries). Default value: 0.
• --max-consecutive-errors=N — Number of allowed consecutive errors. Default value: 0.
• --ignore-error,--continue_on_errors — Continue testing even if queries failed.
• --client-side-time — Display the time including network communication instead of server-side time; Note that for server versions before 22.8 we always display client-side time.
• --proto-caps — Enable/disable chunking in data transfer. choices (can be comma-separated): chunked_optional, notchunked, notchunked_optional, send_chunked, send_chunked_optional, send_notchunked, send_notchunked_optional, recv_chunked, recv_chunked_optional, recv_notchunked, recv_notchunked_optional. Default value: notchunked.
• --help — Shows the help message.
• --verbose — Increase help message verbosity.

If you want to apply some settings for queries, pass them as a key --<session setting name>= SETTING_VALUE. For example, --max_memory_usage=1048576.

Environment variable options

The user name, password and host can be set via environment variables CLICKHOUSE_USER, CLICKHOUSE_PASSWORD and CLICKHOUSE_HOST.
Command line arguments --user, --password or --host take precedence over environment variables.

Output

By default, clickhouse-benchmark reports for each --delay interval.

Example of the report:

Queries executed: 10.

localhost:9000, queries 10, QPS: 6.772, RPS: 67904487.440, MiB/s: 518.070, result RPS: 67721584.984, result MiB/s: 516.675.

0.000%      0.145 sec.
10.000%     0.146 sec.
20.000%     0.146 sec.
30.000%     0.146 sec.
40.000%     0.147 sec.
50.000%     0.148 sec.
60.000%     0.148 sec.
70.000%     0.148 sec.
80.000%     0.149 sec.
90.000%     0.150 sec.
95.000%     0.150 sec.
99.000%     0.150 sec.
99.900%     0.150 sec.
99.990%     0.150 sec.

In the report you can find:

• Number of queries in the Queries executed: field.

• Status string containing (in order):

  • Endpoint of ClickHouse server.
  • Number of processed queries.
  • QPS: How many queries the server performed per second during a period specified in the --delay argument.
  • RPS: How many rows the server reads per second during a period specified in the --delay argument.
  • MiB/s: How many mebibytes the server reads per second during a period specified in the --delay argument.
  • result RPS: How many rows placed by the server to the result of a query per second during a period specified in the --delay argument.
  • result MiB/s. How many mebibytes placed by the server to the result of a query per second during a period specified in the --delay argument.

• Percentiles of queries execution time.

Comparison Mode

clickhouse-benchmark can compare performances for two running ClickHouse servers.

To use the comparison mode, specify endpoints of both servers by two pairs of --host, --port keys. Keys matched together by position in arguments list, the first --host is matched with the first --port and so on. clickhouse-benchmark establishes connections to both servers, then sends queries. Each query addressed to a randomly selected server. The results are shown in a table.

Example

$ echo "SELECT * FROM system.numbers LIMIT 10000000 OFFSET 10000000" | clickhouse-benchmark --host=localhost --port=9001 --host=localhost --port=9000 -i 10

Loaded 1 queries.

Queries executed: 5.

localhost:9001, queries 2, QPS: 3.764, RPS: 75446929.370, MiB/s: 575.614, result RPS: 37639659.982, result MiB/s: 287.168.
localhost:9000, queries 3, QPS: 3.815, RPS: 76466659.385, MiB/s: 583.394, result RPS: 38148392.297, result MiB/s: 291.049.

0.000%          0.258 sec.      0.250 sec.
10.000%         0.258 sec.      0.250 sec.
20.000%         0.258 sec.      0.250 sec.
30.000%         0.258 sec.      0.267 sec.
40.000%         0.258 sec.      0.267 sec.
50.000%         0.273 sec.      0.267 sec.
60.000%         0.273 sec.      0.267 sec.
70.000%         0.273 sec.      0.267 sec.
80.000%         0.273 sec.      0.269 sec.
90.000%         0.273 sec.      0.269 sec.
95.000%         0.273 sec.      0.269 sec.
99.000%         0.273 sec.      0.269 sec.
99.900%         0.273 sec.      0.269 sec.
99.990%         0.273 sec.      0.269 sec.

No difference proven at 99.5% confidence