2020 5/13 コマンド追記、タイトル修正
2020 6/27 タイトル再修正
hyperfineはRustのベンチマークツール。コマンドを自動で複数回実行し、結果をまとめてくれる。
インストール
cargoで導入できる(cargo導入)。
cargo install hyperfine
> hyperfine
$ hyperfine -h
hyperfine 1.10.0
A command-line benchmarking tool.
USAGE:
hyperfine [OPTIONS] <command>...
OPTIONS:
-w, --warmup <NUM>
Perform NUM warmup runs before the actual benchmark. This can be used to fill
(disk) caches for I/O-heavy programs.
-m, --min-runs <NUM>
Perform at least NUM runs for each command (default: 10).
-M, --max-runs <NUM>
Perform at most NUM runs for each command. By default, there is no limit.
-r, --runs <NUM>
Perform exactly NUM runs for each command. If this option is not specified,
hyperfine automatically determines the number of runs.
-p, --prepare <CMD>...
Execute CMD before each timing run. This is useful for clearing disk caches,
for example.
The --prepare option can be specified once for all commands or multiple times,
once for each command. In the latter case, each preparation command will be
run prior to the corresponding benchmark command.
-c, --cleanup <CMD>
Execute CMD after the completion of all benchmarking runs for each individual
command to be benchmarked. This is useful if the commands to be benchmarked
produce artifacts that need to be cleaned up.
-P, --parameter-scan <VAR> <MIN> <MAX>
Perform benchmark runs for each value in the range MIN..MAX. Replaces the
string '{VAR}' in each command by the current parameter value.
Example: hyperfine -P threads 1 8 'make -j {threads}'
This performs benchmarks for 'make -j 1', 'make -j 2', …, 'make -j 8'.
-D, --parameter-step-size <DELTA>
This argument requires --parameter-scan to be specified as well. Traverse the
range MIN..MAX in steps of DELTA.
Example: hyperfine -P delay 0.3 0.7 -D 0.2 'sleep {delay}'
This performs benchmarks for 'sleep 0.3', 'sleep 0.5' and 'sleep 0.7'.
-L, --parameter-list <VAR> <VALUES>
Perform benchmark runs for each value in the comma-separated list VALUES.
Replaces the string '{VAR}' in each command by the current parameter value.
Example: hyperfine -L compiler gcc,clang '{compiler} -O2 main.cpp'
This performs benchmarks for 'gcc -O2 main.cpp' and 'clang -O2 main.cpp'.
-s, --style <TYPE>
Set output style type (default: auto). Set this to 'basic' to disable output
coloring and interactive elements. Set it to 'full' to enable all effects even
if no interactive terminal was detected. Set this to 'nocolor' to keep the
interactive output without any colors. Set this to 'color' to keep the colors
without any interactive output. Set this to 'none' to disable all the output
of the tool.
-S, --shell <SHELL>
Set the shell to use for executing benchmarked commands.
-i, --ignore-failure
Ignore non-zero exit codes of the benchmarked programs.
-u, --time-unit <UNIT>
Set the time unit to be used. Possible values: millisecond, second.
--export-asciidoc <FILE>
Export the timing summary statistics as an AsciiDoc table to the given FILE.
--export-csv <FILE>
Export the timing summary statistics as CSV to the given FILE. If you need the
timing results for each individual run, use the JSON export format.
--export-json <FILE>
Export the timing summary statistics and timings of individual runs as JSON to
the given FILE.
--export-markdown <FILE>
Export the timing summary statistics as a Markdown table to the given FILE.
--show-output
Print the stdout and stderr of the benchmark instead of suppressing it. This
will increase the time it takes for benchmarks to run, so it should only be
used for debugging purposes or when trying to benchmark output speed.
-h, --help
Print this help message.
-V, --version
Show version information.
ARGS:
<command>...
Command to benchmark
実行方法
コマンドを指定する。自動でおよそ10回実行する。
hyperfine 'sleep 1'
実行中、回数ごとにゲージが増加して進捗がわかるようになっている。
$ hyperfine 'sleep 1s'
Benchmark #1: sleep 1s
⠦ Current estimate: 1.006 s ██████████████████████████████░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░ ETA 00:00:08
一秒スリープを10回行っているので、トータルで10秒超の時間がかかる。 終わると平均タイム、最大タイム、最小タイムがまとめられる。
Benchmark #1: sleep 1s
Time (mean ± σ): 1.006 s ± 0.002 s [User: 1.1 ms, System: 1.3 ms]
Range (min … max): 1.003 s … 1.009 s 10 runs
複数のコマンドを同時に測定できる。
hyperfine --min-runs 5 'sleep 0.2' 'sleep 3.2'
複数指定した場合、個別のタイムと合計タイムが個別にまとめられる。
$ hyperfine --min-runs 2 'sleep 0.2' 'sleep 3.2'
Benchmark #1: sleep 0.2
Time (mean ± σ): 206.9 ms ± 1.9 ms [User: 1.1 ms, System: 1.4 ms]
Range (min … max): 203.7 ms … 209.7 ms 14 runs
Benchmark #2: sleep 3.2
Time (mean ± σ): 3.207 s ± 0.002 s [User: 1.7 ms, System: 1.7 ms]
Range (min … max): 3.206 s … 3.209 s 2 runs
Summary
'sleep 0.2' ran
15.50 ± 0.15 times faster than 'sleep 3.2'
CSV出力
hyperfine 'sleep 1' --export-csv result.csv
JSON出力
hyperfine 'sleep 1' --export-json result.json
markdown出力
hyperfine 'sleep 1' --export-markdown result.md
デフォルトでは10回程度実行するが、3回に変更。
hyperfine -r 3 'minimap2 -t 8 -ax sr ref.fa pair_*fq.gz |samtools sort -@ 8 -O BAM - > short.bam'
- -r Perform exactly NUM runs for each command. If this option is not specified, hyperfine automatically determines the number of runs.
引用