macでインフォマティクス

macでインフォマティクス

HTS (NGS) 関連のインフォマティクス情報についてまとめています。

コマンドの平均実行時間を計測するRustのコマンド hyperfine

2020 5/13 コマンド追記、タイトル修正

2020 6/27 タイトル再修正

 

hyperfineはRustのベンチマークツール。コマンドを自動で複数回実行し、結果をまとめてくれる。

 

インストール

GIthub

cargoで導入できる(cargo導入)。

cargo install hyperfine

> hyperfine

$ hyperfine -h

hyperfine 1.10.0

A command-line benchmarking tool.

 

USAGE:

    hyperfine [OPTIONS] <command>...

 

OPTIONS:

    -w, --warmup <NUM>

            Perform NUM warmup runs before the actual benchmark. This can be used to fill

            (disk) caches for I/O-heavy programs.

    -m, --min-runs <NUM>

            Perform at least NUM runs for each command (default: 10).

 

    -M, --max-runs <NUM>

            Perform at most NUM runs for each command. By default, there is no limit.

 

    -r, --runs <NUM>

            Perform exactly NUM runs for each command. If this option is not specified,

            hyperfine automatically determines the number of runs.

    -p, --prepare <CMD>...

            Execute CMD before each timing run. This is useful for clearing disk caches,

            for example.

            The --prepare option can be specified once for all commands or multiple times,

            once for each command. In the latter case, each preparation command will be

            run prior to the corresponding benchmark command.

    -c, --cleanup <CMD>

            Execute CMD after the completion of all benchmarking runs for each individual

            command to be benchmarked. This is useful if the commands to be benchmarked

            produce artifacts that need to be cleaned up.

    -P, --parameter-scan <VAR> <MIN> <MAX>

            Perform benchmark runs for each value in the range MIN..MAX. Replaces the

            string '{VAR}' in each command by the current parameter value.

            

              Example:  hyperfine -P threads 1 8 'make -j {threads}'

            

            This performs benchmarks for 'make -j 1', 'make -j 2', …, 'make -j 8'.

    -D, --parameter-step-size <DELTA>

            This argument requires --parameter-scan to be specified as well. Traverse the

            range MIN..MAX in steps of DELTA.

            

              Example:  hyperfine -P delay 0.3 0.7 -D 0.2 'sleep {delay}'

            

            This performs benchmarks for 'sleep 0.3', 'sleep 0.5' and 'sleep 0.7'.

    -L, --parameter-list <VAR> <VALUES>

            Perform benchmark runs for each value in the comma-separated list VALUES.

            Replaces the string '{VAR}' in each command by the current parameter value.

            

            Example:  hyperfine -L compiler gcc,clang '{compiler} -O2 main.cpp'

            

            This performs benchmarks for 'gcc -O2 main.cpp' and 'clang -O2 main.cpp'.

    -s, --style <TYPE>

            Set output style type (default: auto). Set this to 'basic' to disable output

            coloring and interactive elements. Set it to 'full' to enable all effects even

            if no interactive terminal was detected. Set this to 'nocolor' to keep the

            interactive output without any colors. Set this to 'color' to keep the colors

            without any interactive output. Set this to 'none' to disable all the output

            of the tool.

    -S, --shell <SHELL>

            Set the shell to use for executing benchmarked commands.

 

    -i, --ignore-failure

            Ignore non-zero exit codes of the benchmarked programs.

 

    -u, --time-unit <UNIT>

            Set the time unit to be used. Possible values: millisecond, second.

 

        --export-asciidoc <FILE>

            Export the timing summary statistics as an AsciiDoc table to the given FILE.

 

        --export-csv <FILE>

            Export the timing summary statistics as CSV to the given FILE. If you need the

            timing results for each individual run, use the JSON export format.

        --export-json <FILE>

            Export the timing summary statistics and timings of individual runs as JSON to

            the given FILE.

        --export-markdown <FILE>

            Export the timing summary statistics as a Markdown table to the given FILE.

 

        --show-output

            Print the stdout and stderr of the benchmark instead of suppressing it. This

            will increase the time it takes for benchmarks to run, so it should only be

            used for debugging purposes or when trying to benchmark output speed.

    -h, --help

            Print this help message.

 

    -V, --version

            Show version information.

 

 

ARGS:

    <command>...

            Command to benchmark

 

 

 

 実行方法

コマンドを指定する。自動でおよそ10回実行する。

hyperfine 'sleep 1s'

実行中、回数ごとにゲージが増加して進捗がわかるようになっている。

$ hyperfine 'sleep 1s'

Benchmark #1: sleep 1s

 ⠦ Current estimate: 1.006 s      ██████████████████████████████░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░ ETA 00:00:08

 

一秒スリープを10回行っているので、トータルで10秒超の時間がかかる。 終わると平均タイム、最大タイム、最小タイムがまとめられる。

Benchmark #1: sleep 1s

  Time (mean ± σ):      1.006 s ±  0.002 s    [User: 1.1 ms, System: 1.3 ms]

  Range (minmax):    1.003 s  1.009 s    10 runs

 

 

複数のコマンドを同時に測定できる。

hyperfine --min-runs 5 'sleep 0.2' 'sleep 3.2'

複数指定した場合、個別のタイムと合計タイムが個別にまとめられる。

$ hyperfine --min-runs 2 'sleep 0.2' 'sleep 3.2'

Benchmark #1: sleep 0.2

  Time (mean ± σ):     206.9 ms ±   1.9 ms    [User: 1.1 ms, System: 1.4 ms]

  Range (minmax):   203.7 ms209.7 ms    14 runs

 

Benchmark #2: sleep 3.2

  Time (mean ± σ):      3.207 s ±  0.002 s    [User: 1.7 ms, System: 1.7 ms]

  Range (minmax):    3.206 s  3.209 s    2 runs

 

Summary

  'sleep 0.2' ran

   15.50 ± 0.15 times faster than 'sleep 3.2'

 

CSV出力

hyperfine 'sleep 1s' --export-csv result.csv

JSON出力

hyperfine 'sleep 1s' --export-json result.json

markdown出力

hyperfine 'sleep 1s' --export-markdown result.md

  

デフォルトでは10回程度実行するが、3回に変更。

hyperfine -r 3 'minimap2 -t 8 -ax sr ref.fa pair_*fq.gz |samtools sort -@ 8 -O BAM - > short.bam'
  • -r   Perform exactly NUM runs for each command. If this option is not specified, hyperfine automatically determines the number of runs.

 

引用 

https://github.com/sharkdp/hyperfine