Analysis performance#

CPU profiling#

For profiling the CPU time of each function please select the iterative processor and then run python as:

python -m cProfile -o profiling/output.prof scripts/runner.py --cfg config.py --test -lf 10 -lc 10

Running on a few files should be enough to get stable results.

After getting the profiler output we analyze it with the Snakeviz library

snakeviz output.prof -s

and open on a browser the link shown by the program.

For memory profiling we use the memray library in single thread (iterative processor.)

python -m memray run -o profiling/memtest.bin scripts/runner.py --cfg config.py --test -lf 10 -lc 10

the output can be visualized in many ways. One of the most useful is the flamegraph:

memray flamegraph profiling/memtest.bin

then open the output .html file in you browser to explore the peak memory allocation.

Alternatively the process can be monitored live during execution by doing:

memray run --live scripts/runner.py --cfg config/config.py --test -lf 10 -lc 10