pyquickbench.run_benchmark#

run_benchmark(all_args, all_funs, *, mode='timings', setup=<function default_setup>, n_repeat=1, nproc=None, pooltype=None, time_per_test=0.2, filename=None, ForceBenchmark=False, PreventBenchmark=False, allow_pickle=False, StopOnExcept=False, ShowProgress=False, WarmUp=False, MonotonicAxes=[], timeout=1.0, show=False, return_array_descriptor=False, **plot_kwargs)[source]#

Runs a full benchmark.

Parameters:

  • all_args (dict | iterable) – Describes the arguments to be given to the functions in the benchmark.

  • all_funs (dict | iterable) – Functions to be benchmarked.

  • mode (str, optional) –

    Benchmark mode, i.e. target of the benchmark.

    See Plotting scalar values for usage example.

    Possible values: "timings" or "scalar_output". By default "timings".

  • setup (callable, optional) –

    Function that prepares the inputs for the functions to be benchmarked.

    See Preparing inputs for usage example.

    By default lambda n: {pyquickbench.default_ax_name: n}.

  • n_repeat (int, optional) –

    Number of times to repeat the benchmark for variability studies.

    By default 1.

  • nproc (int, optional) –

    Number of workers in concurrent.futures.Executor.

    By default multiprocessing.cpu_count().

  • pooltype (str, optional) –

    Type of concurrent.futures.Executor.

    Possible values: "phony", "thread" or "process".

    By default "phony".

  • time_per_test (float, optional) –

    Minimum time in seconds for benchmark in "timings" mode.

    By default 0.2.

  • filename (str | None, optional) –

    Filename for results caching.

    See Caching benchmarks for usage example.

    Possible file extensions : *.npy or *.npz.

    By default None.

  • ForceBenchmark (bool, optional) – Whether to disregard existing cache and force a full re-run, by default False.

  • PreventBenchmark (bool, optional) – Whether to prevent a possibly lengthy full re-run, by default False.

  • allow_pickle (bool, optional) – Whether to allow pickling of data when loading benchmarks from disk. By default, False.

  • StopOnExcept (bool, optional) – Whether to interrupt the benchmark if exceptions are thrown, by default False.

  • ShowProgress (bool, optional) – Whether to show a progress bar in the CLI during benchmark, by default False.

  • WarmUp (bool, optional) –

    Whether to run the function once without measurement. Can help with jit compilation caching for instance.

    By default False.

  • MonotonicAxes (list, optional) –

    List of argument names for which timings are expected to get longer and longer.

    By default [].

  • timeout (float, optional) –

    Time in seconds after which a timing is considered too long.

    When a timing reaches this value, all longer timings (as detected by MonotonicAxes) are cancelled.

    By default 1..

  • show (bool, optional) – Whether to issue a call to pyquickbench.plot_benchmark() after the benchmark is run, by default False.

  • return_array_descriptor (bool, optional) –

    Whether to exit the function without performing the benchmark and only return a description of the benchmark result array that would be generated.

    See Running benchmarks manually for usage example.

    By default False.

  • **plot_kwargs – Arguments to pass on to pyquickbench.plot_benchmark() after the benchmark is run.

Returns:

Benchmark results.

Return type:

numpy.ndarray [numpy.float64] | None