Micro-benchmark inference

[jainapurva]

This is the first PR, in the benchmarking effort. It provides the outline to setup inference microbenchmarking for quant api's in torchao.

The different inputs like quantization techniques, matrix sizes, compile, sparsity etc, will be given as input to the python script. The options are re-defined in the scipt for quantization techniques, and a developer can add new quant technique. The script will generate a csv with performance numbers, and that'll be used to plot charts and as an input to dashboard . The script performs the following tasks:

• Take input as .yml
• Performs benchmarking for quantize_ APIs eval time for configurations
• Record all the config params and their respective time in csv file.
• Test cases

Future PRs will include more config options and process the generated results.

Run command:

python benchmarks/microbenchmarks/benchmark_runner.py --config benchmarks/microbenchmarks/benchmark_config.yml

Output will be stored in

benchmarks/microbenchmarks/results/results.csv

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/ao/1759

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure

As of commit 22b3ddd with merge base bc4f51d ():

NEW FAILURE - The following job has failed:

• Code Analysis with Ruff / build (3.9) (gh)
  benchmarks/microbenchmarks/benchmark_inference.py:47:8: F632 [*] Use != to compare constant literals

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[HDCharles]

nit: I tend to prefer to consolidate multiple variables into 1, like compile vs compile_mode is kind of redundant when you could just have

compile: "max-autotune" or compile: "false"

[HDCharles]

lgtm