Conference150th anniversary of the Metre Convention — From Units to the Universe
Year2025
LocationPalais des congrès de Versailles, Versailles, France
PublisherBureau International des Poids et Mesures (BIPM)
LanguageEN
ABSTRACT
Quantum computers have the potential to provide an advantage over classical computers in a number of areas. Numerous metrics to benchmark the performance of quantum computers, ranging from their individual hardware components to entire applications, have been proposed over the years. Objective comparisons are further hampered in practice as different variations of the same metric are used, and the data disclosed together with a reported metric value is often not sufficient to reproduce the measurements. In this poster, we present our recent article, where we address these challenges by providing a review of metrics and benchmarks for quantum computers [1], including 1) a comprehensive collection of benchmarks allowing holistic comparisons of quantum computers, 2) a consistent format of the definitions across all metrics including a transparent description of the methodology and of the main assumptions and limitations, and 3) a reproducible approach by linking the metrics to open-source software used to evaluate them . We identify five areas where international standardization working groups could be established, namely: i) the identification and agreement on the categories of metrics that comprehensively benchmark device performance; ii) the identification and agreement on a set of well-established metrics that together comprehensively benchmark performance; iii) the identification of metrics specific to hardware platforms, including non-gate-based quantum computers; iv) inter-laboratory comparison studies to develop best practice guides for measurement methodology; and v) agreement on what data and software should be reported together with a metric value to ensure trust, transparency and reproducibility. We provide potential routes to advancing these areas. We expect that this work will accelerate the progress of quantum computing hardware towards quantum advantage.
[1] D. Lall et al., A Review and Collection of Metrics and Benchmarks for Quantum Computers: Definitions, Methodologies and Software, arXiv:2502.06717.
CITE THIS POSTER
Include:
Lall, D., Agarwal, A., Zhang, W., Lindoy, L., Lindström, T., Webster, S., Hall, S., Chancellor, N., Wallden, P., Garcia-Patron, R., Kashefi, E., Kendon, V., Pritchard, J., Rossi, A., Datta, A., Kapourniotis, T., Georgopoulos, K., & Rungger, I.. (2025). A Review and Collection of Metrics and Benchmarks for Quantum Computers: definitions, methodologies and software. 150th anniversary of the Metre Convention — From Units to the Universe. Palais des congrès de Versailles, Versailles, France. Bureau International des Poids et Mesures (BIPM). https://doi.org/10.59161/AZYE8097
Lall, Deep, et al.. “A Review and Collection of Metrics and Benchmarks for Quantum Computers: definitions, methodologies and software.”. 150th anniversary of the Metre Convention — From Units to the Universe, 2025, Palais des congrès de Versailles, Versailles, France, Bureau International des Poids et Mesures (BIPM). DOI: 10.59161/AZYE8097
Lall, Deep, et al.. 2025. “A Review and Collection of Metrics and Benchmarks for Quantum Computers: definitions, methodologies and software.”. 150th anniversary of the Metre Convention — From Units to the Universe, Palais des congrès de Versailles, Versailles, France. Bureau International des Poids et Mesures (BIPM). https://doi.org/10.59161/AZYE8097
@inproceedings{lall2025a_review_and_collect,
author = {Lall, Deep and Agarwal, Abhishek and Zhang, Weixi and Lindoy, Lachlan and Lindström, Tobias and Webster, Stephanie and Hall, Simon and Chancellor, Nicholas and Wallden, Petros and Garcia-Patron, Raul and Kashefi, Elham and Kendon, Viv and Pritchard, Jonathan and Rossi, Alessandro and Datta, Animesh and Kapourniotis, Theodoros and Georgopoulos, Konstantinos and Rungger, Ivan},
title = {A Review and Collection of Metrics and Benchmarks for Quantum Computers: definitions, methodologies and software},
booktitle = {150th anniversary of the Metre Convention — From Units to the Universe},
year = {2025},
address = {Palais des congrès de Versailles, Versailles, France},
publisher = {Bureau International des Poids et Mesures (BIPM)},
doi = {10.59161/AZYE8097},
url = {https://thebipm150.org/poster-page/10eugfe},
}
TY - CONF
AU - Lall, Deep
AU - Agarwal, Abhishek
AU - Zhang, Weixi
AU - Lindoy, Lachlan
AU - Lindström, Tobias
AU - Webster, Stephanie
AU - Hall, Simon
AU - Chancellor, Nicholas
AU - Wallden, Petros
AU - Garcia-Patron, Raul
AU - Kashefi, Elham
AU - Kendon, Viv
AU - Pritchard, Jonathan
AU - Rossi, Alessandro
AU - Datta, Animesh
AU - Kapourniotis, Theodoros
AU - Georgopoulos, Konstantinos
AU - Rungger, Ivan
TI - A Review and Collection of Metrics and Benchmarks for Quantum Computers: definitions, methodologies and software
T2 - 150th anniversary of the Metre Convention — From Units to the Universe
PB - Bureau International des Poids et Mesures (BIPM)
PY - 2025
CY - Palais des congrès de Versailles, Versailles, France
DO - 10.59161/AZYE8097
UR - https://thebipm150.org/poster-page/10eugfe
ER -