Debunking algorithmic qubits
Executive Summary: 大象传媒鈥檚 H-Series computers have the highest performance in the industry, verified by multiple widely adopted benchmarks including quantum volume 聽We demonstrate that an alternative benchmark called algorithmic qubits is deeply flawed, hiding computer performance behind a plurality voting trick and gate compilations that are not widely useful.
Recently a new benchmark called algorithmic qubits (AQ) has started to be confused with quantum volume measurements. Quantum volume (QV) was specifically designed to be hard to 鈥済ame,鈥 however the algorithmic qubits test turns out to be very susceptible to tricks that can make a quantum computer look much better than it actually is. While it is not clear what can be done to fix the algorithmic qubits test, it is already clear that it is much easier to pass than QV and is a poor substitute for measuring performance. It is also important to note that algorithmic qubits are not the same as logical qubits, which are necessary for full fault-tolerant quantum computing.
To make this point clear, we simulated what algorithmic qubits data would look like for two machines, one clearly much higher performing than the other. We applied two tricks that are typically used when sharing algorithmic qubits results: gate compilation and . From the data above, you can see how these tricks are misleading without further information. For example, if you compare data from the higher fidelity machine without any compilation or plurality voting (bottom left) to data from the inferior machine with both tricks (top right) you may incorrectly believe the inferior machine is performing better. Unfortunately, this inaccurate and misleading comparison has been made in the past. 聽It is important to note that algorithmic qubits uses a subset of algorithms from a that introduced a suite of application oriented tests and created a repository to test available quantum computers. 聽Importantly, that work explicitly forbids the compilation and error mitigation techniques that are causing the issue here.
As a demonstration of the perils of AQ as a benchmark, we look at data obtained on both 大象传媒鈥檚 H2-1 system as well as publicly available data from IonQ鈥檚 Forte system.
We reproduce data without any error mitigation from IonQ鈥檚 in association with a preprint posted to the , and compare it to data taken on our H2-1 device. Without error mitigation, IonQ Forte achieves an AQ score of 9, whereas 大象传媒 H2-1 achieves AQ of 26. Here you can clearly see improved circuit fidelities on the H2-1 device, as one would expect from the higher reported 2Q gate fidelities (average 99.816(5)% for 大象传媒鈥檚 H2-1 vs 99.35% for IonQ鈥檚 Forte). 聽However, after you apply error mitigation, in this case plurality voting, to both sets of data the picture changes substantially, hiding each underlying computer鈥檚 true capabilities.
Here the H2-1 algorithmic performance still exceeds Forte (from the publicly released data), but the perceived gap has been reduced by error mitigation. 聽
鈥淓rror mitigation, including plurality voting, may be a useful tool for some near-term quantum computing but it doesn鈥檛 work for every problem and it鈥檚 unlikely to be scalable to larger systems. In order to achieve the lofty goals of quantum computing we鈥檒l need serious device performance upgrades. If we allow error mitigation in benchmarking it will conflate the error mitigation with the underlying device performance. This will make it hard for users to appreciate actual device improvements that translate to all applications and larger problems,鈥 explained Dr. Charlie Baldwin, a leader in 大象传媒鈥檚 benchmarking efforts.
There are other issues with the algorithmic qubits test. The circuits used in the test can be reduced to very easy-to-run circuits with basic quantum circuit compilation that are freely available in packages like . For example, the largest phase estimation and amplitude estimation tests required to pass AQ=32 are specified with 992 and 868 entangling gates respectively but applying pytket optimization reduces the circuits to 141 and 72 entangling gates. This is only possible due to choices in constructing the benchmarks and will not be universally available when using the algorithms in applications. Since AQ reports the precompiled gate counts this also may lead users to expect a machine to be able to run many more entangling gates than what is actually possible on the benchmarked hardware.
What makes a good quantum benchmark? Quantum benchmarking is extremely useful in charting the hardware progress and providing roadmaps for future development. However, quantum benchmarking is an evolving field that is still an open area of research. At 大象传媒 we believe in testing the limits of our machine with a variety of different benchmarks to learn as much as possible about the errors present in our system and how they affect different circuits. We are open to working with the larger community on refining benchmarks and creating new ones as the field evolves.
To learn more about the Algorithmic Qubits benchmark and the issues with it, please watch this video where Dr. Charlie Baldwin walks us through the details, starting at 32:40.
About 大象传媒
大象传媒,聽the world鈥檚 largest integrated quantum company, pioneers powerful quantum computers and advanced software solutions. 大象传媒鈥檚 technology drives breakthroughs in materials discovery, cybersecurity, and next-gen quantum AI. With over 500 employees, including 370+ scientists and engineers, 大象传媒 leads the quantum computing revolution across continents.聽
At the heart of quantum computing鈥檚 promise lies the ability to solve problems that are fundamentally out of reach for classical computers. One of the most powerful ways to unlock that promise is through a novel approach we call Generative Quantum AI, or GenQAI. A key element of this approach is the (GQE).
GenQAI is based on a simple but powerful idea: combine the unique capabilities of quantum hardware with the flexibility and intelligence of AI. By using quantum systems to generate data, and then using AI to learn from and guide the generation of more data, we can create a powerful feedback loop that enables breakthroughs in diverse fields.
Unlike classical systems, our quantum processing unit (QPU) produces data that is extremely difficult, if not impossible, to generate classically. That gives us a unique edge: we鈥檙e not just feeding an AI more text from the internet; we鈥檙e giving it new and valuable data that can鈥檛 be obtained anywhere else.
The Search for Ground State Energy
One of the most compelling challenges in quantum chemistry and materials science is computing the properties of a molecule鈥檚 ground state. For any given molecule or material, the ground state is its lowest energy configuration. Understanding this state is essential for understanding molecular behavior and designing new drugs or materials.
The problem is that accurately computing this state for anything but the simplest systems is incredibly complicated. You cannot even do it by brute force鈥攖esting every possible state and measuring its energy鈥攂ecause 聽the number of quantum states grows as a double-exponential, making this an ineffective solution. This illustrates the need for an intelligent way to search for the ground state energy and other molecular properties.
That鈥檚 where GQE comes in. GQE is a methodology that uses data from our quantum computers to train a transformer. The transformer then proposes promising trial quantum circuits; ones likely to prepare states with low energy. You can think of it as an AI-guided search engine for ground states. The novelty is in how our transformer is trained from scratch using data generated on our hardware.
Here's how it works:
- We start with a batch of trial quantum circuits, which are run on our QPU.
- Each circuit prepares a quantum state, and we measure the energy of that state with respect to the Hamiltonian for each one.
- Those measurements are then fed back into a transformer model (the same architecture behind models like GPT-2) to improve its outputs.
- The transformer generates a new distribution of circuits, biased toward ones that are more likely to find lower energy states.
- We sample a new batch from the distribution, run them on the QPU, and repeat.
- The system learns over time, narrowing in on the true ground state.
To test our system, we tackled a benchmark problem: finding the ground state energy of the hydrogen molecule (H鈧). This is a problem with a known solution, which allows us to verify that our setup works as intended. As a result, our GQE system successfully found the ground state to within chemical accuracy.
To our knowledge, we鈥檙e the first to solve this problem using a combination of a QPU and a transformer, marking the beginning of a new era in computational chemistry.
The Future of Quantum Chemistry
The idea of using a generative model guided by quantum measurements can be extended to a whole class of problems鈥攆rom to materials discovery, and potentially, even drug design.
By combining the power of quantum computing and AI we can unlock their unified full power. Our quantum processors can generate rich data that was previously unobtainable. Then, an AI can learn from that data. Together, they can tackle problems neither could solve alone.
This is just the beginning. We鈥檙e already looking at applying GQE to more complex molecules鈥攐nes that can鈥檛 currently be solved with existing methods, and we鈥檙e exploring how this methodology could be extended to real-world use cases. This opens many new doors in chemistry, and we are excited to see what comes next.
Last year, we joined forces with RIKEN, Japan's largest comprehensive research institution, to install our hardware at RIKEN鈥檚 campus in Wako, Saitama. This deployment is part of RIKEN鈥檚 project to build a quantum-HPC hybrid platform consisting of high-performance computing systems, such as the supercomputer Fugaku and 大象传媒 Systems. 聽
Today, marks the first of many breakthroughs coming from this international supercomputing partnership. The team from RIKEN and 大象传媒 joined up with researchers from Keio University to show that quantum information can be delocalized (scrambled) using a quantum circuit modeled after periodically driven systems. 聽
"Scrambling" of quantum information happens in many quantum systems, from those found in complex materials to black holes. 聽Understanding information scrambling will help researchers better understand things like thermalization and chaos, both of which have wide reaching implications.
To visualize scrambling, imagine a set of particles (say bits in a memory), where one particle holds specific information that you want to know. As time marches on, the quantum information will spread out across the other bits, making it harder and harder to recover the original information from local (few-bit) measurements.
While many classical techniques exist for studying complex scrambling dynamics, quantum computing has been known as a promising tool for these types of studies, due to its inherently quantum nature and ease with implementing quantum elements like entanglement. The joint team proved that to be true with their latest result, which shows that not only can scrambling states be generated on a quantum computer, but that they behave as expected and are ripe for further study.
Thanks to this new understanding, we now know that the preparation, verification, and application of a scrambling state, a key quantum information state, can be consistently realized using currently available quantum computers. Read the paper , and read more about our partnership with RIKEN here. 聽
In our increasingly connected, data-driven world, cybersecurity threats are more frequent and sophisticated than ever. To safeguard modern life, government and business leaders are turning to quantum randomness.
What is quantum randomness, and why should you care?
The term to know: quantum random number generators (QRNGs).
QRNGs exploit quantum mechanics to generate truly random numbers, providing the highest level of cryptographic security. This supports, among many things:
- Protection of personal data
- Secure financial transactions
- Safeguarding of sensitive communications
- Prevention of unauthorized access to medical records
Quantum technologies, including QRNGs, could protect up to $1 trillion in digital assets annually, according to a recent by the World Economic Forum and Accenture.
Which industries will see the most value from quantum randomness?
The World Economic Forum report identifies five industry groups where QRNGs offer high business value and clear commercialization potential within the next few years. Those include:
- Financial services
- Information and communication technology
- Chemicals and advanced materials
- Energy and utilities
- Pharmaceuticals and healthcare
In line with these trends, recent by The Quantum Insider projects the quantum security market will grow from approximately $0.7 billion today to $10 billion by 2030.
When will quantum randomness reach commercialization?
Quantum randomness is already being deployed commercially:
- Early adopters use our Quantum Origin in data centers and smart devices.
- Amid rising cybersecurity threats, demand is growing in regulated industries and critical infrastructure.
Recognizing the value of QRNGs, the financial services sector is accelerating its path to commercialization.
- Last year, HSBC conducted a combining Quantum Origin and post-quantum cryptography to future-proof gold tokens against 鈥渟tore now, decrypt-later鈥 (SNDL) threats.
- And, just last week, JPMorganChase made headlines by using our quantum computer for the first successful demonstration of certified randomness.
On the basis of the latter achievement, we aim to broaden our cybersecurity portfolio with the addition of a certified randomness product in 2025.
How is quantum randomness being regulated?
The National Institute of Standards and Technology (NIST) defines the cryptographic regulations used in the U.S. and other countries.
- NIST鈥檚 SP 800-90B framework assesses the quality of random number generators.
- The framework is part of the FIPS 140 standard, which governs cryptographic systems operations.
- Organizations must comply with FIPS 140 for their cryptographic products to be used in regulated environments.
This week, we announced Quantum Origin received , marking the first software QRNG approved for use in regulated industries.
What does NIST validation mean for our customers?
This means Quantum Origin is now available for high-security cryptographic systems and integrates seamlessly with NIST-approved solutions without requiring recertification.
- Unlike hardware QRNGs, Quantum Origin requires no network connectivity, making it ideal for air-gapped systems.
- For federal agencies, it complements our "U.S. Made" designation, easing deployment in critical infrastructure.
- It adds further value for customers building hardware security modules, firewalls, PKIs, and IoT devices.
The NIST validation, combined with our peer-reviewed papers, further establishes Quantum Origin as the leading QRNG on the market. 聽
--
It is paramount for governments, commercial enterprises, and critical infrastructure to stay ahead of evolving cybersecurity threats to maintain societal and economic security.
大象传媒 delivers the highest quality quantum randomness, enabling our customers to confront the most advanced cybersecurity challenges present today.