Intel's new iBOT utility triggers benchmark validity warnings for Core Ultra 200S Plus and 300 series CPUs in Geekbench

Geekbench has announced it will display a temporary warning on all benchmark results generated by CPUs supporting Intel's new Binary Optimization Tool (iBOT). The move comes amid growing concerns that the utility modifies instruction sequences in ways that are currently undocumented, preventing a fair apples-to-apples comparison between hardware manufacturers.

Most software is compiled to run on a variety of hardware. Intel argues that this often leaves performance on the table since the code isn't tailored to its specific architecture. Intel Binary Optimization Utility, or iBOT, is a user-mode performance enhancement utility launched by Intel for Core Ultra 200S Plus (Arrow Lake Refresh) and select 300 series (Panther Lake) CPUs to alleviate this problem.

Intel profiles select applications at its labs at the microarchitectural level to find where the compiled code is not performing optimally. If it finds instruction sequences that are not optimized, it reorganizes them post-link to better employ Intel's unique caches, branch predictors, and execution units. Notably, the developer does not have to get involved, and Intel explicitly says no work or instructions are skipped; just reorganized.

Since Geekbench is a utility primarily used for comparing hardware across different manufacturers, it expressed concerns over the undocumented nature of iBOT.

"Since the tool modifies the benchmark, and it is unclear to both Primate Labs and the general public how these changes occur, results generated with the tool are not comparable to results generated without it," says the developer.

As Geekbench currently lacks the telemetry needed to detect these optimizations, it is adopting a zero-trust posture. Resultantly, a cautionary flag will be applied to the results of all CPUs compatible with iBOT, regardless of whether the utility was active during the benchmark.

For context, iBOT has increased overall Intel CPU performance on Geekbench by up to 8%. Likewise, we recently saw a sample of the unreleased Core Ultra 9 290K Plus toppling the Geekbench single-core benchmark for x86 processors.

While Intel has provided a high-level overview of iBOT, Geekbench is seeking deeper and more technical insight. The goal is to make sure these optimizations are general-purpose and well-documented. Likewise, while Intel manually validates and approves each game and application, iBOT's on-the-fly modification of code also presents a challenge for kernel-level anti-cheat software.

If an Intel CPU hits a 3,700 single-core score using iBOT, while an AMD chip hits 3,600 natively, the numbers on the screen suggest Intel is the winner. However, this no longer remains a fair comparison. One chip is running the 'standard' test, while the other is running a 'tailored' version of that test.

2

Until these optimizations are applied to every application a user might run, these scores only represent the CPU's peak potential, rather than real-world performance. For now, the ball is in Intel's court. The company's optimization tool has drawn skepticism from Geekbench, and transparency is the only way forward.

Original Source: Geekbench