Methodology — how every number is made

The rules

Same-binary baselines

Every cross-library ratio comes from a single run of a single harness that measures both sides on the same fixtures, in the same process, back to back — bench_matrix for the XML C races and the yeptris matrix for YAML (which links the reference engines statically). Where a lane bench pairs with a reference twin (XPath lanes, XSLT fixtures, the HTML parse lane), the twin generates a byte-identical fixture and follows the identical timing protocol.

Fresh Release builds, identical flags

Each campaign builds from a fresh build directory withCMAKE_BUILD_TYPE=Release and the repo’s default LTO policy; A/B comparisons never mix build directories. Version strings are recorded from what was actually built and run — including-dev trees — never from marketing version files.

Per-repo timing conventions

Each repo’s harness sets its own statistic, and the result file carries it: the C matrices report min-of-iterations, the XSLT scorecard best of 9 (its transform fixture keeps the older mean-of-400 protocol), the lane benches mean-of-1000, the Ruby matrix mean-per-iteration, and the yeptris-vs-ryml rows interleaved medians with a fair-order referee. Means are avoided where allocator and GC churn would smear the signal.

Machine-relative ratios

Ratios are the currency, and they are always same-machine:ratio = how many times faster the leptris side is on the same operation — latency ratios divide the competitor’s time by ours, throughput ratios divide our rate by theirs. A ratio below 1 is a loss and stays visible. Absolute numbers are per-machine context only; compare CI artifacts across commits on the same runner class, never against a dev machine.

How matrix cells summarize a race

Each matrix cell summarizes every measured operation for one technology, one language and one competitor. When all operations are ahead, the cell is shown in the positive color and displays the smallest margin, so the displayed value is a lower bound for the whole race. When a race contains measured deficits, the cell displays the weakest and the strongest result together, and hovering over the cell lists the operations that remain behind. A deficit can therefore never be hidden by a stronger result elsewhere in the same race, and a favorable cell can never overstate the weakest measured operation.

Artifact marking

A row carried from a published artifact or ledger instead of a fresh run is marked † and its file setsartifact: true, with the carrying harness linked. Stale numbers are replaced, not averaged: when a campaign re-runs, the new file supersedes the old one.

No unmeasured claims

If a race has no committed cross-library harness, it gets no numbers — the validation/XQuery section states capability and conformance gates instead. A number without a published harness behind it does not appear on the pages.