Measuring Performance

Rinch counts the work each frame does. Every document carries a fixed set of per-frame counters: elements cascaded, Parley layouts shaped, whole-document IFC setup passes, Taffy computes, full repaints and why, pixels repainted, hit tests, effect runs, and a few phase timings. The counters are always compiled in. An increment is one store into memory the document already owns, so reading them costs nothing you would notice.

Prefer the counters to timings when you want to prove that a change made something cheaper. Timings move with the machine’s load. A counter moves only when the code does different work: “a colour-only hover now runs 0 Taffy computes instead of 1” means the same thing on every host.

There are three ways to read them.

RINCH_PERF

Set RINCH_PERF (any value) and every completed frame prints one line to stderr: the phase times, then every counter that is not zero.

$ RINCH_PERF=1 cargo run --release -p ui-zoo-desktop
[PERF] frame 212: style 0.08ms layout 0.02ms paint 1.41ms present 0.35ms | style_resolves=1 elements_cascaded=3 ... repaint_partial=1 repainted_px=41236 surface_px=1382400 ...

The variable is read once, the first time a frame ends.

On the desktop, one redraw is one frame. On Android, one pass of the loop that tries to present a frame is one frame. On an embedded RinchContext, one update() plus the scene() call that follows it is one frame.

The perf_stats MCP tool

With the debug feature enabled, the perf_stats MCP tool (debug-protocol command perf_stats) returns this JSON:

{
  "frames": 212,
  "last_frame":    { "elements_cascaded": 3, "taffy_root_computes": 0, ... },
  "current_frame": { ... },
  "total":         { ... }
}

Each object holds every counter, keyed by the counter’s name. Pass reset: true to zero the counters after they are read. That gives you a way to measure a single interaction:

perf_stats(reset: true)       # start from zero
click(x: 120, y: 340)         # the interaction
wait_frame()
perf_stats()                  # `total` now covers only the click and its frames

This is the tool to use for measurements. The DevTools panel is a poor substitute, because while it is open it rebuilds its own tree view on every dirty frame of the app, and that work shows up in what you measure.

From code

• RinchDocument (rinch_dom): doc.tree.perf is a rinch_dom::perf::PerfCounters.

  • frame() returns the frame in progress.
  • last_frame() returns the last frame that ended.
  • total() returns everything counted since the last reset.
  • end_frame() closes the current frame.
  • reset() zeroes everything.

  A document that nothing drives never ends a frame, so frame() then covers everything since the last reset().

• RinchApp: last_frame_perf(), total_perf(), reset_perf() and end_perf_frame().

• RinchContext (embed): perf_last_frame(), perf_total() and reset_perf().

rinch::perf re-exports the module. Each counter is a Counter variant with a stable name(). FrameStats::to_json() and FrameStats::summary_line() produce the two output formats shown above.

#![allow(unused)]
fn main() {
use rinch_dom::perf::Counter;

doc.tree.perf.reset();
doc.set_attribute(row, "class", "row selected");
doc.resolve_layout(800.0, 600.0);
let frame = doc.tree.perf.end_frame();
assert_eq!(frame.get(Counter::TaffyRootComputes), 0, "a colour change must not lay out");
}

The counters

Some limits on what these numbers mean:

• The reactive counters are per thread. Two documents on one thread share them. A desktop window and its DevTools panel are two such documents, as are two embedded contexts. While DevTools is visible, every frame’s effect_runs and signal_notifies include its own work: the shell writes its frame-time and FPS signals on every redraw, and its panels re-run. Close DevTools before reading the reactive counters.
• A frame repaints what its damage names, and nothing else. The damage is the paint-dirty nodes (where each is now and where it was last painted), the rects of removed nodes, and the drag ghost’s and inspect highlight’s old and new rects. A box whose place in the paint order changed — it started or stopped being a stacking context (opacity crossing 1, a transform or filter starting or ending, position: fixed/sticky) or a positioned layer, or its z-index moved, by a restyle or by a transition or animation frame — names everything its subtree paints, since a z-index descendant that overflows it changes order against the boxes beside it. Such a frame repaints the whole window when the subtree cannot be measured: it holds more than 512 nodes, a position: fixed or sticky box, or an absolute box that escapes an overflow box at or under the one that flipped. A fade or a transform on a long list is therefore two full repaints (its first and last frame), where the frames between repaint the list’s own rect. A change that reaches pixels without a DomDocument write must name its node (NodeTree::mark_paint_dirty) or call RinchApp::mark_scene_dirty, which is counted as repaint_full_unattributed.
• The GPU backend always repaints in full. Every painted frame on that backend counts as repaint_full_gpu.
• The software-painter counters are the desktop and Android shells’ only. TinySkiaPainter counts into its own SkiaPainterStats (stats(), take_stats()), which the shell folds into the frame after each software paint. A test painting through paint_document directly reads the painter, not the document.
• What the software painter keeps between frames. Its glyph cache (cleared past about 16 MB of accounted bytes or 512 font/size runs); a premultiplied copy of each translucent cached image; and a pool of up to 8 clip masks (1 byte per surface pixel each) and 8 layer pixmaps (4 bytes per pixel each — 33 MB at 4K). The pool is trimmed after every software frame to the most any of the last 8 frames had open at once, so an app that idles right after a deeply nested frame keeps that frame’s buffers until it paints again.
• A screenshot is a frame. The debug screenshot command paints, and that paint shows up in the counters like any other frame. The capture after it adds nothing: on the software backend it reads the frame just presented back, as the GPU backend always has, rather than painting a second time (#364).

The committed baselines

crates/rinch-dom/tests/perf_counter_baselines.rs runs the paths that matter for responsiveness on a 40-row list and asserts today’s counter values:

• idle
• a colour-only hover, with and without a display: contents wrapper
• a colour-only class toggle
• an attribute no selector reads (nothing is cascaded)
• a container class a descendant rule depends on
• appending a row
• removing a row
• setting one text node’s content
• a 1px resize, with and without vw rules
• a transform animation tick

crates/rinch/src/app/perf_stats_tests.rs covers the counters only the shell fills: the full-repaint reasons, the dirty region, hit testing, and the reactive deltas. It also pins the pointer-move path: a warm move over a 500-row scroller runs one hit test that visits the four boxes under the pointer and builds no stacking sequence, a move with no onmousemove handler anywhere runs one hit test, and five drag moves queued before a frame lay out once. A press shares its hit test the same way: a click on an editor toolbar button runs one hit test for the press and one for the release (perf_regression_editor_tests::a_toolbar_bold_over_a_word, #908).

Each scenario in perf_counter_baselines.rs asserts the whole frame: every counter’s exact value, with any counter the baseline does not list expected to be zero. perf_stats_tests.rs is exact on the counters each test names and does not check the rest of the frame. That catches a path that got more expensive and, just as important, an increment that stopped counting, which an upper bound alone would read as a saving. A test in the same file checks that every counter rinch-dom owns fires on some path.

A performance fix is expected to change these numbers. Updating the baseline to the new values is the proof that the fix worked; say in the PR which counters moved and why. PERF_BASELINE_PRINT=1 prints each scenario’s frame in the form the baseline is written in.

The file’s #[ignore]d perf_scenario_timings prints wall-clock times for the same scenarios, and structural_timings_at_scale times an append, a removal and a toggle on 500 and 2000 rows, with and without a display: contents wrapper per row. Run them in release:

cargo test --release -p rinch-dom --test perf_counter_baselines -- --ignored --nocapture

The regression scenarios

Three more files hold scenarios under the same exact-count contract. They are new files rather than new rows in the two above so that parallel performance PRs do not collide in one file.

• crates/rinch-dom/tests/perf_regression_scenarios.rs has one scenario per increment site of every counter that has more than one site. shape_paint has three (a <select> label, an <input> value, and text with no cached layout — a constructed state since #904, when an inline-flex label stopped reaching it), shape_atomic_inline two and pseudo_element_passes two. ellipsis_builds is down to one site, an IFC root’s: the text-leaf rebuild was deleted in #982 once #998 had closed the last route to it, and the file keeps a scenario pinning that a flex container’s own text builds none (Chrome draws no “…” there). A per-counter check cannot see a deleted increment while a sibling site still fires; one document per site can.
• crates/rinch/src/app/perf_regression_tests.rs drives a real RinchApp the way the desktop loop does (AboutToWait, then the paint the loop asked for):
  • idle: a plain page, a focused <input>, a paused animation, a running Loader, and a Loader in a closed Drawer (which idles: the drawer’s closed rule pauses it, #912);
  • keyed for over 200 rows: move one row, insert one, remove one, and replace them all;
  • a wheel scroll over 500 rows;
  • ten queued Drag::absolute moves;
  • a theme toggle and a scale-factor change;
  • one frame per full-repaint reason: first_frame, resize, unattributed, region_too_large, restyle, theme, invalidated and gpu, plus repaint_none.
• crates/rinch/src/app/perf_regression_editor_tests.rs covers a 30-paragraph editor: idle while focused, one caret blink, typing a character, ArrowRight, Enter, Backspace across a block boundary, and the toolbar’s Bold.

Each scenario also asserts a positive control, a check that it did what its name says (a repaint happened, a row moved, the document grew by one character). Without it, an idle scenario whose app never mounted would pass as a perfect idle.

Some of these scenarios do work they should not, and they pin it as it is. A scenario like that says so in its doc comment and names the counter. A fix then shows up as a baseline that falls.

The baselines are the contract

Four files assert whole frames: perf_counter_baselines.rs and the three regression-scenario files. Each checks every non-timing counter exactly, and any counter the baseline does not list must be 0. perf_stats_tests.rs is exact only on the counters it names. Two counters are left out of the whole frame: the time_* counters, which are wall-clock, and rerender_events_queued in the shell scenarios, which is folded in from a process-wide atomic. queued_rerender_events_are_folded_into_the_frame pins that counter on its own, under a lock it shares with the one other test that writes the atomic.

The numbers must not depend on the host’s fonts. A declared line-height pins only the vertical axis. A width, a caret step or a repaint rect still comes from the glyph advances of whatever font answered, so a scenario whose numbers include horizontal text geometry sets its text in the bundled Inter:

• the shell scenarios build every app with perf_expect::new_app, which makes Inter the sans-serif, and set all text in sans-serif;
• the rinch-dom scenarios register Inter under their own family name.

Before this rule, the editor’s ArrowRight repainted 608 px on one machine and 640 px on CI (Noto Sans against DejaVu Sans). Check a new scenario under a second font set before you push: point FONTCONFIG_FILE at a config that maps sans-serif to DejaVu Sans, and run it both ways. The two runs must print identical frames.

A pinned finding names its issue. Some scenarios pin work that should not happen (issues #904–#914). Each one’s doc comment names its issue and says that a fix must lower the number. A fall there is the fix proving itself, not an increment that stopped counting.

This makes the baselines the performance contract for every change:

• A PR that changes a baseline must explain it in its body. For each scenario that moved, list the counters that moved, old and new, and why. The pull request template has a Performance section for this.
• A counter that rose is a path that got more expensive. Either fix it, or accept the cost and say why.
• A counter that fell is either a fix proving itself or an increment that stopped counting. The PR says which.
• Never loosen an assertion to make it pass. Do not turn an exact value into an upper bound, and do not remove a counter from a baseline. A value that honestly varies from run to run (the rotated Loader’s repainted area depends on the angle the clock lands on) gets a range with its derivation written next to it. Every other counter in that frame stays exact.

Updating a baseline

When an assertion fails, it prints the frame it saw in the form the baseline is written in, ready to paste:

editor: type one character: the frame differs from its recorded baseline.
  taffy_root_computes: 1 (baseline 2)
  ...
Actual frame:
            (StyleResolves, 2),
            (ElementsCascaded, 2),
            ...

PERF_BASELINE_PRINT=1 prints every scenario’s frame, passing or not. Use --test-threads=1 so the frames do not interleave:

PERF_BASELINE_PRINT=1 cargo test -p rinch-dom --test perf_regression_scenarios -- --nocapture --test-threads=1
PERF_BASELINE_PRINT=1 cargo test -p rinch --lib perf_regression -- --nocapture --test-threads=1
PERF_BASELINE_PRINT=1 cargo test -p rinch-dom --test perf_counter_baselines -- --nocapture

Run the shell scenarios with CI’s feature set as well. The theme-toggle and gpu scenarios compile only with theme and with gpu or embed:

cargo test -p rinch --features desktop,embed,theme,clipboard,debug --lib perf_regression

Reading a frame

The names are the ones RINCH_PERF prints and perf_stats returns. To find out why a number moved, run the same interaction in a live app:

1. Call perf_stats(reset: true).
2. Do the interaction.
3. Call wait_frame(), then perf_stats().

Or set RINCH_PERF=1 and read the one line each frame prints. Close DevTools before you read the reactive counters (see the limits above).

What a blurred text-shadow costs

A text-shadow with a blur radius is rasterised into a coverage mask, blurred and filled once per shadowed block of text (#980). The masks are kept between paints, so a repaint of shadowed text that has not changed, and is not cut by the edge of the window, a clip or the damage, pays only the fill — a scroll included, once each whole-pixel position a scroll can put each glyph at has been drawn (a sub-pixel scroll changes a line’s mask only where it moves a glyph to another pixel). A shadow the window, a clip or the damage cuts is rasterised again whenever that cut moves across it. The text_shadow_masks_rasterised counter says how many a frame rasterised. The first paint of a page of blurred shadows costs about twice what the same page with an unblurred shadow does, and more the larger the blur and the display scale: the mask is the text grown by one and a half blur radii on every side, in physical pixels. On a build with no software rasteriser (embed without software-renderer) a blurred shadow is drawn as up to 13 copies — fewer, or none, for a shadow of more than 153 glyphs, decided by that shadow alone.

CI regression job

Counters say what a frame did; they do not say how much it cost. The Perf workflow (.github/workflows/perf.yml) covers cost. On every pull request it counts the instructions each hot path executes. It counts them on the merge commit GitHub builds for the PR, and on that commit’s first parent, the tip of main it was merged onto. It fails when one count grew by more than a threshold. Every run re-bases on main as it is at that moment, so the Δ says what merging the PR would do now. Changes other PRs landed on main are not counted against it.

It counts instructions and not time because a hosted runner’s timings move with whatever else shares the machine, while an instruction count moves only when the code does. The counts come from Valgrind’s Callgrind, driven by Gungraun (the renamed iai-callgrind). The benchmarks live in crates/rinch-bench:

Each benchmark builds its fixture in a setup that Callgrind does not count. The setup also runs the operation once where that warms a cache, so what is counted is the second hover and not the first. All text is set in the bundled Inter-Regular.ttf, so no scenario measures or draws through the host’s fonts. That does not make a count portable: resize_1px is 111.2M instructions on the CI runner and 147.6M on a developer workstation, because the toolchain, glibc’s CPU-specific memcpy and the CPU all move it. Compare two counts only when one machine produced both. The CI job runs base and head on the same runner for this reason, and a local before/after is only comparable with another local run.

Reading the report

The job writes a table to its summary page and to one comment on the PR, which it edits in place on each push. A row looks like this (the numbers are from a deliberately slowed update_hover):

The columns are Callgrind’s Ir, instructions executed inside the measured operation. Two runs of the same binary agree to within about 0.03%. The remaining noise comes from hash-table probing, whose seeds vary per process. So any Δ larger than a fraction of a percent is the code.

The base run first uses the head’s copy of crates/rinch-bench, so both sides run the same scenarios and every Δ is the library alone. That copy does not build on the base when the PR adds a benchmark that calls an API the same PR introduces (#1036). The base then runs its own copy instead, restored with its own Cargo.lock (.github/scripts/perf_bench_base.sh). Every benchmark both copies define is still compared; one only the head defines shows as new, and one only the base defines as removed. Above the table, the report always says how many benchmarks were compared and names the ones that were not.

Two copies of the sources are only comparable while the head’s copy merely adds to the base’s: new functions, new #[bench::…] cases, new entries in a group list. A PR that also edits an existing scenario could make it cheaper and hide a real regression behind a small Δ. So in fallback mode .github/scripts/perf_bench_changes.py looks for any line the head’s copy removes or rewrites, and any line it inserts inside an existing item (a new statement in an existing setup). If it finds one, the job fails and the report lists each edit with the function it is in; the perf-regression-accepted label downgrades that to a warning, and the PR then says what the scenario change was. The check is a text proxy, not a Rust parser, and errs towards reporting an edit.

When neither copy runs, the outcome depends on whether the base has the crate at all:

• The base has no crates/rinch-bench. The report shows the head alone and the job passes. No merge commit’s first parent can lack the crate once it is on main, so this applies only to PRs based before then.
• The base has the crate and neither copy ran. The job fails. Usually one of the base’s own benchmarks panics there. Otherwise, breaking the benchmarks would be a way past the check. The perf-regression-accepted label downgrades this failure to a warning, the same as a regression.

The scripts the job runs have a self-test against a fake cargo (python3 .github/scripts/test_perf_scripts.py), which the job runs first.

If the head’s own benchmarks do not build or run, the job fails. The PR comment then says so, rather than keeping the previous run’s table.

The threshold and the escape hatch

The job fails when any benchmark’s Ir grows by more than 3%. Change the threshold with the repository variable PERF_REGRESSION_THRESHOLD, a number of percent.

A PR that makes a path more expensive on purpose can carry the perf-regression-accepted label. With the label, the regression is still reported, as a warning, and the job passes. Adding or removing the label runs the job again. So does changing the PR’s base branch to main: that run comes from .github/workflows/base-changed.yml, which calls perf.yml, and its check is named perf / instruction-counts. A docs-only PR, which the Perf workflow otherwise skips, gets one run there. Say in the PR which benchmark moved and why, in the pull request template’s Performance section, the same way a counter-baseline update does (see The baselines are the contract). The two checks complement each other. The baselines are exact and say which work a frame did, while the instruction counts say what that work cost. A change can move one without the other.

The comment is posted by a separate comment job, which is the only job with pull-requests: write and runs none of the PR’s code. It updates the comment that carries the <!-- rinch-perf-bench --> marker and is authored by github-actions[bot], so a human comment that quotes the marker is left alone. A pull request from a fork gets a read-only token and cannot comment. For those PRs the report is on the job’s summary page only. A PR that changes only docs/** or Markdown files does not run the job.

The cargo cache holds only dependency artifacts. A fresh checkout makes every workspace crate look stale to cargo, so those rebuild on every run anyway. Only a push to main writes the cache, and only when its key misses. The key hashes every Cargo.lock and Cargo.toml, the toolchain and the compiler environment. Pull requests only read it.

Base and head are both compiled with the Rust version the head’s rust-toolchain.toml pins. A pull request that moves the pin therefore shows no difference from the new compiler’s code generation, and its dependency build is cold. The job does not measure what a toolchain bump itself costs.

Allocation is not what it costs in a real frame

glibc’s malloc and free cost a different number of instructions depending on what the heap already holds. Which blocks can merge and which bin a free lands in depend on everything the process did before, and the order of a setup’s allocations follows HashMap iteration, which std seeds randomly. Measured on set_text_content, identical runs gave 5.34M, 5.71M and 5.83M instructions, and every instruction of the difference was inside malloc.c.

So the benchmark binary installs rinch_bench::alloc::BenchAlloc. Inside the measured operation, every allocation bumps a pointer into a region reserved in setup, and every free does nothing. Both cost the same few instructions each time. Outside the operation, the system allocator handles everything. The cost: an allocation counts as a few instructions instead of glibc’s 50 to 150, so a change that only adds allocations moves the count less than it moves a real frame. The code that builds what it allocates is still counted in full.

Running it locally

You need Valgrind and the Gungraun runner, at exactly the version of the gungraun library in Cargo.lock:

sudo apt-get install valgrind
cargo install gungraun-runner --version 0.19.4 --locked
cargo bench -p rinch-bench                      # all benchmarks, human-readable
cargo bench -p rinch-bench -- '*::hover::*'     # one benchmark (a glob over the module path)

Each run is compared with the previous run of the same benchmark on disk, so running before and after a change prints the Δ directly. To reproduce the CI table, save each side as JSON and compare the two files:

cargo bench -p rinch-bench -- --output-format=json --callgrind-args=--cache-sim=no > before.jsonl
# ...make the change...
cargo bench -p rinch-bench -- --output-format=json --callgrind-args=--cache-sim=no > after.jsonl
python3 .github/scripts/perf_compare.py --base before.jsonl --head after.jsonl

Each benchmark leaves a Callgrind profile at target/gungraun/rinch-bench/hot_paths/<group>/<bench>/callgrind.*.out. callgrind_annotate --inclusive=yes <file> or KCachegrind shows where the instructions went.

A full run takes about 40 seconds on a 24-core workstation with --parallel=4. Valgrind runs code about 50 times slower than native, and the benchmarks are sized for that: 500 rows where the operation touches one row, 200 where it touches every row.