PROTOCOL

Why this exists

Author models are unreliable at regressing pixel coordinates on a cluttered full image (measured: weak vision models ~16–24% center error → rings land off-target) but reliable at picking a numbered strip (~3–4% with the loop below). So: never eyeball coordinates; localize by discrete choice. (RSVP, ACL 2025.)

Routing — pick the cheapest path that’s actually available

1. A strong detector is available (e.g. GEMINI_API_KEY in env) → node grounding/locate.mjs auto <img> "<target>" — one call, done. Never assume the key exists. No key → path 2.
2. No detector (the normal case) → YOU are the localizer; run the grid loop.

The grid loop (you read images between steps)

Ambiguity — resolve BEFORE localizing

If the target description can match several instances (“a drum” when the scene has five), no geometry will save you. First make the reference unique (“the right-most drum”, “the drum on the front boat”), asking the user if needed — then run the loop.

Degradation path (if locate.mjs itself is unavailable)

The idea is 5 lines — reproduce it with any image tool: draw a numbered 9×9 grid → pick the strips the target spans → crop that region (+pad ~0.4 strip) and upscale → draw a finer 6×6 grid → pick again → map back (global = region_origin + local × region_size) → draw the box and LOOK at it.

Notes

• Geometry is frozen in locate.mjs (node + ffmpeg, zero npm deps — both already required by hyperframes). Don’t re-implement it ad hoc; the measured accuracy holds for THIS implementation.
• Consumers: samples/asset-fusion/_ref-circle-highlight.html takes CFG.box directly from final/auto output.
• Measured end-to-end (same agent, same template, only the localization step differs): eyeballing 6.5% avg center error → this protocol 2.3%, no case worse.