Setting the file. One moment.
Prefix Diff · Claude API · anthropics/skills · Skills Docs
Back to the top of the page — line 325
This file
Number 5.84
Position 84 of 94
Type Python
Size 92 KB
Lines 1,532 shared/preserved-thinking-migration/ prefix_diff.py
Python · 1,532 lines · 92 KB
python3 prefix_diff.py --json capture.jsonl # machine-readable
16
17 For every pair (N, N+1) it reports whether request N+1 still carries request N's
18 system prompt, tool set and messages unchanged, and if not: a guessed kind/pattern
19 (same words the API uses in its anthropic-thinking-prefix-mismatch header), the
20 first changed path, and a Claude-Code-style attribution line such as
21 system[0] changed at char 47: "...Be concise." -> "...Be concise. Current time: ..."
22
23 2. SCAN a repository for the code paths that usually cause those edits
24
25 python3 prefix_diff.py --scan path/to/repo [--ext py,ts,js]
26
27 Prints file:line leads grouped by cause. These are regex heuristics -- LEADS to read,
28 not findings. The diff mode (or the API's own response) is the evidence.
29
30 What the comparison ignores, on purpose (the API ignores them too):
31 cache_control markers anywhere; an explicit strict: false / eager_input_streaming: false on a tool (the defaults);
32 string content vs a single text block (same thing); leading/trailing whitespace of a
33 text block, and whitespace-only text blocks; key order; tool ORDER in the tools array
34 (tools bind as a name-keyed set); a defer_loading tool that no tool_reference (at any depth),
35 tool-search result or tool_addition has named yet; thinking / redacted_thinking blocks themselves (they are what is validated, not
36 part of the compared prefix) -- the API only requires that every kept thinking block
37 after the first was minted right after the kept block now in front of it, so any
38 CONTIGUOUS WINDOW of the original sequence replays fine (drop from the front, drop from
39 the back, or both); a block removed from the middle, or a reorder, is flagged separately
40 because the chain check fails for the block that follows the gap; request parameters
41 outside system / tools / messages. A URL-sourced image or document is compared without
42 its URL string: a rotated URL to the same bytes is a MATCH here and at the API, but the
43 same URL serving different bytes is a break the API catches and this script cannot see.
44
45 Interior whitespace, tool_use.input bytes, tool_result text, image bytes, a tool's strict /
46 eager_input_streaming flags and every other difference count.
47
48 A pair where the later request keeps the earlier one's messages up to some point and replaces
49 everything after it, with NO replayed thinking block at or after that point (a branch, a
50 regenerate, a retry of the last turn), is reported as MATCH with a "branch" note: nothing the
51 API would check has changed. This assumes consecutive requests of one conversation; a capture
52 that skips requests can make a real edit look like a branch.
53
54 This script's kind/pattern is a guess from the bodies alone; the API's own response (the
55 input_transformations entries, the 400 text, and the diagnosis header when present) is the
56 authority when they differ. One modelling limit to keep in mind: the pair diff compares each
57 request with the previous one, while the API judges each replayed block against the request that
58 produced it. The two agree for an append-only or steadily edited history; they differ when a
59 harness alternates prompts per model and restores them (handled: the diff then compares against
60 the last request on the same model) and for blocks minted before a later-referenced tool entered
61 the prefix (the "at most" count). In particular its counts of removed, inserted and modified items can differ from these, and its
62 treatment of image and document bytes may become finer-grained than the whole-source comparison here.
63 """
64 import argparse
65 import difflib
66 import glob
67 import json
68 import os
69 import re
70 import sys
71
72 # With stdout/stderr redirected to a pipe (as under a tool runner), CPython on Windows encodes with
73 # the ANSI code page and errors="strict", so one character outside it (an emoji in an excerpt of a
74 # changed block) would abort the run mid-report. Never raise on output.
75 for _stream in (sys.stdout, sys.stderr):
76 try :
77 _stream.reconfigure( errors = "backslashreplace" )
78 except ( AttributeError , ValueError ): # a non-TextIOWrapper stand-in; nothing to configure
79 pass
80
81 IGNORED_TOOL_KEYS = { "cache_control" } # strict and eager_input_streaming ARE compared
82 MEDIA_TYPES = { "image" , "document" , "image_url" , "document_url" }
83 INT_LIMIT = 2 ** 63
84
85
86 def clean (obj):
87 """Recursively drop None-valued keys, collapse integral floats (1.0 -> 1), and sort nothing
88 (fp() sorts keys). Applied to every compared value."""
89 if isinstance (obj, dict ):
90 return {k: clean(v) for k, v in obj.items() if v is not None }
91 if isinstance (obj, list ):
92 return [clean(v) for v in obj]
93 if isinstance (obj, float ) and obj.is_integer() and abs (obj) < INT_LIMIT :
94 return int (obj)
95 return obj
96
97
98 def canon_tool_entry (entry):
99 """One tools[] entry or by-value tool definition, in the shape the API compares: the API
100 stores a member left at its default as the member spelled out, so `type: "custom"` equals
101 type absent, `description: ""` equals description absent, `allowed_callers: ["direct"]`
102 equals allowed_callers absent, and a custom entry's strict / eager_input_streaming default
103 to False (their VALUES are compared; cache_control is not)."""
104 entry = clean({k: v for k, v in entry.items() if k not in IGNORED_TOOL_KEYS })
105 if entry.get( "description" ) == "" :
106 entry.pop( "description" )
107 if entry.get( "allowed_callers" ) == [ "direct" ]:
108 entry.pop( "allowed_callers" )
109 if not entry.get( "type" ) or entry.get( "type" ) == "custom" :
110 entry.pop( "type" , None )
111 entry.setdefault( "strict" , False )
112 entry.setdefault( "eager_input_streaming" , False )
113 return entry
114
115 # ----------------------------------------------------------------------------- loading
116
117
118 def load_requests (paths):
119 """Return a list of (label, body, conversation) in order. A .jsonl file yields one body per
120 line; a .json file yields one body (or a list of bodies); a directory yields its *.json /
121 *.jsonl sorted. The conversation key keeps pairs from crossing conversation boundaries (two
122 conversations legitimately differ in the compared parts, so a cross-boundary pair is a false
123 mismatch): each .jsonl FILE is one conversation -- a wrapper line's conversation_id overrides,
124 so several conversations can share a file, as for the probe -- while bare .json bodies given
125 together, on the command line or inside one directory, stay one conversation in sorted order
126 (the two-.json-files usage)."""
127 out = []
128
129 def add_file (p, default_conv):
130 with open (p, "r" , encoding = "utf-8" ) as f:
131 if p.endswith( ".jsonl" ):
132 conv = os.path.splitext(p)[ 0 ]
133 for i, line in enumerate (f, 1 ):
134 line = line.strip()
135 if not line:
136 continue
137 try :
138 out.append(( " %s : %d " % (os.path.basename(p), i), json.loads(line), conv))
139 except ValueError as e:
140 raise SystemExit ( " %s : %d : not valid JSON ( %s )" % (p, i, e))
141 else :
142 try :
143 data = json.load(f)
144 except ValueError as e:
145 raise SystemExit ( " %s : not valid JSON ( %s )" % (p, e))
146 if isinstance (data, list ):
147 for i, b in enumerate (data, 1 ):
148 out.append(( " %s [ %d ]" % (os.path.basename(p), i), b, default_conv))
149 else :
150 out.append((os.path.basename(p), data, default_conv))
151
152 for p in paths:
153 if os.path.isdir(p):
154 for fp in sorted (glob.glob(os.path.join(p, "*.json" )) + glob.glob(os.path.join(p, "*.jsonl" ))):
155 add_file(fp, p)
156 else :
157 add_file(p, "" )
158 bodies = []
159 for label, b, conv in out:
160 if isinstance (b, dict ) and "request" in b and isinstance (b[ "request" ], dict ) and "messages" in b[ "request" ]:
161 # a capture wrapper {request:..., conversation_id:..., headers:..., response:...}.
162 # Only the documented conversation_id groups; a wrapper's id is a per-request stamp
163 # in some captures and a correlation key in others, so it never decides grouping.
164 if b.get( "conversation_id" ):
165 conv = str (b[ "conversation_id" ])
166 b = b[ "request" ]
167 if not isinstance (b, dict ) or "messages" not in b:
168 raise SystemExit ( " %s : not a Messages API request body (no 'messages')" % label)
169 bodies.append((label, b, conv))
170 return bodies
171
172
173 def is_compaction_request (body):
174 """True for a body carrying the top-level compaction field (beta compact-2026-09-04): a request for a summary, whose
175 reply is the signed block (or nothing), not a conversation turn. drop_block_probe.py keeps the same predicate."""
176 return isinstance (body, dict ) and isinstance (body.get( "compaction" ), dict )
177
178
179 # ----------------------------------------------------------------------------- canonical view
180
181
182 def _text_block (text):
183 return { "type" : "text" , "text" : text}
184
185
186 def canon_blocks (content):
187 """Normalise a content field (string or list of blocks) to the bound view: a list of
188 blocks with cache_control removed, whitespace-only text dropped, text edges trimmed.
189 Returns (blocks, thinking_signatures, thinking_text_by_signature)."""
190 if content is None :
191 return [], [], {}
192 if isinstance (content, str ):
193 content = [_text_block(content)]
194 if not isinstance (content, list ):
195 content = [content]
196 blocks, sigs, texts = [], [], {}
197 for wire_idx, b in enumerate (content):
198 if not isinstance (b, dict ):
199 blocks.append({ "type" : "_raw" , "value" : b, "_wire" : wire_idx})
200 continue
201 t = b.get( "type" )
202 if t in ( "thinking" , "redacted_thinking" ):
203 sig = b.get( "signature" ) or b.get( "data" ) or ""
204 sigs.append((wire_idx, sig))
205 if t == "thinking" :
206 texts[sig] = b.get( "thinking" ) # kept so a later request's text can be compared (a modified block is a break)
207 continue
208 b = clean({k: v for k, v in b.items() if k != "cache_control" })
209 tl = b.get( "tool" )
210 if t == "tool_addition" and isinstance (tl, dict ) and tl.get( "type" ) == "tool_definition" \
211 and isinstance (tl.get( "definition" ), dict ):
212 # compared in the API's canonical shape, exactly as a tools[] entry: a replay that only
213 # spells out a default (type "custom", description "") is not an edit
214 b[ "tool" ] = dict (tl, definition = canon_tool_entry(tl[ "definition" ]))
215 if b.get( "citations" ) in ([], None ):
216 b.pop( "citations" , None )
217 if t == "text" :
218 txt = (b.get( "text" ) or "" )
219 if txt.strip() == "" :
220 continue
221 b[ "text" ] = txt.strip()
222 if t == "tool_result" :
223 if b.get( "is_error" ) is False :
224 b.pop( "is_error" )
225 if isinstance (b.get( "content" ), str ):
226 b[ "content" ] = [_text_block(b[ "content" ])]
227 if isinstance (b.get( "content" ), list ):
228 inner, _s, _t = canon_blocks(b[ "content" ])
229 b[ "content" ] = inner
230 if t in ( "image" , "document" ) and isinstance (b.get( "source" ), dict ) and b[ "source" ].get( "type" ) == "url" :
231 # the API compares the bytes behind the URL, not the URL string: drop it, keep the rest
232 b[ "source" ] = {k: v for k, v in b[ "source" ].items() if k != "url" }
233 b[ "type" ] = t + "_url"
234 b[ "_wire" ] = wire_idx
235 blocks.append(b)
236 return blocks, sigs, texts
237
238
239 def canon_system (system):
240 blocks, _s, _t = canon_blocks(system if system is not None else [])
241 return blocks
242
243
244 def _toolset_key (server):
245 return "mcp_toolset: %s " % server if isinstance (server, str ) and server else None
246
247
248 def canon_tools (tools, messages):
249 """Inline tools as a name-keyed map; deferred tools kept separately and compared only
250 once something in the messages names them (see walk). Server tools (type != custom) are
251 keyed by type+name, MCP toolsets by server name. A compaction block's tool_changes field
252 is not read. Once a compaction block has replaced the messages that named a deferred
253 tool, that tool counts as unnamed and edits to it are not reported (diff_pair adds a
254 note when the block has a tool_changes field). The exception is the first request that
255 carries the block: it is compared against an earlier request whose messages still name
256 the tool."""
257 referenced = set ()
258
259 def walk (blocks):
260 # a deferred tool binds once something names it: a tool_reference at any depth (including inside a
261 # tool_result's content), a tool-search result's tool_references, or a tool_addition block in a
262 # mid-conversation system message, by reference or by value (the docs do not say whether a
263 # definition by value binds a deferred entry of the same name; counting it is the cautious choice)
264 for b in blocks or []:
265 if not isinstance (b, dict ):
266 continue
267 t = b.get( "type" )
268 if t == "tool_reference" :
269 name = b.get( "tool_name" ) or b.get( "name" )
270 if isinstance (name, str ) and name:
271 referenced.add(name)
272 elif t == "tool_addition" :
273 tool = b.get( "tool" ) if isinstance (b.get( "tool" ), dict ) else b
274 if tool.get( "type" ) == "tool_definition" and isinstance (tool.get( "definition" ), dict ):
275 tool = tool[ "definition" ] # inline-tools-2026-09-15 sends a whole tools[] entry in this wrapper
276 name = tool.get( "name" ) or tool.get( "tool_name" )
277 if tool.get( "type" ) in ( "mcp_toolset" , "mcp_toolset_reference" ):
278 name = _toolset_key(tool.get( "mcp_server_name" ) or tool.get( "server_name" ))
279 elif tool.get( "server_name" ) and name and not str (name).startswith( str (tool[ "server_name" ])):
280 name = " %s _ %s " % (tool[ "server_name" ], name)
281 if isinstance (name, str ) and name:
282 referenced.add(name)
283 inner = b.get( "content" )
284 if isinstance (inner, list ):
285 walk(inner)
286 elif isinstance (inner, dict ):
287 walk([inner])
288 refs = b.get( "tool_references" )
289 if isinstance (refs, list ):
290 walk(refs)
291 if isinstance (inner, dict ) and isinstance (inner.get( "tool_references" ), list ):
292 walk(inner[ "tool_references" ])
293
294 for m in messages or []:
295 c = m.get( "content" )
296 if isinstance (c, list ):
297 walk(c)
298 inline, deferred, server_names = {}, {}, {}
299 for t in tools or []:
300 if not isinstance (t, dict ):
301 continue
302 t = canon_tool_entry(t)
303 name = t.get( "name" )
304 if isinstance (name, str ) and name:
305 key = name
306 elif not name:
307 key = " %s " % t.get( "type" )
308 else :
309 key = fp(name)
310 if t.get( "type" ) == "mcp_toolset" and _toolset_key(t.get( "mcp_server_name" )):
311 key = _toolset_key(t[ "mcp_server_name" ])
312 elif t.get( "type" ) and t.get( "type" ) != "custom" and isinstance (name, str ) and name:
313 key = " %s : %s " % (t[ "type" ], name)
314 server_names[key] = name
315 if t.get( "defer_loading" ):
316 d = {k: v for k, v in t.items() if k != "defer_loading" }
317 deferred[key] = d
318 else :
319 inline[key] = t
320 # a server tool is keyed type:name, but a reference or a by-value definition names it by name alone
321 referenced |= {k for k, n in server_names.items() if n in referenced}
322 return inline, deferred, referenced
323
324
325 def canon_messages (messages):
326 out = []
327 for m in messages or []:
328 blocks, sigs, texts = canon_blocks(m.get( "content" ))
329 entry = { "role" : m.get( "role" ), "content" : blocks}
330 for k in ( "clear_at" , "output_config" ):
331 if m.get(k) is not None :
332 entry[k] = m[k]
333 out.append({ "msg" : entry, "sigs" : sigs, "texts" : texts})
334 return out
335
336
337 def _strip_wire (obj):
338 if isinstance (obj, dict ):
339 return {k: _strip_wire(v) for k, v in obj.items() if k != "_wire" }
340 if isinstance (obj, list ):
341 return [_strip_wire(v) for v in obj]
342 return obj
343
344
345 def fp (obj):
346 return json.dumps(clean(_strip_wire(obj)), sort_keys = True , separators = ( "," , ":" ), ensure_ascii = False )
347
348
349 def wire (block, fallback):
350 return block.get( "_wire" , fallback) if isinstance (block, dict ) else fallback
351
352
353 # ----------------------------------------------------------------------------- attribution
354
355
356 def first_diff_char (a, b):
357 n = min ( len (a), len (b))
358 for i in range (n):
359 if a[i] != b[i]:
360 return i
361 return n if len (a) != len (b) else - 1
362
363
364 def excerpt (s, at, width = 40 ):
365 lo = max ( 0 , at - 12 )
366 return ( "..." if lo else "" ) + s[lo:at + width].replace( " \n " , " \\ n" ) + ( "..." if at + width < len (s) else "" )
367
368
369 def attribute (label, old, new):
370 """Claude-Code-style one-liner for a changed scalar/object."""
371 so, sn = (old if isinstance (old, str ) else fp(old)), (new if isinstance (new, str ) else fp(new))
372 at = first_diff_char(so, sn)
373 if at < 0 :
374 return " %s changed (representation only)" % label
375 return ' %s changed at char %d : " %s " -> " %s "' % (label, at, excerpt(so, at), excerpt(sn, at))
376
377
378 # ----------------------------------------------------------------------------- the diff
379
380
381 def diff_system (a, b):
382 """Returns list of attribution strings (empty if equal)."""
383 notes = []
384 n = max ( len (a), len (b))
385 for i in range (n):
386 if i >= len (a):
387 notes.append( "system[ %d ] added: %s " % (i, excerpt(fp(b[i]), 0 )))
388 elif i >= len (b):
389 notes.append( "system[ %d ] removed: %s " % (i, excerpt(fp(a[i]), 0 )))
390 elif fp(a[i]) != fp(b[i]):
391 if a[i].get( "type" ) == "text" and b[i].get( "type" ) == "text" :
392 notes.append(attribute( "system[ %d ]" % i, a[i][ "text" ], b[i][ "text" ]))
393 else :
394 notes.append(attribute( "system[ %d ]" % i, a[i], b[i]))
395 return notes
396
397
398 def diff_tools (ta, tb):
399 ia, da, refa = ta
400 ib, db, refb = tb
401 notes, set_changed = [], False
402 for name in sorted ( set (ia) | set (ib)):
403 if name not in ib:
404 notes.append( "tools: %s removed" % name); set_changed = True
405 elif name not in ia:
406 notes.append( "tools: %s added" % name); set_changed = True
407 elif fp(ia[name]) != fp(ib[name]):
408 for field in sorted ( set (ia[name]) | set (ib[name])):
409 if fp(ia[name].get(field)) != fp(ib[name].get(field)):
410 notes.append(attribute( "tools: %s %s " % (name, field), ia[name].get(field, "" ), ib[name].get(field, "" )))
411 # deferred tools: bound only once referenced in request N (the minting request's view)
412 for name in sorted (refa | refb):
413 if name in da or name in db:
414 if name not in db:
415 notes.append( "tools: deferred %s (referenced) removed" % name); set_changed = True
416 elif name not in da:
417 pass # newly loaded by reference: allowed
418 elif fp(da[name]) != fp(db[name]):
419 notes.append(attribute( "tools: deferred %s (referenced)" % name, da[name], db[name]))
420 return notes, set_changed
421
422
423 def block_type (b):
424 return b.get( "type" ) if isinstance (b, dict ) else type (b). __name__
425
426
427 def _item_count (msgs):
428 """Server-style item count: one item per message plus one per content block."""
429 return sum ( 1 + len (x[ "msg" ][ "content" ]) for x in msgs)
430
431
432 def _block_diffs (A, B, changed_idx):
433 """Block-level opcodes for in-place message edits. Returns (mod, rem, ins, notes,
434 first_path, first_pos) where mod/rem/ins are lists of (msg_index, block_type)."""
435 mod, rem, ins, notes = [], [], [], []
436 first_path, first_pos, first_wire = None , None , None
437 for i in changed_idx:
438 ai, bi = A[i][ "msg" ], B[i][ "msg" ]
439 ca, cb = ai[ "content" ], bi[ "content" ]
440 if ai.get( "role" ) != bi.get( "role" ) or ai.get( "clear_at" ) != bi.get( "clear_at" ):
441 notes.append( "messages[ %d ] role or clear_at changed" % i)
442 mod.append((i, "message" ))
443 if first_path is None :
444 first_path, first_pos, first_wire = "messages. %d " % i, "at" , (i, - 1 )
445 bsm = difflib.SequenceMatcher( a = [fp(x) for x in ca], b = [fp(x) for x in cb], autojunk = False )
446 for bop, a1, a2, b1, b2 in bsm.get_opcodes():
447 if bop == "equal" :
448 continue
449 if first_path is None :
450 if bop == "delete" :
451 w = wire(ca[a1 - 1 ], a1 - 1 ) if a1 else 0
452 first_path, first_pos = ( "messages. %d .content. %d " % (i, w), "after" if a1 else "before" )
453 first_wire = (i, wire(ca[a1], a1))
454 else :
455 first_path, first_pos = ( "messages. %d .content. %d " % (i, wire(cb[b1], b1)), "at" )
456 first_wire = (i, wire(cb[b1], b1))
457 if bop == "replace" and (a2 - a1) == (b2 - b1):
458 for q in range (a2 - a1):
459 ta, tb = block_type(ca[a1 + q]), block_type(cb[b1 + q])
460 mod.append((i, ta)) # the API keys the change on the STORED (old) item's type
461 label = "messages[ %d ] ( %s ) content[ %d ] ( %s%s )" % (i, ai[ "role" ], wire(ca[a1 + q], a1 + q), ta, "" if ta == tb else " -> " + tb)
462 notes.append(attribute(label, ca[a1 + q], cb[b1 + q]))
463 elif bop == "replace" :
464 # unequal lengths: pair old and new blocks of the same type in order and call those
465 # changes; only the leftovers are removals / insertions
466 old_q, new_q = list ( range (a1, a2)), list ( range (b1, b2))
467 paired = []
468 for q in old_q:
469 for r in new_q:
470 if r not in [pr for _, pr in paired] and block_type(ca[q]) == block_type(cb[r]):
471 paired.append((q, r))
472 break
473 for q, r in paired:
474 mod.append((i, block_type(ca[q])))
475 notes.append(attribute( "messages[ %d ] ( %s ) content[ %d ] ( %s )" % (i, ai[ "role" ], wire(ca[q], q), block_type(ca[q])), ca[q], cb[r]))
476 for q in old_q:
477 if q not in [pq for pq, _ in paired]:
478 rem.append((i, block_type(ca[q])))
479 notes.append( "messages[ %d ] ( %s ) content[ %d ] ( %s ) removed: %s " % (i, ai[ "role" ], wire(ca[q], q), block_type(ca[q]), excerpt(fp(ca[q]), 0 )))
480 for r in new_q:
481 if r not in [pr for _, pr in paired]:
482 ins.append((i, block_type(cb[r])))
483 notes.append( "messages[ %d ] ( %s ) content[ %d ] ( %s ) inserted: %s " % (i, ai[ "role" ], wire(cb[r], r), block_type(cb[r]), excerpt(fp(cb[r]), 0 )))
484 else :
485 for q in range (a1, a2):
486 rem.append((i, block_type(ca[q])))
487 notes.append( "messages[ %d ] ( %s ) content[ %d ] ( %s ) removed: %s " % (i, ai[ "role" ], wire(ca[q], q), block_type(ca[q]), excerpt(fp(ca[q]), 0 )))
488 for q in range (b1, b2):
489 ins.append((i, block_type(cb[q])))
490 notes.append( "messages[ %d ] ( %s ) content[ %d ] ( %s ) inserted: %s " % (i, ai[ "role" ], wire(cb[q], q), block_type(cb[q]), excerpt(fp(cb[q]), 0 )))
491 return mod, rem, ins, notes, first_path, first_pos, first_wire
492
493
494 def _classify_in_place (A, B, changed_idx):
495 mod, rem, ins, notes, first_path, first_pos, first_wire = _block_diffs(A, B, changed_idx)
496 n_mod, n_rem, n_ins = len (mod), len (rem), len (ins)
497 if n_mod and not n_rem and not n_ins:
498 kind = "blocks_modified"
499 elif n_rem and not n_mod and not n_ins:
500 kind = "blocks_removed"
501 elif n_ins and not n_mod and not n_rem:
502 kind = "blocks_inserted"
503 else :
504 kind = "blocks_replaced"
505 touched = sorted ( set (i for i, _ in mod + rem + ins) | set (changed_idx))
506 first_message_only = touched and max (touched) == 0
507 types_changed = {t for _, t in mod} | {t for _, t in rem} | {t for _, t in ins}
508 only_media = bool (types_changed) and types_changed <= MEDIA_TYPES
509 blocks_carried = sum ( len (x[ "msg" ][ "content" ]) for x in A)
510 roles_touched = {A[i][ "msg" ][ "role" ] for i in touched}
511
512 def user_text_removed (entry):
513 i, t = entry
514 return t == "text" and A[i][ "msg" ][ "role" ] == "user"
515
516 def later_user_turns (i):
517 return sum ( 1 for x in A[i + 1 :] if x[ "msg" ][ "role" ] == "user" )
518
519 pattern = "unknown"
520 if kind == "blocks_modified" :
521 if types_changed <= { "image_url" , "document_url" }:
522 pattern = "image_url_resigned"
523 elif types_changed <= { "tool_result" }:
524 pattern = "tool_results_rewritten"
525 elif types_changed <= { "tool_use" , "server_tool_use" , "mcp_tool_use" }:
526 pattern = "tool_use_rewritten"
527 elif roles_touched == { "system" }:
528 pattern = "system_block_rerendered"
529 elif only_media:
530 pattern = "media_stripped"
531 elif n_mod >= 4 and ((n_mod * 2 >= blocks_carried and len (types_changed) > 1 ) or n_mod >= blocks_carried):
532 pattern = "reserialized"
533 elif first_message_only:
534 pattern = "first_message_rewritten"
535 elif kind == "blocks_removed" :
536 if n_rem == 1 :
537 i, t = rem[ 0 ]
538 if user_text_removed(rem[ 0 ]) and later_user_turns(i) <= 1 and i != 0 :
539 pattern = "reminder_stripped"
540 elif only_media:
541 pattern = "media_stripped"
542 elif first_message_only:
543 pattern = "first_message_rewritten"
544 elif user_text_removed(rem[ 0 ]):
545 pattern = "history_block_stripped"
546 else :
547 if roles_touched == { "system" }:
548 pattern = "system_blocks_stripped"
549 elif only_media:
550 pattern = "media_stripped"
551 elif first_message_only:
552 pattern = "first_message_rewritten"
553 elif kind == "blocks_inserted" :
554 if n_ins == 1 and ins[ 0 ][ 1 ] in ( "text" ,) and ins[ 0 ][ 0 ] != 0 :
555 pattern = "block_inserted"
556 elif any (t == "compaction" for _, t in ins):
557 pattern = "compaction_summary"
558 elif first_message_only:
559 pattern = "first_message_rewritten"
560 else :
561 if only_media:
562 pattern = "media_stripped"
563 elif first_message_only:
564 pattern = "first_message_rewritten"
565 return { "kind" : kind, "pattern" : pattern, "notes" : notes, "changed_validated" : first_path,
566 "position" : first_pos, "items_modified" : n_mod, "items_removed" : n_rem, "items_inserted" : n_ins,
567 "messages_delta" : 0 , "first_changed_msg" : min (touched), "first_changed_wire" : first_wire, "reading" : "in-place" }
568
569
570 def _classify_runs (A, B, ops):
571 notes = []
572 deletes = [op for op in ops if op[ 0 ] == "delete" ]
573 replaces = [op for op in ops if op[ 0 ] == "replace" ]
574 inserts = [op for op in ops if op[ 0 ] == "insert" ]
575 first_i1 = min (op[ 1 ] for op in ops)
576 for op, i1, i2, j1, j2 in ops:
577 if op == "delete" :
578 notes.append( "messages[ %d .. %d ] removed ( %s )" % (i1, i2 - 1 , ", " .join(A[q][ "msg" ][ "role" ] for q in range (i1, i2))))
579 elif op == "insert" :
580 notes.append( " %d message(s) inserted before messages[ %d ] ( %s )" % (j2 - j1, i1, ", " .join(B[q][ "msg" ][ "role" ] for q in range (j1, j2))))
581 else :
582 notes.append( "messages[ %d .. %d ] replaced by %d message(s): %s " % (i1, i2 - 1 , j2 - j1, excerpt(fp(B[j1][ "msg" ]), 0 , 60 )))
583 removed_msgs = sum (i2 - i1 for op, i1, i2, j1, j2 in ops if op != "insert" )
584 inserted_msgs = sum (j2 - j1 for op, i1, i2, j1, j2 in ops if op != "delete" )
585 items_removed = sum ( 1 + len (A[q][ "msg" ][ "content" ]) for op, i1, i2, j1, j2 in ops if op != "insert" for q in range (i1, i2))
586 items_inserted = sum ( 1 + len (B[q][ "msg" ][ "content" ]) for op, i1, i2, j1, j2 in ops if op != "delete" for q in range (j1, j2))
587 res = { "notes" : notes, "items_removed" : items_removed, "items_inserted" : items_inserted, "items_modified" : 0 ,
588 "messages_delta" : inserted_msgs - removed_msgs, "first_changed_msg" : first_i1, "reading" : "runs" }
589 removed_types = {block_type(b) for op, i1, i2, j1, j2 in ops if op != "insert" for q in range (i1, i2) for b in A[q][ "msg" ][ "content" ]}
590 only_media = bool (removed_types) and removed_types <= MEDIA_TYPES and not inserts
591 if len (deletes) == 1 and not replaces and not inserts:
592 op, i1, i2, j1, j2 = deletes[ 0 ]
593 res[ "kind" ] = "blocks_removed"
594 if i1 == 0 and (i2 - i1) >= 1 :
595 res.update( pattern = "rolling_truncation" , changed_validated = "messages.0" , position = "before" )
596 elif all (A[q][ "msg" ][ "role" ] == "system" for q in range (i1, i2)) and (i2 - i1) >= 2 :
597 res.update( pattern = "system_blocks_stripped" , changed_validated = "messages. %d " % (i1 - 1 ), position = "after" )
598 elif only_media:
599 res.update( pattern = "media_stripped" , changed_validated = "messages. %d " % (i1 - 1 ), position = "after" )
600 elif (i2 - i1) >= 2 and (i2 - i1) >= 2 * i1:
601 res.update( pattern = "tail_kept" , changed_validated = "messages. %d " % (i1 - 1 ), position = "after" )
602 else :
603 res.update( pattern = "unknown" , changed_validated = "messages. %d " % (i1 - 1 ), position = "after" )
604 return res
605 if len (replaces) == 1 and not deletes and not inserts:
606 op, i1, i2, j1, j2 = replaces[ 0 ]
607 has_compaction = any ( isinstance (b, dict ) and b.get( "type" ) == "compaction" for q in range (j1, j2) for b in B[q][ "msg" ][ "content" ])
608 res[ "kind" ] = "blocks_replaced"
609 if has_compaction or (items_removed >= 8 and items_removed >= 2 * items_inserted and (j2 - j1) < (i2 - i1)):
610 res.update( pattern = "compaction_summary" , changed_validated = "messages. %d " % j1, position = "at" )
611 elif (j2 - j1) < (i2 - i1):
612 res.update( pattern = "unknown" , changed_validated = "messages. %d " % j1, position = "at" ,
613 note = "shorter run in place, but fewer than 8 items removed: the API names this compaction_summary only past that size" )
614 res[ "notes" ].append(res.pop( "note" ))
615 else :
616 res.update( pattern = "unknown" , changed_validated = "messages. %d " % j1, position = "at" )
617 return res
618 if inserts and not deletes and not replaces:
619 res.update( kind = "blocks_inserted" , pattern = "unknown" , changed_validated = "messages. %d " % inserts[ 0 ][ 3 ], position = "at" )
620 return res
621 res.update( kind = "many_changes" if len (ops) > 2 else "blocks_replaced" , pattern = "unknown" ,
622 changed_validated = "messages. %d " % first_i1, position = "at" )
623 return res
624
625
626 def diff_messages (A, B):
627 """A, B: canonical message lists (request N, request N+1). Returns a dict describing the
628 messages-section verdict, or None when A is an unchanged prefix of B (chain warnings aside)."""
629 fa = [fp(x[ "msg" ]) for x in A]
630 fb = [fp(x[ "msg" ]) for x in B]
631 res = { "notes" : [], "thinking_notes" : []}
632
633 # thinking-chain check over the shared region: the API requires every kept thinking block
634 # after the first to have been minted right after the kept block now in front of it, so the
635 # kept blocks must be a contiguous window of the original sequence (any window). A missing
636 # predecessor of a kept block, or a reorder, fails the block after the gap.
637 shared = min ( len (A), len (B))
638 seq_a = [(i, s) for i in range (shared) for _w, s in A[i][ "sigs" ] if fa[i] == fb[i]]
639 seq_b = [(i, s) for i in range (shared) for _w, s in B[i][ "sigs" ] if fa[i] == fb[i]]
640 sa = [s for _i, s in seq_a]
641 sb = [s for _i, s in seq_b]
642 later = [(i, s) for i in range (shared, len (B)) for _w, s in B[i][ "sigs" ]] # blocks minted after request N
643 if sb and sa != sb and later and [s for s in sb if s in sa] and [s for s in sb if s in sa][ - 1 ] != sa[ - 1 ]:
644 res[ "thinking_notes" ].append(
645 "the newest already-sent thinking block (messages[ %d ]) was removed while a later block (messages[ %d ]) remains: "
646 "that later block was minted right after the removed one, so its recorded predecessor is gone (predecessor_missing)"
647 % (seq_a[ - 1 ][ 0 ], later[ 0 ][ 0 ]))
648 elif sb and sa != sb:
649 kept = [s for s in sb if s in sa]
650 new_in_b = [s for s in sb if s not in sa]
651 idx = [sa.index(s) for s in kept]
652 ok = ( not new_in_b) and idx == sorted (idx) and ( not idx or idx == list ( range (idx[ 0 ], idx[ 0 ] + len (idx))))
653 if not ok:
654 if idx != sorted (idx):
655 what = "re-sent in a different order (predecessor_reordered)"
656 elif new_in_b:
657 what = "re-sent with thinking blocks that were not in the earlier request"
658 else :
659 gaps = [sa[q] for q in range (idx[ 0 ], idx[ - 1 ]) if q not in idx]
660 first_gap = next ((i for i, s in seq_a if s in gaps), None )
661 what = "re-sent with a thinking block removed from the MIDDLE of the kept run (first gap in messages[ %s ]); the kept block after the gap fails the predecessor check (predecessor_missing)" % first_gap
662 res[ "thinking_notes" ].append(
663 "thinking blocks in the already-sent turns %s : the API accepts any contiguous window of the original "
664 "blocks (drop from the front, from the back, or both) and nothing else" % what)
665
666 # a thinking block replayed with different text than the earlier request sent under the same signature
667 texts_a = {s: t for x in A for s, t in x.get( "texts" , {}).items()}
668 for i, x in enumerate (B):
669 for s, t in x.get( "texts" , {}).items():
670 if s in texts_a and texts_a[s] != t:
671 res[ "thinking_notes" ].append(
672 "the thinking text of the block in messages[ %d ] differs from the earlier request that carried the same signature: "
673 "the API rejects a modified thinking block with a 400 (truncated, summarized or re-wrapped thinking is an edit)" % i)
674 break
675
676 if len (B) >= len (A) and fb[: len (A)] == fa:
677 return None if not res[ "thinking_notes" ] else { "pattern" : None , "kind" : None , ** res}
678
679 # branch / regenerate / retry: the shared head is intact up to the divergence and no replayed
680 # thinking block sits at or after it -> nothing the API checks has changed
681 k = 0
682 while k < min ( len (A), len (B)) and fa[k] == fb[k]:
683 k += 1
684 if all (i < k for i, _w in thinking_positions(B)):
685 res.update({ "kind" : None , "pattern" : None , "branch" : True , "divergence" : k})
686 res[ "notes" ].append( "tail replaced from messages[ %d ] on, after the last replayed thinking block -- "
687 "nothing the API checks has changed (a compaction that replays no earlier thinking, a branch, a regenerate, or a retry)" % k)
688 return res
689
690 # two readings: in place (same positions, some messages differ) vs runs removed/replaced
691 in_place = None
692 if len (B) >= len (A):
693 changed_idx = [i for i in range ( len (A)) if fa[i] != fb[i]]
694 in_place = _classify_in_place(A, B, changed_idx)
695 sm = difflib.SequenceMatcher( a = fa, b = fb, autojunk = False )
696 ops = [ list (op) for op in sm.get_opcodes() if op[ 0 ] != "equal" ]
697 if ops and ops[ - 1 ][ 0 ] == "replace" and ops[ - 1 ][ 2 ] == len (fa) and (ops[ - 1 ][ 4 ] - ops[ - 1 ][ 3 ]) > (ops[ - 1 ][ 2 ] - ops[ - 1 ][ 1 ]):
698 op, i1, i2, j1, j2 = ops[ - 1 ] # A's tail replaced by a longer B tail = edit + new turns
699 ops[ - 1 ] = [ "replace" , i1, i2, j1, j1 + (i2 - i1)]
700 if ops and ops[ - 1 ][ 0 ] == "insert" and ops[ - 1 ][ 2 ] == len (fa):
701 ops = ops[: - 1 ] # the newly appended turns are expected
702 runs = _classify_runs(A, B, [ tuple (op) for op in ops]) if ops else None
703 if in_place is not None and runs is not None :
704 # server-style item costs: a changed message counts its own item plus its changed blocks
705 cost_in_place = (in_place[ "items_modified" ] + in_place[ "items_removed" ] + in_place[ "items_inserted" ]
706 + len ([i for i in range ( len (A)) if fa[i] != fb[i]]))
707 cost_runs = runs[ "items_removed" ] + runs[ "items_inserted" ]
708 chosen = in_place if cost_in_place <= cost_runs else runs
709 else :
710 chosen = in_place or runs
711 if chosen is None :
712 return None
713 res.update(chosen)
714 return res
715
716
717 def thinking_positions (B):
718 """(msg_index, wire_block_index) of every thinking block in request N+1."""
719 out = []
720 for i, x in enumerate (B):
721 for w, _s in x[ "sigs" ]:
722 out.append((i, w))
723 return out
724
725
726 def _model_key (model):
727 """Dated aliases of one model (claude-x-5-1 and claude-x-5-1-20260901, or -latest) read each other's blocks;
728 compare on the undated name so they count as the same model here."""
729 return re.sub( r "- (\d {8}| latest )$ " , "" , model or "" )
730
731
732 def compaction_start (messages):
733 """Index of the message holding the last compaction block with non-null content, or 0. The API compares the
734 messages from that block on (with server-side compaction the checked prefix restarts there); everything
735 before it is outside the check, so a client that drops those messages has not edited anything."""
736 start = None
737 for i, m in enumerate (messages or []):
738 c = m.get( "content" ) if isinstance (m, dict ) else None
739 if isinstance (c, list ):
740 for b in c:
741 if isinstance (b, dict ) and b.get( "type" ) == "compaction" and b.get( "content" ) is not None :
742 start = i
743 return start
744
745
746 def _has_signed_block (message):
747 """True when a message holds a compaction block with a signature and a summary (the compact-2026-09-04 kind)."""
748 c = message.get( "content" ) if isinstance (message, dict ) else None
749 return isinstance (c, list ) and any ( isinstance (b, dict ) and b.get( "type" ) == "compaction" and b.get( "signature" )
750 and b.get( "content" ) is not None for b in c)
751
752
753 def _signed_block_alone (messages):
754 """True when messages[0] is a compaction block signed with a summary (the compact-2026-09-04 block) sent as a message
755 of its own. Before it, compaction blocks with null content and fallback blocks are skipped; after it, a pinned MCP
756 listing (an mcp_tool_listing block) and null-content compaction blocks are skipped. Anything else returns False, and
757 diff_pair falls back to the compaction-boundary comparison (nothing before the block is compared)."""
758 first = (messages or [ None ])[ 0 ]
759 c = first.get( "content" ) if isinstance (first, dict ) else None
760 if not isinstance (c, list ) or not c or not all ( isinstance (b, dict ) for b in c):
761 return False
762 signed = next ((i for i, b in enumerate (c) if b.get( "type" ) == "compaction" and b.get( "signature" )
763 and b.get( "content" ) is not None ), None )
764 if signed is None :
765 return False
766 ignored = lambda b: b.get( "type" ) == "compaction" and b.get( "content" ) is None
767 return ( all (ignored(b) or b.get( "type" ) == "fallback" for b in c[:signed])
768 and all (ignored(b) or b.get( "type" ) == "mcp_tool_listing" for b in c[signed + 1 :]))
769
770
771 def _signed_blocks (messages):
772 """Positions ("messages[i].content[j]") of every compaction block signed with a summary, in order as sent."""
773 out = []
774 for i, m in enumerate (messages or []):
775 c = m.get( "content" ) if isinstance (m, dict ) else None
776 for j, b in enumerate (c if isinstance (c, list ) else []):
777 if isinstance (b, dict ) and b.get( "type" ) == "compaction" and b.get( "signature" ) and b.get( "content" ) is not None :
778 out.append( "messages[ %d ].content[ %d ]" % (i, j))
779 return out
780
781
782 def _carries_tool_changes (messages):
783 first = (messages or [ None ])[ 0 ]
784 c = first.get( "content" ) if isinstance (first, dict ) else None
785 return any ( isinstance (b, dict ) and b.get( "type" ) == "compaction" and "tool_changes" in b
786 for b in (c if isinstance (c, list ) else []))
787
788
789 def _kept_alignment (A_all, B_msgs):
790 """For a later request whose messages[0] holds a compaction block the earlier request lacks: the index in the
791 earlier request's messages that lines up with messages[1] of the later request, or None. The anchor is a thinking
792 signature: a kept assistant message whose block the earlier request also carries fixes the offset (signatures are
793 unique, so this is never a coincidence, where equal message bytes - a repeated "continue" - could be); with no such
794 anchor this is the older shape (the block arrived in the reply and everything behind it is new) and None is
795 returned. Several anchors vote by how many kept messages they line up byte for byte, the later offset on a tie.
796 A kept message that was edited lines up all the same and the edit is then reported. Called only for a signed block
797 sent as a message of its own (_signed_block_alone)."""
798 fa = [fp(x[ "msg" ]) for x in A_all]
799 fb = [fp(x[ "msg" ]) for x in B_msgs]
800 sig_at = {}
801 for j, x in enumerate (A_all):
802 for _w, s in x[ "sigs" ]:
803 if s: # an unsigned block anchors nothing: "" is not unique
804 sig_at.setdefault(s, j)
805 best, best_score = None , - 1
806 for k in range ( 1 , len (B_msgs)):
807 for _w, s in B_msgs[k][ "sigs" ]:
808 j = sig_at.get(s) if s else None
809 if j is None or j - (k - 1 ) < 0 :
810 continue
811 offset = j - (k - 1 )
812 score = sum ( 1 for q in range ( 1 , len (fb)) if offset + q - 1 < len (fa) and fa[offset + q - 1 ] == fb[q])
813 if score > best_score or (score == best_score and offset > best):
814 best, best_score = offset, score
815 return best
816
817
818 def from_compaction (messages):
819 """The messages the check compares: from the last applied compaction block's message on.
820 Returns (messages, start index or None when there is no compaction block)."""
821 s = compaction_start(messages)
822 return ((messages or [])[s:] if s is not None else (messages or [])), s
823
824
825 def _signed_signature (message):
826 """The signature of the first compaction block with a summary in a canonical message's content, or None."""
827 c = message.get( "content" ) if isinstance (message, dict ) else None
828 return next ((b.get( "signature" ) for b in (c if isinstance (c, list ) else [])
829 if isinstance (b, dict ) and b.get( "type" ) == "compaction" and b.get( "signature" ) and b.get( "content" ) is not None ), None )
830
831
832 def _pick_compaction_request (pending, A_all, kept_at):
833 """Which of the pending compaction requests the adopting request took its block from: one whose messages are the
834 first messages of the earlier request (or the earlier request's messages plus the reply), and, when a kept thinking
835 block fixes the dropped range (kept_at), whose message count equals it; the latest such request wins. Returns
836 (label, body, n) or (None, None, 0)."""
837 tied = []
838 a_fp = [fp(x[ "msg" ]) for x in A_all]
839 for label, body in pending or []:
840 C_msgs = canon_messages(body.get( "messages" )) if isinstance (body.get( "messages" ), list ) else []
841 n = len (C_msgs)
842 c_fp = [fp(x[ "msg" ]) for x in C_msgs]
843 # the request carried the first n messages of the earlier request, or every message of it plus the reply and
844 # any tool results added before the compaction was sent: either way the block replaces a prefix of the earlier
845 # request
846 if n and a_fp and (a_fp[:n] == c_fp or c_fp[: len (a_fp)] == a_fp):
847 tied.append((label, body, n))
848 if kept_at is not None :
849 exact = [t for t in tied if t[ 2 ] == kept_at]
850 if exact:
851 return exact[ - 1 ]
852 return tied[ - 1 ] if tied else ( None , None , 0 )
853
854
855 def diff_pair (reqA, reqB, mint_models = None , compaction_requests = None ):
856 """mint_models: optional {signature: model id that produced the block}, built by run_diff from the
857 capture order. Used only to separate blocks minted by the model now being called from blocks
858 minted by another model, which are judged by the model check rather than this prefix comparison.
859 compaction_requests: the captured compaction requests (beta compact-2026-09-04) not yet adopted, as (label, body),
860 in capture order. When the later request adopts a block, the API checks the kept turns against the system prompt
861 and tools of the compaction request that produced it, not the previous turn's, so that request becomes the
862 reference side, and the adopting request must have dropped exactly the messages it carried. The result then
863 carries adopted_compaction=True and reference_request=<label> (None when no captured request could be tied)."""
864 B_sys = canon_system(reqB.get( "system" ))
865 B_tools = canon_tools(reqB.get( "tools" ), reqB.get( "messages" ))
866 B_msgs = canon_messages(reqB.get( "messages" ))
867 b_start = compaction_start(reqB.get( "messages" ))
868 compaction_note = None
869 boundary_notes, boundary_mismatch, adopted, reference, reference_label, block_altered = [], None , False , reqA, None , False
870 if b_start is not None :
871 # The check restarts at the last applied compaction block. Everything before it is outside the comparison, so
872 # the earlier request's messages before that block are replaced by the later request's own copy (identical by
873 # construction) and the earlier request is aligned on the same compaction message after it. Indices therefore
874 # stay those of the later request as sent, which is what the API's own diagnosis names. If the earlier request
875 # does not carry the block (the compaction arrived in the reply to it), nothing after the boundary existed
876 # before, so the earlier side contributes no prefix to compare against.
877 A_all = canon_messages(reqA.get( "messages" ))
878 signed_blocks = _signed_blocks(reqB.get( "messages" ))
879 target = fp(B_msgs[b_start][ "msg" ])
880 j = next ((i for i, x in enumerate (A_all) if fp(x[ "msg" ]) == target), None )
881 if j is None :
882 # The same signed block with different bytes (a round trip through domain objects trimmed its summary) is an
883 # edit to a signed block, which the API rejects: find it by its signature, where the earlier request sent it
884 # as a message of its own, so the comparison below reports the byte difference instead of treating the
885 # block as new. A block whose signature changed too is caught further down, when nothing else was dropped.
886 sig = _signed_signature(B_msgs[b_start][ "msg" ])
887 j = next ((i for i, x in enumerate (A_all) if sig and _signed_block_alone([x[ "msg" ]]) and _signed_signature(x[ "msg" ]) == sig), None )
888 block_altered = j is not None
889 kept_at = (_kept_alignment(A_all, B_msgs)
890 if j is None and b_start == 0 and _signed_block_alone(reqB.get( "messages" )) else None )
891 if j is None and b_start == 0 and _signed_block_alone(reqB.get( "messages" )):
892 label, creq, n = _pick_compaction_request(compaction_requests, A_all, kept_at)
893 if kept_at == 1 and _signed_block_alone(reqA.get( "messages" )) and creq is None :
894 # The earlier request already started with a signed block sent alone, the later one replaced it with
895 # another block while dropping nothing else, and no captured compaction request ties to it. Compacting
896 # again summarizes the old block and everything after it, so a legitimate new block drops more than one
897 # message: this is the old block re-sent with an altered signature, an edit the API rejects. Compare the
898 # two block messages.
899 j, kept_at, block_altered = 0 , None , True
900 if j is None and b_start == 0 and _signed_block_alone(reqB.get( "messages" )):
901 adopted = True
902 if creq is not None :
903 # The API checks the kept turns against the system prompt and tools the compaction request had, and
904 # the adopting request must drop exactly the messages that request carried (fewer: the model sees
905 # them twice; more: the kept turns no longer follow the summarized messages and their thinking fails).
906 reference, reference_label = creq, label
907 if kept_at is None :
908 # No kept thinking block anchors the dropped range, so take it from the compaction request: the kept
909 # messages are the earlier request's messages after the ones it summarized.
910 kept_at = n
911 if kept_at > n:
912 boundary_mismatch = ( "the adopting request dropped messages[0.. %d ] of the earlier request, but the compaction request "
913 "( %s ) carried only messages[0.. %d ]: the kept turns no longer directly follow the summarized "
914 "messages, so their thinking fails the check" % (kept_at - 1 , label, n - 1 ))
915 elif kept_at < n:
916 boundary_notes.append( "the adopting request dropped messages[0.. %d ] of the earlier request, but the compaction request "
917 "( %s ) carried messages[0.. %d ]: the summarized messages left behind the block are not rejected, "
918 "the model sees them twice (summary, then verbatim), and whether the thinking inside them still "
919 "verifies is not documented -- drop them" % (kept_at - 1 , label, n - 1 ))
920 if len (compaction_requests or []) > 1 :
921 boundary_notes.append( " %d compaction requests were pending; %s is the one whose messages tie to the dropped range"
922 % ( len (compaction_requests), label))
923 elif compaction_requests:
924 boundary_notes.append( "no captured compaction request carries the first messages of the earlier request ( %s pending), so "
925 "the system prompt and tools the summary ran under, and the exact messages it summarized, were "
926 "not verified" % ", " .join(l for l, _b in compaction_requests))
927 else :
928 boundary_notes.append( "no compaction request was captured before this adoption, so the system prompt and tools it ran "
929 "under, and the exact messages it summarized, were not verified" )
930 if j is None and kept_at is not None :
931 # A signed block sent first, in place of the messages it summarizes (the compact-2026-09-04 shape): the
932 # earlier request lacks the block, but the messages behind it are the earlier request's own later messages,
933 # still checked against the summarized ones as they stood when the compaction request was sent. Align the
934 # earlier request on a kept thinking block that it still carries (or on the captured compaction request's
935 # message count), so an edit to a kept turn is reported.
936 A_msgs = B_msgs[: 1 ] + A_all[kept_at:]
937 if kept_at >= len (A_all):
938 compaction_note = ( "the leading compaction block (messages[0] of the later request, which the earlier request does not "
939 "carry) stands in for every message of the earlier request: nothing of it is kept, and the messages "
940 "behind the block are taken as new (no kept thinking ties them to the earlier request)" )
941 else :
942 compaction_note = ( "compared the kept messages behind a leading compaction block (messages[0] of the later request, "
943 "which the earlier request does not carry) against messages[ %d ..] of the earlier request: the block "
944 "stands in for everything before those messages" % kept_at)
945 if reference is not reqA:
946 compaction_note += ( "; system and tools on this pair are compared against the compaction request %s , which is what "
947 "the API checks the kept turns against, not against the earlier turn" % reference_label)
948 elif j is None and b_start == 0 and _has_signed_block((reqB.get( "messages" ) or [ None ])[ 0 ]):
949 A_msgs = B_msgs[:b_start]
950 # with two signed blocks the warning below says why nothing was compared; this note would give a wrong reason
951 compaction_note = None if len (signed_blocks) > 1 else ( "the later request starts with a compaction block the earlier request does not carry, and no kept "
952 "thinking block behind it (in a message of its own, signed) ties the kept messages to the earlier "
953 "request: nothing was compared. The signed-block shape is checked only when the block is sent as a "
954 "message of its own and a kept turn's thinking, or a captured compaction request, ties it to the "
955 "earlier request" )
956 elif j is None :
957 A_msgs = B_msgs[:b_start]
958 compaction_note = ( "compared from the compaction block in messages[ %d ] of the later request, which the earlier request "
959 "does not carry (it arrived in the reply): the check restarts there, so nothing before it is compared" % b_start)
960 else :
961 A_msgs = B_msgs[:b_start] + A_all[j:]
962 compaction_note = ( "compared from the compaction block in messages[ %d ] of the later request (messages[ %d ] of the earlier one): "
963 "the check restarts there, so earlier messages and thinking are outside it" % (b_start, j))
964 else :
965 A_msgs = canon_messages(reqA.get( "messages" ))
966 signed_blocks = []
967 A_sys = canon_system(reference.get( "system" ))
968 A_tools = canon_tools(reference.get( "tools" ), reference.get( "messages" ))
969 ambiguous_note = None
970 if len (signed_blocks) > 1 :
971 # The API takes exactly one signed block with content per request (a duplicated block is a 400), so this
972 # comparison describes a request the API rejects outright; reported as a warning, the way a broken chain is.
973 ambiguous_note = ( " %d signed compaction blocks with content ( %s ): the API accepts one per request and rejects this "
974 "request with a 400 -- send only the block the last compaction returned"
975 % ( len (signed_blocks), ", " .join(signed_blocks)))
976
977 sys_notes = diff_system(A_sys, B_sys)
978 tool_notes, set_changed = diff_tools(A_tools, B_tools)
979 msg = diff_messages(A_msgs, B_msgs)
980 if boundary_mismatch and not (msg and msg.get( "kind" )):
981 # Too many messages dropped at adoption: reported as the kept turns' thinking failing, the way the API would
982 msg = { "kind" : "blocks_removed" , "pattern" : "tail_kept" , "changed_validated" : "messages.1" , "notes" : [boundary_mismatch],
983 "first_changed_msg" : 1 , "thinking_notes" : list ((msg or {}).get( "thinking_notes" , []))}
984 elif boundary_mismatch:
985 # every kept block fails from the boundary, whatever the edit inside the kept turns
986 msg.setdefault( "notes" , []).append(boundary_mismatch)
987 msg[ "first_changed_msg" ], msg[ "first_changed_wire" ] = 1 , None
988 if block_altered and msg and msg.get( "kind" ):
989 msg.setdefault( "notes" , []).append( "messages[0] is the signed compaction block of the earlier request with different bytes: the API "
990 "rejects the whole request (a 400 whose error.details.error_code starts with compaction_) rather than dropping "
991 "thinking, so the block counts on this row are what a corrected request would replay" )
992
993 in_check = [(i, w) for i, w in thinking_positions(B_msgs) if b_start is None or i >= b_start] # blocks the check can judge
994 replayed = len (in_check)
995 demoted = []
996 if adopted and replayed == 0 and (sys_notes or tool_notes):
997 # The block stands first with no kept thinking behind it (a full compaction, or kept turns without thinking),
998 # so a changed system prompt or tool set has nothing to invalidate: the declared boundary the recipe
999 # recommends (compact everything, then change), accepted by the API. Reported as a note, not an edit.
1000 demoted = [ " %s changed at a compaction boundary with no kept thinking behind the block: accepted by the API, "
1001 "nothing to fail ( %s )" % ( " and " .join(s for s, n in (( "system" , sys_notes), ( "tools" , tool_notes)) if n),
1002 "; " .join(sys_notes + tool_notes))]
1003 sys_notes, tool_notes = [], []
1004
1005 sections = []
1006 if sys_notes:
1007 sections.append( "system" )
1008 if tool_notes:
1009 sections.append( "tools" )
1010 if msg and msg.get( "kind" ):
1011 sections.append( "messages" )
1012
1013 result = { "replayed_thinking_blocks" : replayed, "sections" : sections, "notes" : [], "thinking_notes" : []}
1014 if b_start is not None :
1015 result[ "compaction_boundary" ] = b_start
1016 if adopted:
1017 result[ "adopted_compaction" ] = True
1018 result[ "reference_request" ] = reference_label
1019 if msg:
1020 result[ "thinking_notes" ] = list (msg.get( "thinking_notes" , []))
1021 if ambiguous_note:
1022 result[ "thinking_notes" ].append(ambiguous_note)
1023 extra_notes = demoted + boundary_notes
1024 if B_tools[ 1 ] and _carries_tool_changes(reqB.get( "messages" )):
1025 extra_notes.append( "the compaction block carries tool_changes, which this script does not read: an edit to a "
1026 "deferred tool that only the summarized messages named is reported on the first request "
1027 "that carries the block and not on later ones" )
1028 # The model is not part of the compared prefix, so a model change is NOT a prefix edit and never
1029 # turns a match into a mismatch. It is reported separately because the API runs a second, model
1030 # check on every replayed block, whose drops carry reason model_binding_mismatch.
1031 model_a, model_b = reqA.get( "model" ), reqB.get( "model" )
1032 if model_a and model_b and model_a != model_b:
1033 result[ "model_switch" ] = { "from" : model_a, "to" : model_b}
1034 if replayed:
1035 model_note = (
1036 "model switch: %s -> %s with %d replayed thinking block(s). Not a prefix edit (the model is not part of what the "
1037 "signature binds), so this pair still reads as %s for the prefix check; the model check decides separately whether "
1038 " %s can read blocks produced by %s (a drop shows as reason=model_binding_mismatch, not prefix_binding_mismatch)."
1039 % (model_a, model_b, replayed, "MATCH" if not sections else "MISMATCH" , model_b, model_a))
1040 if sections:
1041 model_note += ( " If %s cannot read blocks produced by %s (a switch to an older model), the model check drops them as "
1042 "model_binding_mismatch before the prefix check sees them, so this edit goes unreported on this request and "
1043 "surfaces on the next request that runs on a model that can read them; if it can read them (a switch to a "
1044 "newer model), the prefix check applies now." % (model_b, model_a))
1045 else :
1046 model_note = "model switch: %s -> %s (no replayed thinking block, so nothing for the model check to read)" % (model_a, model_b)
1047 else :
1048 model_note = None
1049
1050 if not sections:
1051 result[ "verdict" ] = "match"
1052 if msg and msg.get( "branch" ):
1053 result[ "notes" ] = msg.get( "notes" , [])
1054 result[ "branch" ] = True
1055 if result[ "thinking_notes" ]:
1056 result[ "verdict" ] = "chain-warning"
1057 if model_note:
1058 result[ "notes" ].append(model_note)
1059 if compaction_note:
1060 result[ "notes" ].append(compaction_note)
1061 result[ "notes" ].extend(extra_notes)
1062 return result
1063
1064 result[ "verdict" ] = "mismatch"
1065 result[ "notes" ] = sys_notes + tool_notes + (msg.get( "notes" , []) if msg and msg.get( "kind" ) else [])
1066 if model_note:
1067 result[ "notes" ].append(model_note)
1068 if compaction_note:
1069 result[ "notes" ].append(compaction_note)
1070 result[ "notes" ].extend(extra_notes)
1071 # kind / pattern in the API's words
1072 if len (sections) == 1 :
1073 if sections == [ "system" ]:
1074 result.update( kind = "system_changed" , pattern = "system_rerendered" , changed_validated = "system. %s " % next ((n.split( "[" )[ 1 ].split( "]" )[ 0 ] for n in sys_notes if n.startswith( "system[" )), "0" ))
1075 elif sections == [ "tools" ]:
1076 result.update( kind = "tools_changed" , pattern = "tool_set_changed" if set_changed else "tool_schema_changed" ,
1077 changed_validated = "tools" )
1078 else :
1079 result.update( kind = msg[ "kind" ], pattern = msg[ "pattern" ], changed_validated = msg.get( "changed_validated" ),
1080 position = msg.get( "position" ))
1081 else :
1082 result[ "kind" ] = "multiple"
1083 if "messages" in sections:
1084 result[ "pattern" ] = msg[ "pattern" ]
1085 result[ "changed_validated" ] = msg.get( "changed_validated" )
1086 result[ "position" ] = msg.get( "position" )
1087 else :
1088 result[ "pattern" ] = "system_and_tools_changed"
1089 result[ "changed_validated" ] = "system" if "system" in sections else "tools"
1090 for k in ( "items_removed" , "items_inserted" , "items_modified" , "messages_delta" ):
1091 if msg and k in msg:
1092 result[k] = msg[k]
1093 # which replayed thinking blocks would fail: every one if system/tools changed,
1094 # else every one at or after the first changed message (request N+1 coordinates)
1095 tp = in_check
1096 if "system" in sections or "tools" in sections:
1097 failing = tp
1098 else :
1099 fc = msg.get( "first_changed_msg" , 0 ) # same index in request N+1: later messages moved up to it
1100 fw = msg.get( "first_changed_wire" ) # (msg index, wire block index) for an in-place edit
1101 if fw and fw[ 0 ] == fc and fw[ 1 ] >= 0 :
1102 failing = [(i, w) for i, w in tp if i > fc or (i == fc and w > fw[ 1 ])]
1103 else :
1104 failing = [(i, w) for i, w in tp if i >= fc]
1105 # blocks minted by a model other than the one this request calls are judged by the model check, and their
1106 # prefix record belongs to the model that minted them; count them apart so the expectation is not overstated
1107 if mint_models:
1108 sig_at = {(i, w): s for i, x in enumerate (B_msgs) for w, s in x[ "sigs" ]}
1109 own, foreign = [], {}
1110 for pos in failing:
1111 m = mint_models.get(sig_at.get(pos))
1112 if m and reqB.get( "model" ) and _model_key(m) != _model_key(reqB.get( "model" )):
1113 foreign[m] = foreign.get(m, 0 ) + 1
1114 else :
1115 own.append(pos)
1116 if foreign:
1117 result[ "thinking_blocks_minted_by_other_models" ] = foreign
1118 result[ "notes" ].append( " %d replayed thinking block(s) minted by another model ( %s ) are not counted below: the model check "
1119 "decides whether %s reads them, and their prefix record belongs to the model that produced them."
1120 % ( sum (foreign.values()), ", " .join( " %s x %d " % kv for kv in sorted (foreign.items())), reqB.get( "model" )))
1121 failing = own
1122 result[ "thinking_blocks_that_would_fail" ] = len (failing)
1123 result[ "first_failing_thinking_msg" ] = failing[ 0 ][ 0 ] if failing else None
1124 return result
1125
1126
1127
1128 def diff_compaction_request (prev, creq):
1129 """Check a captured compaction request (beta compact-2026-09-04) against the conversation turn before it. The API
1130 verifies the kept turns' thinking only while the system prompt and the tools other than defer_loading ones match
1131 the compaction request, so a summarizer prompt or a trimmed tool set on that request fails every kept block at
1132 adoption: reported as a mismatch here, where it is caused (a note when the request summarizes the whole turn, since
1133 nothing of it is kept). Its messages should be the first messages of the turn before it, or all of them plus the
1134 reply and any tool results added before the compaction was sent (the recipe: compact exactly the messages of a
1135 request already sent); anything else is a warning. A different model is a note: the block is accepted, and whether
1136 the kept thinking survives is the model check's call."""
1137 result = { "compaction_request" : True , "replayed_thinking_blocks" : 0 , "sections" : [], "notes" : [], "thinking_notes" : []}
1138 if not isinstance (creq.get( "messages" ), list ) or not isinstance (prev.get( "messages" ), list ):
1139 result[ "thinking_notes" ].append( "the compaction request or the turn before it has no messages list: not checked" )
1140 result[ "verdict" ] = "chain-warning"
1141 return result
1142 P_msgs, C_msgs = canon_messages(prev.get( "messages" )), canon_messages(creq.get( "messages" ))
1143 sys_notes = diff_system(canon_system(prev.get( "system" )), canon_system(creq.get( "system" )))
1144 tool_notes, set_changed = diff_tools(canon_tools(prev.get( "tools" ), prev.get( "messages" )),
1145 canon_tools(creq.get( "tools" ), creq.get( "messages" )))
1146 n = len (C_msgs)
1147 p_fp, c_fp = [fp(x[ "msg" ]) for x in P_msgs], [fp(x[ "msg" ]) for x in C_msgs]
1148 if n == 0 :
1149 result[ "thinking_notes" ].append( "the compaction request carries no messages: the API rejects a request with nothing to summarize" )
1150 elif n < len (P_msgs) and p_fp[:n] == c_fp:
1151 result[ "notes" ].append( "summarizes messages[0.. %d ] of the previous turn; messages[ %d ..] would be kept behind the block" % (n - 1 , n))
1152 elif n == len (P_msgs) and p_fp == c_fp:
1153 result[ "notes" ].append( "summarizes every message of the previous turn; nothing of it is kept behind the block (turns taken "
1154 "while the summary ran are kept, and are checked at adoption)" )
1155 elif P_msgs and c_fp[: len (P_msgs)] == p_fp:
1156 result[ "notes" ].append( "summarizes every message of the previous turn and the %d message(s) that followed it (the reply and "
1157 "any tool results): nothing of the previous turn is kept behind the block" % (n - len (P_msgs)))
1158 else :
1159 result[ "thinking_notes" ].append( "the compaction request's %d message(s) are neither the first messages of the previous turn "
1160 "(which has %d ) nor all of them plus what followed: compact exactly the messages of a request "
1161 "already sent, so the kept turns directly follow the summarized ones" % (n, len (P_msgs)))
1162 if prev.get( "model" ) and creq.get( "model" ) and prev[ "model" ] != creq[ "model" ]:
1163 result[ "model_switch" ] = { "from" : prev[ "model" ], "to" : creq[ "model" ]}
1164 result[ "notes" ].append( "the compaction request runs on %s , the conversation on %s : the API accepts the block on any model "
1165 "that supports the beta, but thinking in the turns kept after the block stays valid only if every "
1166 "compaction request since it was produced ran on a model with preserved thinking, and only on a "
1167 "model that can read it. This script cannot check either, so a MATCH on the pair that follows does "
1168 "not cover them -- if unsure, send compaction requests to the conversation's model"
1169 % (creq[ "model" ], prev[ "model" ]))
1170 sections = [s for s, notes in (( "system" , sys_notes), ( "tools" , tool_notes)) if notes]
1171 if sections and P_msgs and n >= len (P_msgs) and c_fp[: len (P_msgs)] == p_fp:
1172 result[ "notes" ].append( "the %s on the compaction request %s from the conversation's, but nothing of the previous turn is kept "
1173 "behind the block, so no kept thinking of this turn fails; thinking in turns taken while the summary "
1174 "ran would fail ( %s )" % ( " and " .join(( "system prompt" if s == "system" else "tool set" ) for s in sections),
1175 "differ" if len (sections) > 1 else "differs" , "; " .join(sys_notes + tool_notes)))
1176 sections = []
1177 if not sections:
1178 result[ "verdict" ] = "chain-warning" if result[ "thinking_notes" ] else "match"
1179 return result
1180 result[ "sections" ] = sections
1181 result[ "verdict" ] = "mismatch"
1182 result[ "notes" ] = sys_notes + tool_notes + result[ "notes" ] + [
1183 "the compaction request ran under a different %s than the conversation: the kept turns' thinking verifies only "
1184 "while system and the non-deferred tools match the compaction request, so every kept block fails at adoption. Send "
1185 "the conversation's own system and tools on the compaction request, and put a summarization prompt in "
1186 "compaction.instructions" % " and " .join(( "system prompt" if s == "system" else "tool set" ) for s in sections)]
1187 if sections == [ "system" ]:
1188 result.update( kind = "system_changed" , pattern = "system_rerendered" ,
1189 changed_validated = "system. %s " % next ((x.split( "[" )[ 1 ].split( "]" )[ 0 ] for x in sys_notes if x.startswith( "system[" )), "0" ))
1190 elif sections == [ "tools" ]:
1191 result.update( kind = "tools_changed" , pattern = "tool_set_changed" if set_changed else "tool_schema_changed" , changed_validated = "tools" )
1192 else :
1193 result.update( kind = "multiple" , pattern = "system_and_tools_changed" , changed_validated = "system" )
1194 kept = [i for i, _w in thinking_positions(P_msgs) if i >= n]
1195 result[ "thinking_blocks_that_would_fail" ] = len (kept)
1196 result[ "first_failing_thinking_msg" ] = (kept[ 0 ] - n + 1 ) if kept else None
1197 return result
1198
1199
1200 # ----------------------------------------------------------------------------- output
1201
1202
1203 def header_line (r):
1204 parts = [ "kind= %s " % r.get( "kind" ), "pattern= %s " % r.get( "pattern" )]
1205 if r.get( "sections" ):
1206 parts.append( "sections= %s " % "," .join(r[ "sections" ]))
1207 if r.get( "changed_validated" ):
1208 parts.append( "changed_validated= %s " % r[ "changed_validated" ])
1209 if r.get( "position" ):
1210 parts.append( "position= %s " % r[ "position" ])
1211 if r.get( "model_switch" ):
1212 parts.append( "model_switch= %s -> %s " % (r[ "model_switch" ][ "from" ], r[ "model_switch" ][ "to" ]))
1213 for k in ( "items_removed" , "items_inserted" , "items_modified" ):
1214 if k in r:
1215 parts.append( " %s = %d " % (k.replace( "items_" , "" ), r[k]))
1216 if "messages_delta" in r:
1217 parts.append( "messages_delta= %d " % r[ "messages_delta" ])
1218 return "; " .join(parts)
1219
1220
1221 def print_pair (la, lb, r):
1222 tag = { "match" : "MATCH" , "mismatch" : "MISMATCH" , "chain-warning" : "CHAIN-BREAK" }[r[ "verdict" ]]
1223 print ( " %s -> %s : %s " % (la, lb, tag + ( " (compaction request)" if r.get( "compaction_request" ) else "" )))
1224 if r[ "verdict" ] == "mismatch" :
1225 print ( " " + header_line(r))
1226 elif r.get( "model_switch" ):
1227 print ( " model_switch= %s -> %s " % (r[ "model_switch" ][ "from" ], r[ "model_switch" ][ "to" ]))
1228 if r.get( "compaction_request" ):
1229 if r[ "verdict" ] == "mismatch" :
1230 n = r.get( "thinking_blocks_that_would_fail" , 0 )
1231 if n:
1232 print ( " kept thinking blocks of the previous turn that fail at adoption: %d (first one at messages[ %s ] of the adopting "
1233 "request); thinking in turns taken while the summary ran fails too" % (n, r.get( "first_failing_thinking_msg" )))
1234 else :
1235 print ( " kept thinking blocks of the previous turn that fail at adoption: none; thinking in turns taken while the "
1236 "summary ran fails" )
1237 for n in r.get( "notes" , []):
1238 print ( " " + n)
1239 for n in r.get( "thinking_notes" , []):
1240 print ( " ! " + n)
1241 return
1242 print ( " replayed thinking blocks in the later request: %d " % r[ "replayed_thinking_blocks" ]
1243 + ( " (a match proves nothing about binding when this is 0)" if r[ "replayed_thinking_blocks" ] == 0 else "" ))
1244 if r[ "verdict" ] == "mismatch" :
1245 n = r.get( "thinking_blocks_that_would_fail" , 0 )
1246 if n:
1247 print ( " thinking blocks that would be dropped/rejected: at most %d (first one in messages[ %s ]; a block minted before a later-referenced tool or tool_addition entered the prefix is not bound to it, so the API's count can be lower)" % (n, r.get( "first_failing_thinking_msg" )))
1248 else :
1249 print ( " no replayed thinking block sits after the change -- nothing would be dropped on THIS request, but the edit is still a bug" )
1250 for n in r.get( "notes" , []):
1251 print ( " " + n)
1252 for n in r.get( "thinking_notes" , []):
1253 print ( " ! " + n)
1254
1255
1256 def run_diff (args):
1257 loaded = load_requests(args.paths)
1258 convs = []
1259 for _l, _b, c in loaded:
1260 if c not in convs:
1261 convs.append(c)
1262 multi = len (convs) > 1
1263 results, compared = [], 0
1264 for conv in convs:
1265 captured = [(l, b) for l, b, c in loaded if c == conv]
1266 display = os.path.basename(conv) or conv or "(command line)"
1267 n_turns = sum ( 1 for _l, b in captured if not is_compaction_request(b))
1268 if multi and n_turns < 2 :
1269 sys.stderr.write( " %s : only %d conversation turn(s), nothing to compare within it (pairs never cross conversations) \n "
1270 % (display, n_turns))
1271 continue
1272 if multi and not args.json:
1273 print ( "== conversation %s ==" % display)
1274 before = len (results)
1275 diff_conversation(captured, args, results)
1276 if multi:
1277 for r in results[before:]:
1278 r[ "conversation" ] = display
1279 compared += 1
1280 if not compared:
1281 raise SystemExit ( "need at least two request bodies of one conversation (a .jsonl capture, two .json files, or a directory)" )
1282 if args.json:
1283 print (json.dumps(results, indent = 2 ))
1284 n_bad = sum ( 1 for r in results if r[ "verdict" ] == "mismatch" )
1285 n_chain = sum ( 1 for r in results if r[ "verdict" ] == "chain-warning" )
1286 if not args.json:
1287 print ( " \n%d pair(s) compared, %d mismatch(es), %d chain break(s)." % ( len (results), n_bad, n_chain))
1288 return 1 if (n_bad or n_chain) else 0
1289
1290
1291 def diff_conversation (captured, args, results):
1292 """Diff the (label, body) turns of ONE conversation in send order, appending a record per pair
1293 to results. Pairs never cross conversations: run_diff calls this once per conversation."""
1294 # A body carrying the top-level compaction field is a compaction request (beta compact-2026-09-04), not a
1295 # conversation turn: its reply is the block. It is checked against the turn before it (diff_compaction_request)
1296 # and then held as the reference for the request that adopts its block; it is never paired as a turn itself.
1297 # The capture is read in send order: a compaction request is expected before the request that adopts its block.
1298 bodies, comp_after = [], {}
1299 for l, b in captured:
1300 if is_compaction_request(b):
1301 comp_after.setdefault( len (bodies) - 1 , []).append((l, b))
1302 else :
1303 bodies.append((l, b))
1304 pending = list (comp_after.pop( - 1 , []))
1305 if pending:
1306 sys.stderr.write( " %d compaction request(s) captured before any conversation turn, not checked against one: %s\n "
1307 % ( len (pending), ", " .join(l for l, _b in pending)))
1308 if len (bodies) < 2 :
1309 raise SystemExit ( "need at least two request bodies (a .jsonl capture, two .json files, or a directory)" )
1310 # which model minted each replayed block: a signature first seen in request N+1 was produced by request N's model
1311 mint_models, seen = {}, set ()
1312 for _l, body in bodies:
1313 for m in body.get( "messages" ) or []:
1314 for blk in (m.get( "content" ) if isinstance (m.get( "content" ), list ) else []):
1315 if isinstance (blk, dict ) and blk.get( "type" ) == "thinking" and blk.get( "signature" ):
1316 seen.add(blk[ "signature" ])
1317 seen_so_far = set ()
1318 for idx, (_l, body) in enumerate (bodies):
1319 for m in body.get( "messages" ) or []:
1320 for blk in (m.get( "content" ) if isinstance (m.get( "content" ), list ) else []):
1321 if isinstance (blk, dict ) and blk.get( "type" ) == "thinking" and blk.get( "signature" ) and blk[ "signature" ] not in seen_so_far:
1322 seen_so_far.add(blk[ "signature" ])
1323 if idx > 0 and bodies[idx - 1 ][ 1 ].get( "model" ):
1324 mint_models[blk[ "signature" ]] = bodies[idx - 1 ][ 1 ][ "model" ]
1325 last_on_model = {}
1326 for idx, ((la, a), (lb, b)) in enumerate ( zip (bodies, bodies[ 1 :])):
1327 for lc, c in comp_after.get(idx, []):
1328 rc = diff_compaction_request(a, c)
1329 results.append({ "from" : la, "to" : lc, ** rc})
1330 if not args.json:
1331 print_pair(la, lc, rc)
1332 pending.append((lc, c))
1333 r = diff_pair(a, b, mint_models, compaction_requests = pending)
1334 pending_here = list (pending)
1335 if r.get( "adopted_compaction" ):
1336 # the adopted request and any older one are done with; a request captured after it stays pending. When no
1337 # captured request tied to this adoption, every pending one is older than the block and is dropped too.
1338 used = r.get( "reference_request" )
1339 cut = next ((i for i, (l, _c) in enumerate (pending) if l == used), len (pending) - 1 )
1340 pending = pending[cut + 1 :]
1341 if a.get( "model" ):
1342 last_on_model[_model_key(a[ "model" ])] = idx
1343 # on a switch pair with an edit, also compare against the last request that ran on the model now being
1344 # called: the prefix check judges a block against the prefix it was minted under, so a prompt or tool set
1345 # that is the same on every request of that model is not reported by the API even though this pair differs
1346 if (r.get( "model_switch" ) and r[ "verdict" ] == "mismatch" and _model_key(b.get( "model" )) in last_on_model
1347 and not r.get( "reference_request" )):
1348 # (when the pair's reference is a compaction request, its system/tools difference is against that request,
1349 # not the previous turn, so the per-model-prompt downgrade below does not apply)
1350 k = last_on_model[_model_key(b[ "model" ])]
1351 same = diff_pair(bodies[k][ 1 ], b, mint_models, compaction_requests = pending_here)
1352 if not same.get( "sections" ) and not same.get( "thinking_notes" ):
1353 # The API judges each block against the request that produced it, not against the previous request.
1354 # The last request on this model is an unchanged prefix of this one, so every block this model minted
1355 # is replayed under the prefix it was minted with: the API reports nothing here. Downgrade the verdict
1356 # and keep the previous-request difference as a note.
1357 r[ "same_model_prefix_unchanged" ] = bodies[k][ 0 ]
1358 r[ "previous_request_diff" ] = header_line(r)
1359 r[ "verdict" ] = "match"
1360 r[ "thinking_blocks_that_would_fail" ] = 0
1361 r[ "first_failing_thinking_msg" ] = None
1362 r[ "notes" ] = [n for n in r[ "notes" ] if not n.startswith( "model switch:" )] + [
1363 "differs from the previous request ( %s ), but that request ran on another model: against %s , the last request "
1364 "that ran on %s , nothing the check compares has changed, so the blocks this model produced are replayed under the "
1365 "prefix they were minted with and the API reports nothing on this request (the other model's turns still lose "
1366 "this model's blocks through the model check, and blocks the other model produced are judged against its own "
1367 "requests, where that model records a prefix at all). A prompt or tool set that is a pure function of the model called "
1368 "is stable on each model's own turns; confirm with the probe." % (r[ "previous_request_diff" ], bodies[k][ 0 ], b[ "model" ])]
1369 results.append({ "from" : la, "to" : lb, ** r})
1370 if not args.json:
1371 print_pair(la, lb, r)
1372 for lc, c in comp_after.get( len (bodies) - 1 , []):
1373 rc = diff_compaction_request(bodies[ - 1 ][ 1 ], c)
1374 results.append({ "from" : bodies[ - 1 ][ 0 ], "to" : lc, ** rc})
1375 if not args.json:
1376 print_pair(bodies[ - 1 ][ 0 ], lc, rc)
1377 pending.append((lc, c))
1378 if pending:
1379 sys.stderr.write( " %d compaction request(s) whose block no later request in the capture adopted: %s\n "
1380 % ( len (pending), ", " .join(l for l, _c in pending)))
1381
1382
1383 # ----------------------------------------------------------------------------- scan mode
1384
1385 # Each lead: (title, line pattern, context pattern or None, context window in lines, scope)
1386 # scope "toolfn" = only inside a function whose name mentions tool (append/extend/push leads).
1387 LEADS = [
1388 ( "system prompt re-rendered (time, counters, live state in the system prompt)" ,
1389 r " ^ (?! . * \b( log | logger | LOG | logging | audit | print | console \. | metrics | trace )\b ) . * ( datetime \. ( now | utcnow | today ) \( | date \. today \( | time \. ( time | strftime | ctime ) \( | new Date \( | Date \. now \( | time \. Now \( | Time \. now | DateTime \. ( Now | UtcNow ) | moment \( | dayjs \( | \b strftime \( | \b toISOString \( ) " ,
1390 r " ( system | SYSTEM | prompt | Prompt | PROMPT | template | TEMPLATE | instructions | persona | PERSONA | preamble | boilerplate | BOILERPLATE | opener | render | \. format \( | f \" | f' | \$\{ | % \s * [\w {( ]) " , 8 , None ),
1391 ( "system prompt re-rendered (environment, ids or random values interpolated into prompt text)" ,
1392 r " ( os \. environ | process \. env | getenv \( | uuid | Math \. random \( | random \. choice | request_id | session_id | user_id | cwd \(\) | getcwd \(\) | hostname ) " ,
1393 r " ( system | prompt | instructions | persona )\b. * ( = | \+ | format | f \" | f' | \$\{ | \. replace \( ) | ( \. format \( | f \" | f' | \$\{ ). * ( system | prompt | instructions ) " , 0 , "sameline" ),
1394 ( "tool set changed mid-conversation (tools list mutated after the first request)" ,
1395 r " (\b tools ? \b\s * ( \. append | \. push | \. extend | \. remove | \. pop | \. splice | \. filter | \. concat | \s * = \s * \[ | \s * \+ = | \s * = \s * \w + \s * \+ ) | \[ [ ' \" ] tools [ ' \" ] \] \s * = | \. tools \s * = | del \s + tools \[ | list_tools \( | listTools \( | get_tools \( | getTools \( | available_tools | plugin_tools | mcp . * ( connect | tools ) | ( \+ = | \. append \( | \. extend \( | \. push \( | \. concat \( ). * ( functions \(\) | tools \(\) | plugin | mcp | connector )) " ,
1396 None , 0 , None ),
1397 ( "tool set changed mid-conversation (a list built inside a tool-related function)" ,
1398 r " ( \. append \( | \. extend \( | \. push \( | \. concat \( ) " , None , 0 , "toolfn" ),
1399 ( "tool definitions re-rendered (same names, description/schema text rebuilt each request)" ,
1400 r " ( description [ \" ' ] ? \s * [ := ]\s * f [ \" ' ] | description [ \" ' ] ? \s * [ := ]. * ( \$\{ | \. format \( | \+ \s * \w | % \s * [\w ( ]) | input_schema [ \" ' ] ? \s * [ := ]. * ( \$\{ | \. format \( ) | render_tool_ ( def | definition | schema | description ) s ? \( | describe_tools ? \( | tool_description \( | build_tool_ ( def | definition | schema ) s ? \( ) " ,
1401 None , 0 , None ),
1402 ( "older turns dropped or collapsed (sliding window / keep-last / compaction)" ,
1403 r " ([\w . ] + \s * \[ \s * - \s * [\w . ] + \s * : \s * \] | [\w . ] + \s * \[ \s * [\w . ] + \s * : \s * \] \s * $ | \. slice \( \s * - \s * [\w . ] + \s * \) | \. splice \( \s * 0 \s * , | len \( [\w . ] + \) \s * - \s * [\w . ] + \s * : | \. shift \(\) | \. pop \( \s * 0 \s * \) | del \s + [\w . ] + \[ \s * 0 | keep_last | keepLast | max_turns | maxTurns | max_messages | maxMessages | max_history | maxHistory | window_size | windowSize | \b window \b | summar ( y | ize | ise | ies )\b | compact | recap | condense | \b fold \b | digest | gist ) " ,
1404 r "messages | history | turns | conversation | transcript | context | memory | chat | thread | dialogue | exchange | convo" , 4 , None ),
1405 ( "opening message rewritten (message 0 rebuilt from current state each request)" ,
1406 r " ([\w . ] + \s * \[ \s * 0 \s * \] \s * ( = | \[ . content . \] \s * = | \. content \s * = ) | first_message | firstMessage | opening_message | context_header | contextHeader | build_context \( | buildContext \( | initial_context | initialContext ) " ,
1407 r "messages | history | conversation | transcript | turns | thread | dialogue | exchange | convo" , 3 , None ),
1408 ( "old tool results rewritten (truncated / cleared after they were sent)" ,
1409 r " ( \[ \s * : \s * \d | \[ \s * : \s * [ A-Z_ ] + \s * \] | truncat | \.\.\.\[ | \. slice \( 0 | shorten | elide | clear_old | micro . ? compact | \[ truncated \] | \[\" ']content [ \" ' ] \] \s * = | \. content \s * = ) " ,
1410 r "tool_result | toolResult | tool_results | results ? \b | output" , 3 , None ),
1411 ( "per-turn reminder injected then stripped" ,
1412 r " ( system-reminder | system_reminder | <reminder | reminder> | \b remind ( er ) ? s ? \b. * ( strip | remove | re \. sub | replace | filter | del \b | pop ) | ( strip | remove | re \. sub | replace | filter ) \( . * remind | \b note \b. * ( strip | remove | replace \( )) " ,
1413 None , 0 , None ),
1414 ( "message content rewritten in place with a regex or replace (generic)" ,
1415 r " ( re \. sub \( | \. replace \( | re \. subn \( | \. replaceAll \( ) " ,
1416 r " ( content | text | message | msg | turn )\b. * ( re \. sub \( | \. replace \( | \. replaceAll \( ) | ( re \. sub \( | \. replace \( ). * ( content | text | message | msg ) " , 0 , "sameline" ),
1417 ( "thinking blocks removed or reordered (only a contiguous window of the original blocks replays; a middle gap or a reorder fails)" ,
1418 r " ([ \" ' ] thinking [ \" ' ]. * ( filter | remove | del \b | not in | != | strip | pop ) | ( filter | remove | del \b | strip | != | not in ). * [ \" ' ] thinking [ \" ' ] | redacted_thinking | \b signature \b) " ,
1419 None , 0 , None ),
1420 ( "history re-serialised (lossy round trip through the app's own message model)" ,
1421 r " ( json \. loads \( json \. dumps \( | JSON \. parse \( JSON \. stringify \( | to_dict \(\) | from_dict \( | from_json \( | as_json \( | to_json \( | fromJSON \( | toJSON \( | model_dump \( | model_dump_json \( | model_validate \( | model_validate_json \( | marshal | pydantic | dataclasses \. asdict | \. strip \(\) | normalize | normalise | sort_keys | indent= | \. dict \(\) | parse_obj \( | default=str ) " ,
1422 r "messages | history | content | transcript | turns | conversation | thread | dialogue | exchange | convo" , 4 , None ),
1423 ( "media in earlier turns dropped or re-encoded (client media cap, resize, URL re-sign)" ,
1424 r " ( max_images | maxImages | image_cap | image_limit | strip . * image | image . * ( strip | drop | remove | resize | downscale | cap \b) | attachments ? \b. * ( \[ \s * : | \[ - | filter | remove | drop | cap ) | presign | signed_url | signedUrl | expires_in | \b resize \( | thumbnail | ( kind | type )\s * !== ? \s * [ \" ' ]( image | document ) | ( filter | \b for \b. * \b in \b | \b if \b | \b not \b | && | \|\| ). * ( kind | type )\s * === ? \s * [ \" ' ]( image | document )[ \" ' ] | pictures ?| photos ? \b. * ( drop | remove | stale | old )) " ,
1425 None , 0 , None ),
1426 # --- structural leads (shape of the code rather than names) ---
1427 ( "system prompt produced by a call at request time (read that function for anything per-request: time, user, state)" ,
1428 r " [ \" ' ] ? system [ \" ' ] ? \s * [ := ]\s *(?! [ \" ' \[ f ] ) [\w . ] + \( " ,
1429 None , 0 , None ),
1430 ( "a time or date value passed into a prompt builder or template (the rendered prompt changes every call)" ,
1431 r " ( \. format \( [ ^) ] * \b\w * ( now | today | date | time | clock | stamp )\w * \s * = | \b( prompt | system | rules | instructions | persona | preamble | render | compose | build | banner | header | brief | opener )\w * \( [ ^) ] * \b\w * ( now | today | clock | stamp )\w * \(\) ) " ,
1432 None , 0 , None ),
1433 ( "opening message produced by a call and placed first (rebuilt from current state each request)" ,
1434 r " ( \[ \s * \w[\w . ] * \( [ ^) ] * \) \s * , \s * ( \.\.\. | \* ) | \b( opening | opener | first | banner | preface | header | brief )\w * \( [ ^) ] * \) \s * , \s * ( \.\.\. | \* ) | ^\s * [\w . ] + \s * \[ \s * 0 \s * \] \s * = \s * \{ ) " ,
1435 None , 0 , None ),
1436 ( "a placeholder written into a content block (an earlier block replaced by a caption)" ,
1437 r " [ \" ' ] text [ \" ' ]\s * : \s * [ \" ' ][ ^ \" ' ] * ( omitted | no longer | earlier | elided | removed | truncated | redacted | placeholder | not shown | dropped )[ ^ \" ' ] * [ \" ' ] " ,
1438 None , 0 , None ),
1439 ( "media counted against a budget while walking the history (older images or documents replaced or dropped)" ,
1440 r " \b( seen | count | kept | shown | n_media | n_images )\s * \+ = \s * 1" ,
1441 r "image | document | media | attachment | still | picture | photo | frame | visual" , 3 , None ),
1442 ( "turns removed in a loop until a budget fits (sliding window by tokens)" ,
1443 r " ( \. splice \( \s * \d + \s * , \s * \d + \s * \) | \. shift \(\) | \. pop \( \s * 0 \s * \) | del \s + [\w . ] + \[ \s * \d + \s * \] | \. splice \( \s * \d + \s * , \s * 0 \s * , ) " ,
1444 r "while | budget | tokens ?| fit | limit | estimate | recap | summar | digest | gist | fold" , 3 , None ),
1445 ( "text blocks filtered by prefix (an injected note removed on replay)" ,
1446 r " ( startswith | startsWith ) \( " ,
1447 r "text | content | note | hint | remind" , 0 , "sameline" ),
1448 ]
1449 DEFAULT_EXT = "py,ts,tsx,js,jsx,mjs,go,rb,java,kt,rs,cs,php,scala,swift"
1450 SKIP_DIRS = { ".git" , "node_modules" , "dist" , "build" , "__pycache__" , ".venv" , "venv" , "vendor" , "target" , ".next" , "coverage" , ".claude" }
1451
1452
1453 def run_scan (args):
1454 exts = tuple ( "." + e.strip().lstrip( "." ) for e in args.ext.split( "," ))
1455 root = args.paths[ 0 ]
1456 if not os.path.isdir(root):
1457 raise SystemExit ( "--scan needs a directory (got %s )" % root)
1458 compiled = [(title, re.compile(pat, re.I), re.compile(ctx, re.I) if ctx else None , win, scope) for title, pat, ctx, win, scope in LEADS ]
1459 fn_re = re.compile( r " ^\s * (?: async \s + ) ? (?: def | function | func | fn | fun )\s + (\w + ) | ^\s * (?: export \s + ) ? (?: const | let | var )\s + (\w + )\s * = \s * (?: async \s * ) ? \( | ^\s * (?: public | private | protected | static | \s) * \s * (?: async \s + ) ? (\w + )\s * \( [ ^) ] * \) \s * (?: : \s * [\w <> \[\] |, ] + ) ? \s * \{ " )
1460 hits = {title: [] for title, _, _, _, _ in LEADS }
1461 nfiles = 0
1462 own_dir = os.path.dirname(os.path.abspath( __file__ )) # never scan this skill's own scripts
1463 for dirpath, dirnames, filenames in os.walk(root):
1464 dirnames[:] = [d for d in dirnames if d not in SKIP_DIRS and os.path.abspath(os.path.join(dirpath, d)) != own_dir]
1465 if os.path.abspath(dirpath) == own_dir:
1466 continue
1467 for fn in filenames:
1468 if not fn.endswith(exts):
1469 continue
1470 path = os.path.join(dirpath, fn)
1471 try :
1472 with open (path, "r" , encoding = "utf-8" , errors = "replace" ) as f:
1473 lines = f.read().split( " \n " )
1474 except OSError :
1475 continue
1476 nfiles += 1
1477 current_fn = ""
1478 for i, line in enumerate (lines):
1479 m = fn_re.match(line)
1480 if m:
1481 current_fn = next ((g for g in m.groups() if g), "" ) or ""
1482 if len (line) > 400 or line.lstrip().startswith(( "#" , "//" , "*" , "/*" )):
1483 continue
1484 const_array = re.match( r " ^\s * ( export \s + ) ? ( const | let | var | final | static ) ? \s * [ A-Z ][ A-Z0-9_ ] * \s * ( : [ ^= ] + ) ? = \s * \[ " , line)
1485 for title, pat, ctx, win, scope in compiled:
1486 if not pat.search(line):
1487 continue
1488 if const_array and title.startswith( "tool set changed" ):
1489 continue
1490 if scope == "toolfn" and not re.search( r "tool | function | capabilit | abilit | available | enabled | allowed" , current_fn, re.I):
1491 continue
1492 if scope == "sameline" :
1493 if ctx is not None and not ctx.search(line):
1494 continue
1495 elif ctx is not None :
1496 lo, hi = max ( 0 , i - win), min ( len (lines), i + win + 1 )
1497 if not any (ctx.search(lines[q]) for q in range (lo, hi)):
1498 continue
1499 hits[title].append((os.path.relpath(path, root), i + 1 , line.strip()))
1500 print ( "prefix_diff --scan: %d file(s) read under %s " % (nfiles, root))
1501 print ( "These are regex LEADS (places worth reading), not findings. Confirm each one with the" )
1502 print ( "request-pair diff or the API's input_transformations / anthropic-thinking-prefix-mismatch response. \n " )
1503 total = 0
1504 for title, _, _, _, _ in LEADS :
1505 rows = hits[title]
1506 if not rows:
1507 continue
1508 total += len (rows)
1509 print ( "== %s ( %d lead %s )" % (title, len (rows), "" if len (rows) == 1 else "s" ))
1510 for path, ln, src in rows[: args.max_per_cause]:
1511 print ( " %s : %d %s " % (path, ln, src[: 140 ]))
1512 if len (rows) > args.max_per_cause:
1513 print ( " ... %d more" % ( len (rows) - args.max_per_cause))
1514 print ()
1515 if total == 0 :
1516 print ( "no leads matched. That is not proof of compliance -- run the request-pair diff on a capture." )
1517 return 0
1518
1519
1520 def main (argv = None ):
1521 ap = argparse.ArgumentParser( description = __doc__ , formatter_class = argparse.RawDescriptionHelpFormatter)
1522 ap.add_argument( "paths" , nargs = "+" , help = "capture.jsonl | reqA.json reqB.json | directory of captures (or a repository root with --scan)" )
1523 ap.add_argument( "--json" , action = "store_true" , help = "machine-readable output (diff mode)" )
1524 ap.add_argument( "--scan" , action = "store_true" , help = "scan a repository for likely causes instead of diffing requests" )
1525 ap.add_argument( "--ext" , default = DEFAULT_EXT , help = "comma-separated source extensions for --scan" )
1526 ap.add_argument( "--max-per-cause" , type = int , default = 25 )
1527 args = ap.parse_args(argv)
1528 return run_scan(args) if args.scan else run_diff(args)
1529
1530
1531 if __name__ == "__main__" :
1532 sys.exit(main())