CFGFast reports a decoding error for instructions that decode fine, when the lift budget is clamped below one instruction

THIS MESSAGE WAS GENERATED BY AN AUTOMATED PROCESS

CFGFast reports Decoding error occurred at address ... at addresses whose bytes decode perfectly well. The Ijk_NoDecode is manufactured by angr itself: _generate_cfgnode clamps the lifter's byte budget to less than one instruction, and libVEX reports Ijk_NoDecode for any budget shorter than one instruction, which is indistinguishable from a real decoding failure.

Measured on angr 23b470d9f, pyvex 2324a19, vex e3062871, archinfo 06a207f, cle 0e77ade3, binaries fc07821.

The chain, on a RISC-V object

1. CFGFast._add_data_reference (cfg_fast.py:4544, from _collect_data_references → _process_irsb_data_refs) records MemoryData(addr, sort=unknown, size=0) for a constant the lifter reported as a data reference. On RISC-V that constant is often the intermediate value of an auipc — pc + (imm << 12), before the addi that completes the address — so it lands in the middle of .text. At all five sites below, data_addr - insn_addr is an exact multiple of 0x1000.
2. CFGModel._guess_data_type (cfg_model.py:1136-1151) then calls a single printable byte followed by a NUL a MemoryDataSort.String of size 2. That branch has no minimum length.
3. CFGModel.tidy_data_references (cfg_model.py:988) runs seg_list.occupy(addr, 2, "string"). This happens inside the scan loop — cfg_fast.py:2674, in _job_queue_empty before each force-scan step — not only in post-analysis.
4. CFGFast._generate_cfgnode (cfg_fast.py:6030-6033) clamps distance to self._seg_list.next_pos_with_sort_not_in(addr, {"code"}), which is now addr + 2.
5. self._lift(addr, size=2) hands libVEX two bytes. pyvex flattens the panic to size=0, and _generate_cfgnode reports a decoding error and occupies the address as nodecode.

Reproducer

Both objects are tracked in angr/binaries at fc07821, so this needs no new fixture.

A plain proj.analyses.CFGFast() reports a decoding error at each and leaves no CFG node covering the address. Lifting the same bytes with an 8-byte budget gives size=8, Ijk_Boring; with the 2-byte budget angr used, size=0, Ijk_NoDecode. At all five asterisk sites the guessed one-character string sits exactly two bytes past the reported address:

0x43cc90 -> memory_data 0x43cc92 size=2 sort=string content=b'\t'
0x47e5d4 -> memory_data 0x47e5d6 size=2 sort=string content=b'\r'
0x486caa -> memory_data 0x486cac size=2 sort=string content=b'\t'
0x4a2872 -> memory_data 0x4a2874 size=2 sort=string content=b'\t'
0x4bb0d2 -> memory_data 0x4bb0d4 size=2 sort=string content=b'\r'

All six addresses are inside .eh_frame FDE ranges, so they are real code. (find_symbol(..., fuzzy=True) is no evidence here — it returns the nearest preceding symbol, and neither object has a sized symbol covering these addresses.)

It is not RISC-V specific, and the clamp has more than one source

Counting the reported decoding errors on master whose bytes lift cleanly with an 8-byte budget while angr's own recorded budget was shorter than the instruction:

Examples: mipsel/busybox 0x413188, budget 4, bytes 05004014ec89998f lift in 8; x86_64/langdetect_gcc_stripped 0x429b21, budget 1, bytes e94dffffff are a 5-byte jmp rel32.

What shortened the budget differs by object, which is why I do not think there is a one-line fix:

• mips64/libc.so.6 — the segment list holds unknown at the point the budget ran out, from _process_irsb_data_refs's occupy(ref.data_addr, ref.data_size, "unknown"). Same shape as the RISC-V case with a different classifier.
• mipsel/busybox — the budget ran out at a position already occupied as code, so the clamp came from the next-function boundary rather than from data. 4 bytes is arch.max_inst_bytes on MIPS and still too short, because VEX lifts a branch together with its delay slot. A floor at max_inst_bytes would not fix this.
• x86_64/langdetect_gcc_stripped — the budget ran out at a position marked nodecode, i.e. an earlier decoding error propagates into the next one.

Four repairs, implemented and measured; none of them is right

Corpus for the A/Bs: 216 objects from angr/binaries across 20 architectures, whole-binary CFGFast() each, 208 comparable (the rest fail identically on both sides — core files, unsupported architectures, one timeout).

1. Floor distance at arch.max_inst_bytes before lifting. Rejected: it also widens every case where the short budget already lifted fine, so it re-splits blocks corpus-wide. A 2-byte THUMB instruction with distance=2 becomes a two-instruction block at 4. 54 of 208 objects changed and total decoding errors went up, 19416 → 19544.

2. Keep the clamp, and retry the lift only when it failed. Corpus decoding errors 19416 → 19210, but it breaks four tests that exist to encode the opposite contract: test_cfgfast_preclassify_code_as_data and test_cfgfast_reclassify_code_as_data put a MemoryData into the model by hand and assert that no CFG node overlaps it, and test_no_code_nodes_in_export_table asserts the same for a PE export table. It is a policy change, not a bug fix. It is also wrong on the merits: on tests/armel/fpijr_cortexm_console it gained 925 nodes of which 893 overlap a data item angr itself recorded — 4-byte "instructions" walked straight through "\nISR stack overflowed\0" and "Stack pointer corrupted, reset to top of stack\0" in .text. 98.2% of master's nodes on that object are inside a sized STT_FUNC symbol, against 28.9% of the gained ones.

3. Give _guess_data_type's string branch the minimum length its sibling has. _scan_for_printable_strings has required min_length_nullterminated=3 since 2c8a51060; _guess_data_type never got it. This one is clean on the CFG: tests/analyses/cfg/ is identical to master (238 passed), 6,189 phantom entries disappear corpus-wide, decoding errors 19416 → 19385, nodes +27, functions +10, and most changed objects change only memory_data. It is still wrong, because the two functions face different evidence: the scanner is looking at unreferenced bytes, while _guess_data_type is asked about an address the code points at, where a one-character string is ordinary. Across 10 objects and 8 architectures it strips the string classification from 4,288 referenced entries, and 1,005 of those are genuinely long strings that _guess_data_type only saw 1-2 bytes of because max_size — the distance to the next data item — truncated the window. It is visible in decompiled output:

mipsel/busybox 0x483e94, ".rodata", 35 xrefs, bytes 72 00 00 00

  master     memory_data (2, string, b'r')     ->  fp = fopen64(a0, "r");
  with the   memory_data (4, unknown, None)    ->  extern char g_483e94;
  minimum                                          fp = fopen64(a0, &g_483e94);

4. Pass exec_mem_regions at cfg_fast.py:2674. The mid-scan tidy_data_references call omits the argument its post-analysis sibling at cfg_fast.py:3076 passes, so the call that runs inside the scan loop treats executable memory exactly like data. Passing it is a no-op: the exec branch at cfg_model.py:810 leaves max_size at 0, and _guess_data_type has non_zero_max_size = 1024 if max_size == 0 else max_size (cfg_model.py:1096), so 0 means "read 1024 bytes", not "read nothing". Instrumented, the branch is taken and _guess_data_type(0x41c462, 0) still returns ('string', 2).

That overload is what blocks the narrow repair. max_size == 0 means "no room established" to the caller that declines to size an entry in executable memory, and "no limit" to the function it is passed to, and unpicking that is a decision about a field you own rather than something I should guess at.

What I think the question is

A guessed data item can currently veto a decode, and the test suite says that is deliberate for a declared one. MemoryData carries no provenance, so there is no way today to let a caller's declaration win while a guess tidy_data_references invented from an auipc intermediate does not. Is adding that distinction the direction you want, or would you rather _guess_data_type stop classifying inside executable memory, or something else? I have the corpus and the harness and am happy to measure whichever you prefer.

Related

• #6920 — the same family from the other side: _next_code_addr_core's classifiers occupy real code on m68k and SuperH, so the address is never scanned at all. Here the address is scanned and the lift is truncated.
• #6835 — a zero run shorter than one instruction recorded as alignment, eating the front of an AArch64 ldr.
• #6928 and #6837 — what happens after a decoding error, when the scan restarts inside an undecodable instruction. This issue is about a decoding error that should never have been reported.
• #4304 — Cannot lift block with no data (max_bytes <= 0), the same downstream family, never diagnosed.