CFGFast: an ARM GOT displacement in the literal pool is read as a code pointer

THIS MESSAGE WAS GENERATED BY AN AUTOMATED PROCESS

Description

On ARM, CFGFast reads a literal-pool word that is a PIC displacement of _GLOBAL_OFFSET_TABLE_ and takes it for a code pointer, so it starts a function at an address in the middle of another function's instruction stream.

On binaries/tests/armel/tenda-httpd it does that at 0xd4930 and 0xd4c80. Neither is a real function. In the answer that does not contain them -- this binary gives more than one, see below -- neither is even a block start: both are addresses inside blocks that belong to sub_d4368, whose 174 blocks span 0xd4204-0xd4d14.

0xd492c  cmp r0, #0
0xd4930  str r0, [r5, #0x10]     <- gets a function
0xd4934  beq #0xd44f4

0xd4c78  add r3, r3, #1
0xd4c7c  mov r0, r6
0xd4c80  rsb r1, r3, r4          <- gets a function
0xd4c84  mov r2, fp
0xd4c88  str r3, [sp, #0x18c]
0xd4c8c  bl #0xd8798

Each of those two values occurs exactly once in the file, as an ARM literal-pool word, and in both cases the next instruction adds pc to it:

0x2aa7c  ldr r4, [pc, #0x1b4]     ; literal at 0x2ac38 = 0x000d4930
0x2aa80  add r4, pc, r4           ; pc = 0x2aa88  ->  r4 = 0xff3b8

0x2a72c  ldr r4, [pc, #0x1c8]     ; literal at 0x2a8fc = 0x000d4c80
0x2a730  add r4, pc, r4           ; pc = 0x2a738  ->  r4 = 0xff3b8

.got is at 0xff3b8, so both literals are _GLOBAL_OFFSET_TABLE_ - (add_insn + 8). They are offsets, not addresses. .text runs 0xf9b0-0xda5b8, so the raw displacements land inside it by arithmetic coincidence.

Related: #589, which reports literal pools being identified as functions on ARM. That one is about the pool bytes being scanned as code; this is about the value of a pool word being taken as a pointer.

Where it happens

angr/analyses/cfg/cfg_fast.py, _process_irsb_data_ref_inlined_data. The function tries two readings of an inline ARM pointer, and the comment on the second one describes exactly the ldr rX, [pc, #imm] / add rX, pc idiom above. But "attempt 1: a direct pointer" is tried first, succeeds on the raw value because it lands in a readable section, and returns -- so "attempt 2: pc + offset" is never reached for these words. Before returning, if that value also sits in an executable section, is instruction-aligned and is not already occupied, it queues a job that starts a function there.

Why the wrong function does not always appear

Two checks hide this most of the time, and both ask the same question: is the address already occupied in the segment list?

• cfg_fast.py:3990, in _process_irsb_data_ref_inlined_data -- not self._seg_list.is_occupied(v) decides whether the job is created at all.
• cfg_fast.py:4979, in _pop_pending_job -- a DATAREF_HINTS job is dropped if self._seg_list.is_occupied(job.addr) by the time it is popped.

Whether it is occupied by then is a matter of scheduling, so the same binary gives a different function set in different processes. In 52 runs at master this object gave 3024, 3025 or 3026 functions, and the extras are exactly these two addresses.

That scheduling half is #6840, and #6844 fixes it: with #6844 cherry-picked onto master this object gives 3024 in 32/32 runs. The misread is not fixed by it.

Both of the following were run on master with one local modification -- the pending indirect jumps forced into a fixed order, so that the two answers can be produced on demand rather than by chance -- and with CFGJob.__init__ and _pop_pending_job hooked:

descending address order
  ### CREATE HINT 0xd4930 (cfg_fast.py:3998)
  ### POP 0xd4930 job_type=CFGJobType.DATAREF_HINTS occupied=False -> KEPT
  ### CREATE HINT 0xd4c80 (cfg_fast.py:3998)
  ### POP 0xd4c80 job_type=CFGJobType.DATAREF_HINTS occupied=False -> KEPT
  n_functions 3026
  0xd4930 is a function: True
  0xd4c80 is a function: True

#6844's discovery order
  ### CREATE HINT 0xd4930 (cfg_fast.py:3998)
  ### POP 0xd4930 DATAREF_HINTS occupied=True -> DROPPED at 4979
  n_functions 3024
  0xd4930 is a function: False
  0xd4c80 is a function: False

Under the first order the stack that reaches cfg_fast.py:3998 is, from _get_successors down:

cfg_fast.py, line 1851, in _get_successors      jobs = self._scan_block(job)
cfg_fast.py, line 3087, in _scan_block          entries = self._scan_irsb(cfg_job, current_func_addr)
cfg_fast.py, line 3306, in _scan_irsb           self._collect_data_references(irsb, addr)
cfg_fast.py, line 3923, in _collect_data_references
                                                self._process_irsb_data_refs(irsb.addr, irsb.data_refs)
cfg_fast.py, line 3963, in _process_irsb_data_refs
                                                self._process_irsb_data_ref_inlined_data(irsb_addr, ref)
cfg_fast.py, line 3998, in _process_irsb_data_ref_inlined_data
                                                ce = CFGJob(

So on this object #6844 reaches the right answer without the misread being fixed: 0xd4930's bogus hint is still built and then thrown away, and 0xd4c80's is never built at all.

Steps to reproduce the bug

import angr
p = angr.Project("binaries/tests/armel/tenda-httpd", auto_load_libs=False)
cfg = p.analyses.CFGFast()
for a in (0xd4930, 0xd4c80):
    print(hex(a), "is a function:", a in cfg.functions,
          "| is a block start:", cfg.model.get_any_node(a) is not None)

Because of #6840 this prints True only in some processes. To get it on demand, take the pending indirect jumps in descending address order -- replace the for idx, jump in enumerate(self._indirect_jumps_to_resolve): loop in CFGBase._process_unresolved_indirect_jumps with sorted(..., key=lambda ij: ij.addr, reverse=True) -- and both addresses become functions.

Environment

• angr 540f9112e9024ee0b6d80459deb663f2eb534cd7
• cle 3812052df2ad284cd16684fb7b7eb66e8d14dc6d
• archinfo, pyvex 9.3.5.dev0; pypcode 4.0.1.dev0
• Python 3.12.13, Linux x86-64
• binaries/tests/armel/tenda-httpd, sha256 e2880dc6a19a9ac5122d8686047db12a223d062324de7c5b9c5b36e60674a7ab

Why this is an issue and not a pull request

Any repair changes how _process_irsb_data_ref_inlined_data reads inline pointers, which is on the path for ARM objects generally, not just this one. The evidence bar for that is an A/B over an ARM corpus, and we have not run it. Filing the diagnosis rather than sitting on it.

session: sharpen