Decompiler: blocks are dropped from the output and the gotos that target them have no labels

Description

A decompiled function sometimes contains only part of what was recovered. The blocks that are left out appear nowhere in the output, and the goto statements that jump into them name labels the function never defines.

0x42939e (group_nodes_into_DFAstates) in du:

long long group_nodes_into_DFAstates(struct_1 *a0, struct_0 *a1, unsigned long a2, unsigned long a3)
{
    ...
    iter1 = 0;
    idx = 0;
    v15 = *((long long *)(v14->field_10 + idx * 8)) * 16 + a0->field_0;
    ...
    if (!v3)
        goto LABEL_0x4298e7;
    else
        goto LABEL_0x4295cc;
}

The function has 112 basic blocks and comes out as 65 lines. iter1 is assigned 0 and never read, and idx is read only at 0, so the loop that indexes them is gone as well.

Both goto targets are block starts of this function. The binary is PIE and loads at 0x400000, so LABEL_0x4295cc is 0x295cc in the file, the fall-through of the je the output ends on, and none of it is emitted:

   295c6:	0f 84 1b 03 00 00    	je     298e7 <group_nodes_into_DFAstates+0x549>
   295cc:	8b 85 30 ff ff ff    	mov    -0xd0(%rbp),%eax
   295d2:	83 e0 20             	and    $0x20,%eax

Every block recovered for a function should appear in the output, and every goto should name a label the output defines.

0x40dabb (server_main_setup) in lighttpd, built at -O2, has 500 basic blocks and comes out as 1108 lines. Most of the function is present, and one three-way branch has a defined label in one arm and undefined labels in the other two:

    v25 = v14 & 0xf000;
    if (v25 == 0x1000)
    {
        goto LABEL_0x40df1b;
    }
    else if (v25 == 0xc000)
    {
        goto LABEL_40df85;
    }
    else
    {
        goto LABEL_0x40df5e;
    }

LABEL_40df85 is defined further up and its block is emitted. The blocks the other two labels point at are not:

    df1b:	bf 01 00 00 00       	mov    $0x1,%edi
    df20:	e8 0b ed ff ff       	call   cc30 <dup@plt>
    df25:	89 05 ed 10 05 00    	mov    %eax,0x510ed(%rip)        # 5f018 <oneshot_fdout>
    ...
    df7b:	e8 b4 b8 00 00       	call   19834 <log_error>

In lighttpd 1.4.58 those are the S_ISFIFO arm, which sets oneshot_fdout from dup(STDOUT_FILENO) and logs on failure, and the !S_ISSOCK arm, which logs and returns. dup(1) never appears in the output — the only dup calls emitted are two dup(0) — and neither do the __LINE__ values of those two log calls, 0x482 (1154) and 0x489 (1161). https://github.com/lighttpd/lighttpd1.4/blob/992ba517abcfe9c162df0ba1383a051fe4ebd77c/src/server.c#L1151-L1163

The loss does not always leave a dangling label behind. 0x414072 (config_parse) in lighttpd has 148 basic blocks, comes out as 35 lines, and stops after a call with no return and no goto, so nothing in the output marks what is gone. Where a label does get defined, as LABEL_40df85 is, its name carries no 0x; every undefined one above does.

Steps to reproduce the bug

Decompile 0x42939e in tests/x86_64/decompiler/du and 0x40dabb in tests/x86_64/decompiler/lighttpd.

Both binaries are already in the binaries repository, so nothing is attached.

Environment

Platform: linux-x86_64
Python: 3.12.3
angr    9.3.4.dev0  (master, 4964e66dee654c6071451b06a3c6886a61d06399)
pyvex   9.3.4.dev0  (master, bdd5441035e02920eaa72c1c3cf9a4f0d572104d)

Reproduced on angr master at 4964e66dee654c6071451b06a3c6886a61d06399.

Additional context

du is GCC 9.4.0 at -O0 and lighttpd is GCC 11.4.0 at -O2. Both are PIE x86-64 ELF executables, neither is stripped. The disassembly quoted here uses file addresses; the decompiled output uses those plus 0x400000.

Decompiling several functions in one session changes what is recovered for the functions decompiled later, so every output quoted here comes from a session that built the CFG over the whole binary and then decompiled that one function.