a tool for reinterpreting ELF executables and shared libraries
a tool for reinterpreting ELF executables and shared libraries
Superlinker is a tool that can combine executables and shared libraries into even larger products, just like object files are combined into executables and shared libraries. It works well enough for building a self-contained Python distribution from off-the-shelf packages.
Wouldn't it be funny if your entire OS image consisted of only one shared object that was really, really large?
Superlinker is structured essentially like a compiler whose inputs and outputs are interpreted programs (ELF ET_DYN PIE executables or shared libraries). Its frontend lifts an ELF ET_DYN object into an abstract and simple intermediate representation, and its backend lowers this representation back to an ELF ET_DYN object. While memory mappings (PT_LOAD segments) are retained essentially intact, none of the ELF headers are copied from the inputs to the outputs. Of the many possible transformations, the currently implemented one rebases and merges several ELF objects. This approach is quite robust.
Additionally, Superlinker is able to merge the dynamic linker itself into an executable, which enables transforming system-dependent executables into executables that run anywhere. (The resulting executable is still an ET_DYN object to retain the benefits of ASLR, but it has no load-time dependencies.) This is implemented with an executable shim that emulates the kernel ABI for PT_INTERP loaded objects, and so is not tied to a specific libc, but currently only tested with musl libc.
The intermediate representation features architecture-, target-, and (somewhat) format-independent model of loadable segments, relocations, symbols, and image interpreters, biased towards ELF without directly requiring it. The frontend and backend are currently ported to amd64 only. Although not strictly required for functioning, section headers are emitted as a courtesy for libbfd based tools (e.g. objdump).
First, install Rust and run cargo build.
Usage: ./target/debug/superlinker [ [ ...]]…Although the core approach is sound, this implementation has flaws, most of which are fixable:
DT_JMPREL, DT_PLTREL, and DT_PLTRELSZ entries are stripped.)global-dynamic TLS model is supported.DT_GNU_HASH is not supported, and the number of DT_HASH buckets is randomly fixed at 4.PT_GNU_STACK and PT_GNU_RELRO are not supported and stripped.PT_GNU_EH_FRAME is stripped.The implementation is less than a thousand lines long, written with portability in mind, and extensively commented, so it should not be too difficult to address most of these flaws. It should even run on Windows!
Although tedious, it is possible to use Superlinker to build a fully self-contained Python distribution without source modifications or, in fact, touching source at all. First, link the combination of the Python executable, its dependencies, and essential modules. Using Alpine Linux 3.20 as the base distribution, run:
…Python has a little known function where it can treat a zip archive as if it was a directory, which will come in handy when packaging the (portable subset of) standard library modules:
# apk add fastjar
$ fastjar 0cvf py.zip -C /usr/lib/python3.12/ .Note the 0 (that's a zero) option for fastjar; Python loads compressed zip archives using its own zipimport standard library module, which means that it cannot be compressed when it is a part of a zip archive itself.
Even though Python has all of these modules linked into it, it's currently unaware of that, and an attempt to import any of them will fail. This can be solved with a little bit of Python code:
$ cat >sitecustomize.py py.run
$ chmod +x py.runThe final touch this distribution needs is the PYTHONPATH environment variable:
…If you like Superlinker, you might also enjoy unfork.
No open issues yet, or sync has not completed.