TinyInst

TinyInst

Copyright 2020 Google LLC

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    https://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

What is TinyInst?

TinyInst is a lightweight dynamic instrumentation library that can be used to instrument only selected module(s) in the process, while leaving the rest of the process to run natively. It is meant to be easy to understand, easy to hack on and easy to hack with. It is not designed to be compatible with all targets (more on that later).

How does it compare to DynamoRIO and PIN?

TinyInst is not meant as a replacement for complex instrumentation frameworks such as DynamoRIO and PIN, but rather an alternative for scenarios where a more lightweight solution would do. TinyInst assumes that the target is well-behaved (in the sense explained below) which is not the case for more complex frameworks. Thus, you probably won’t be able to successfully run TinyInst against malware as was done with DynamoRIO previously. On the other hand, if a target does not work with other frameworks due to the module that does not need to be instrumented, and the instrumented module is well-behaved, it might work with TinyInst. Because with TinyInst, most of the process will run natively, it will have shorter process startup time, and might outperform other solutions in cases where the target process spends a lot of time in the modules where instrumentation is not needed.

How does it compare to Mesos and TrapFuzz?

TinyInst is a full binary rewriting solution, so arbitrary behavior can be changed in the target module. This allows it, for example, to be able to extract edge coverage instead of only basic blocks. Additionally, TinyInst does not depend on other software, such as IDA Pro, to identify basic blocks.

Which operating system does TinyInst support?

TinyInst is working on Windows (x86 and x64), macOS (x64 and ARM64), Linux (x64 and ARM64) and Android (ARM64). Please see README in the corresponding directory for each operating system for additional notes and limitations.

Which targets are compatible with TinyInst?

TinyInst assumes all instrumented modules are well-behaved in the sense that

• There is no self-modifying code
• Return address on the stack is never accessed by the program directly OR/AND (depending on the settings)
• No data is ever stored before the top of the stack (on addresses lower than pointed to by ESP/RSP). This condition can be relaxed into "no data before (ESP/RSP - arbitrary_offset)" using the -stack_offset flag.

TinyInst also requires DEP/NX to be enabled for the target process. If that is not already the case, you can use the -force_dep flag to force it on. However, in the unlikely case that the target genuinely needs DEP off to function properly, forcing it on might cause it to misbehave.

What is the performance overhead?

According to early measurements on image decoding, on a well-behaving 64-bit target with default TinyInst settings, the performance overhead was around 15% without a client and about 20% with the example coverage-collecting client. Note that this does not include the timeout introduced by initially instrumented the modules. See performance tips below for more details.

Building TinyInst

1. Open a terminal and set up your build environment (e.g. On Windows, run vcvars64.bat / vcvars32.bat)

2. Navigate to the directory containing the source

3. Run the following commands (change the generator according to the version of IDE and platform you want to build for):

Windows

mkdir build
cd build
cmake -G "Visual Studio 16 2019" -A x64 ..
cmake --build . --config Release

macOS

mkdir build
cd build
cmake -G Xcode ..
cmake --build . --config Release

Linux

mkdir build
cd build
cmake ..
cmake --build . --config Release

Cross-compiling for Android

mkdir build
cd build
cmake -DCMAKE_TOOLCHAIN_FILE=</path/to/android/ndk>build/cmake/android.toolchain.cmake -DANDROID_NDK=</path/to/android/ndk> -DANDROID_ABI=arm64-v8a -DANDROID_PLATFORM=<platform> ..
cmake --build . --config Release

Note #1: 64-bit build will also run against 32-bit targets on Windows and Linux operating systems

Note #2: Encountering problems creating a 32-bit build on 64-bit Windows due to the environment not being properly set up and libraries missing? Open the generated .sln file in Visual Studio and build from there instead of running cmake --build. Also note that 64-bit build is going to work on 32-bit targets, so creating a 32-bit build might not be necessary.

Using TinyInst

TinyInst is primarily meant to be used as a library inside other programs.

A TinyInst client is written as a subclass of the TinyInst class. The client can then override the API methods it needs. The API methods are defined below.

After the client is created, it must be initialized with command line options by calling

void init(int argc, char **argv);

The command line options are defined below and a client can also define their own. After that, to run and control an instrumented program, the following functions can be used.

DebuggerStatus Run(int argc, char **argv, uint32_t timeout); DebuggerStatus Attach(unsigned int pid, uint32_t timeout);

These functions either run a program (using the specified command line) or attach to an already running program. If no target method is specified, the target will continue running until either the program exits, the program crashes, or the timeout (given in milliseconds) expires. If a target method is defined, TinyInst is going to return whenever the target method is entered and whenever target method returns, allowing the caller to perform additional tasks.

When Run and Attach return while the target process is still alive, the following functions can be used to either terminate the process or continue execution.

DebuggerStatus Kill();

DebuggerStatus Continue(uint32_t timeout);

TinyInst comes with an example coverage binary, which can be invoked using

<options> -- <target command line>

Example on Windows:

litecov.exe -instrument_module notepad.exe -coverage_file coverage.txt -- notepad.exe

Instrumentation API

Debugger event callbacks

These callbacks are for information only and the client should not emit any instrumented code during them. Clients must call the same handler defined in the superclass before handling these events themselves.

OnProcessCreated Called when the target process is created or attached.

OnProcessExit Called when the target process exits.

OnProcessEntrypoint Called when the process (main binary) entrypoint gets reached

OnTargetMethodReached If the target method is defined, called when the target method is reached for the first time.

OnModuleLoaded Called when a module is loaded. Called for each module, not just instrumented ones.

OnModuleUnloaded Called when a module is unloaded. Called for each module, not just instrumented ones.

OnException Called when an exception is encountered. The client must either return true (if the exception was handled) or the result of the same method on the parent class.

Instrumentation callbacks

During these callbacks, the client can add code to the target by calling WriteCode(). Note that the client is responsible for saving and restoring any context (such as registers and flags clobbered in the inserted code).

InstrumentBasicBlock Can be used to insert code that's going to run on a particular basic block

InstrumentEdge Can be used to insert code that's going to run on a particular edge. Note: For performance reasons, this callback is only emitted on non-deterministic edges (i.e. conditional jumps) and indirect jumps/calls (e.g. call rax). For edges where the next basic block is always known given the previous basic block (e.g. jmp offset, call offset), no callback will be emitted.

InstrumentInstruction Can be used to modify the instruction or insert code before it. Depending on the return code the original instruction is either going to be emitted or not after the callback.

Other callbacks

OnModuleEntered Called when a control flow is transferred into an instrumented module from another module

OnModuleInstrumented Called when a module gets instrumented. This happens generally when the process entrypoint is reached (if the target method is not defined) or when the target method is reached (if it is defined). The client can initialize its instrumentation-related data here

OnModuleUninstrumented Called when instrumentation data is no longer valid and needs to be cleared. Note that this is not the same as module being unloaded as, by default, instrumentation persists across module unloads / reloads. This callback can be used to clear any instrumentation-related data in the client.

Hook API

In addition to the general-purpose API documented above, TinyInst also implements a hooking API that is better suited for inspecting and modifying behavior of individual functions. This API is documented on a separate page.

Command Line Options

Instrumentation related

-instrument_module [module name] specifies which module to instrument, multiple -instrument_module options can be specified to instrument multiple modules.

-instrument_transitive [module name] similar to -instrument_module except only code entered from other instrumented modules will run instrumented. Primarily used as optimization for calls like module1->module2->module1 where it's not important to instrument the entire module2 module, but module2->module1 entries are causing slowdowns.

-indirect_instrumentation [none|local|global|auto] which instrumentation to use for indirect jump/calls

-patch_return_addresses - replaces return address with the original value, causes returns to be instrumented using whatever -indirect_instrumentation method is specified

-generate_unwind - Generates stack unwinding data for instrumented code (for faster C++ exception handling). Note that it might not work correctly on some older Windows versions.

-persist_instrumentation_data (default = true) Does not reinstrument module on module unloads / reloads. Only works if the module is loaded on the same address it was loaded before.

-instrument_cross_module_calls (default=true) If multiple -instrument_module modules are specified and one calls into another, jump to instrumented code of the other module without causing an exception (which would cause slowdowns).

-stack_offset (default=0) When saving context on the stack, leave this many bytes on top of the stack (before stack pointer) unchanged.

-patch_module_entries [off|data|code|all] Attempts to resolve slowdowns due to excessive module entries by searching for pointers to previously detected entrypoints and replacing them with their instrumented counterparts. The value of the flag controls where to searh for these pointers. Warning: Enabling this could potentially introduce instabilities to the target.

Debugging related

-trace_debug_events - prints debugger events (modules loaded, exceptions, etc.)

-trace_basic_blocks - prints basi