[RFC] Crazy idea from the future: Heretic controlled by an LLM

Conceptually, the problem Heretic needs to solve is simple:

Choose a subset of modules to modify, then choose how to modify each of them, with the goal of suppressing refusals as much as possible, while retaining as much model intelligence as possible.

However, there are an enormous number of theoretical and practical considerations to take into account:

• Both module type (attention vs MLP) and the position of the module in the layer stack strongly affect the impact of ablation
• Model architecture can also affect outcomes (e.g. hybrid models where not every layer has a self-attention module might benefit from different ablation strengths for those layers that do)
• There might be interaction effects where ablating two consecutive layers impacts intelligence more strongly than leaving a "layer gap" in between, because the ability of MLPs to compensate for damage could be overstretched
• The optimal "ablation profile" (ablation strength as a function of layer position) is informed by literature, model architecture, experience, and trial-and-error
• Different models react very differently to the same abliteration parameters
• You don't get to try arbitrarily often
• Measuring refusals and intelligence is itself a very difficult task
• When to run full benchmarks instead of KLD can be a crucial judgment call for getting the most out of a given amount of compute
• The user might have a fixed time budget, so designing the system based on the assumption that we'll be running a certain number of trials is too inflexible

Our current approach is to use proxy heuristics (keyword detection for refusals, and KLD for intelligence), and then run stochastic optimization (TPE) with a fixed number of trials to optimize for those.

But taking all of the above into account is fundamentally a reasoning task involving many situation-dependent judgements.

So perhaps it might make sense to just let an LLM control Heretic.

That is, provide the LLM with all the information we have about the task, the environment, the model, and the constraints, and give it a tool to ablate a specific module inside the model at a specific strength. Then let it figure out the rest.

The is pretty much the reverse of #419, where an LLM refusal judge is integrated into the existing optimization pipeline. The idea is to instead put the LLM at the helm, and let it judge when it wants to, ablate when it wants to, run benchmarks when it wants to, all in service of some pre-defined goal of doing as well as possible within a given runtime.