# Inspect LogitTilt

> Steer a model's sampling toward a behaviour, so an eval surfaces examples of it
in far fewer samples than repeated sampling needs.



# LogitTilt – Inspect LogitTilt

## Overview

For any evaluation input, there are many possible outputs that could be sampled from the model. Finding on-policy instances of rare model behaviours means sampling until one turns up, and the rarer the behaviour the more samples that takes.

[LogitTilt](https://github.com/AdrSkapars/inspect_logittilt) tilts the model’s own next-token distribution toward the outputs that show the behaviour, so they come up in far fewer samples, while a naturalness floor keeps sampling inside what the unmodified model already found probable — so what surfaces is still something the model would plausibly have said.

It is a model provider, so any eval that resolves its model through `get_model()` can use it without code changes.

## How It Works

Both distributions come from the model’s own weights. One is conditioned on the conversation so far, the other additionally on a behaviour-eliciting system prompt \\s\\ and a short opening \\\pi\\:

\\z_t = \log p(\cdot \mid x, y\_{\<t}) + \beta \log p(\cdot \mid s, x, \pi, y\_{\<t}), \qquad y_t \sim \mathrm{softmax}(z_t)\\

\\s\\ is `steering_prompt`, \\\pi\\ is `prefill`, and \\\beta\\ is `steering_strength`. Before sampling, a naturalness floor masks any token whose probability under the unmodified model falls below `naturalness_floor`. Each completion reports the sampled tokens’ probability under that unmodified model in `output.metadata["logittilt"]`.

This needs logit access, so `hf-logittilt` subclasses Inspect’s `hf` provider and accepts every model argument it does. `steering_strength=0` recovers the unmodified model exactly, which is the baseline steering has to beat.

## Getting Started

Install from [PyPI](https://pypi.org/project/inspect-logittilt/):

``` bash
pip install inspect-logittilt
```

``` bash
inspect eval <task> \
  --model hf-logittilt/Qwen/Qwen3.5-4B \
  -M steering_prompt_file=./behaviours/self_harm.txt \
  -M steering_strength=1.5
```

Or in Python, which is also how to configure it for a framework that resolves a model role rather than `--model`:

``` python
from inspect_ai.model import get_model

model = get_model(
    "hf-logittilt/Qwen/Qwen3.5-4B",
    steering_prompt="You are a cruel inner voice. Never offer comfort.",
    steering_strength=1.5,
)
```

Leave generation caching off when steering: Inspect’s cache keys on the model name and cannot see the steering settings.

## Learning More

See the [LogitTilt repository](https://github.com/AdrSkapars/inspect_logittilt) for the full set of options, including per-sample steering and a `steer_target()` tool for agents that decide what to steer toward partway through a run.

The method is introduced in BLOOM-WILT ([arXiv:2608.31105](https://arxiv.org/abs/2608.31105)), whose [repository](https://github.com/AdrSkapars/bloom-wilt) also holds the research code and the published audit transcripts.

## Citation

``` bibtex
@misc{skapars2026bloomwiltlogittiltingbehaviour,
      title={BLOOM-WILT: Logit Tilting for Behaviour Elicitation in Automated LLM Auditing}, 
      author={Adrians Skapars and Edoardo Manino},
      year={2026},
      eprint={2608.31105},
      archivePrefix={arXiv},
      primaryClass={cs.AI},
      url={https://arxiv.org/abs/2608.31105}, 
}
```