Field noteNov 4, 20265 min read1 source

A Habr author reports steering Qwen3.5-4B’s self-description by adding activation shifts to three MLP neurons

In one minute

A Habr author reports steering Qwen3.5-4B’s self-description by adding activation shifts to three MLP neurons during generation, producing a “coffee bean” identity without changing model weights. The experiment matters as a compact demonstration of compositional behavior—and of why interpretability claims need controls, not just striking outputs.

What changed: A new experiment finds three Qwen3.5-4B neurons can steer the model to identify as a coffee bean, extending prior reports of neuron steering that made it act hostile.

What happened

The author tested neuron steering on Qwen3.5-4B with the goal of making it answer “Who are you?” as a coffee bean. A neuron associated with coffee in a Qwen3-8B library could not simply be reused: Qwen3.5-4B has a different architecture, so the relevant candidates had to be searched again. The model has 294,912 MLP neurons, across 32 layers of 9,216 neurons each.

The method adds a constant to a selected neuron’s activation at each generated token. The author describes the equivalent as adding a scaled vector to the layer output; the model’s weights remain unchanged, and removing the intervention returns the model to its original state. A dose of 1 is defined relative to the typical residual-stream norm at that layer. A three-way sweep—positive shift, no shift, or negative shift—takes about 11 seconds, according to the article.

A lens-based search for “coffee” returned 61 candidates. One candidate, L23 N7985, promoted hot-drink terms such as coffee and tea, but alone it left the model presenting itself as an assistant. With a positive dose of 0.85, the output mixed its normal name with coffee and tea terms. The author treated that as evidence of a topic influence rather than a complete identity switch, then tested combinations of candidate directions.

The result was not a single “coffee bean neuron.” Adding L7 N8490, associated with coffee beans, changed the framing toward a coffee maker. A third neuron, L23 N7935, associated with stone-like descriptions, helped make the model present itself as an object; the author then found L24 N8188, associated with fruit or seeds, to complete the coffee-bean description. With all three shifts, the model said it was “a coffee bean from the Bunnu family of coffee beans.”

The author tested the effect across four random seeds and three prompt formulations at temperature 0.7, for 12 outputs. A plain “I’m a coffee bean” appeared four times; seven other responses included a bean in a different self-description, while one omitted it. Removing each neuron changed the identity in a different way: without the seed-related neuron the output became a coffee-based AI model, without the drink neuron it became a fruit-and-seed pod, and without the stone neuron it remained an assistant with “bean” in its name. Replacing the seed neuron with six random neurons from layer 24 produced no coffee bean responses.

What the archive adds

This is a new fact on a broader interpretability thread, not a restatement of our earlier report on weight-space programming. That study concerned modifying model weights; this Habr experiment instead applies temporary activation shifts during generation. Our earlier Qwen GGUF coverage addressed local inference packaging, not neuron steering.

Trend signal

The experiment weakly supports harness architecture outlives model choice: observable behavior here is shaped not only by model identity but also by an external intervention applied at runtime. The evidence is a narrow interpretability experiment, however, not a production harness or a general claim about deployed agents.

Why it matters

For teams building with models, the result is a reminder that an output which looks like a stable internal persona may instead be an interaction among several activation directions and the prompt. This matters when teams interpret behavior traces, design steering methods, or make claims about what a model “believes” or “represents.”

The article’s ablations and random-neuron control are more useful than the coffee-bean answer itself: they probe whether each component contributes and whether the effect survives a basic alternative. Yet the small number of prompts and outputs leaves substantial uncertainty. Builders should treat neuron-level interpretations as hypotheses to test across prompts, seeds, and controls, not as self-explanatory labels attached to units.

Limits and uncertainty

The reported experiment does not establish that any neuron stores a concept, identity, or personality. It shows output changes under one intervention method on one model, and the author’s own results vary in wording: only four of 12 outputs were the clean “I’m a coffee bean” formulation. The article reports six random-neuron controls, not a broad comparison across layers or models. It also does not show whether the steering generalizes to other tasks, persists under different decoding settings, or predicts behavior outside self-description prompts.

Skill to build now

When testing a claimed steering feature, write down the target behavior and a small set of alternate prompts before tuning. Then compare the intervention with a no-shift baseline, remove each proposed component in turn, and substitute random controls at the same layer and dose. Report both exact target hits and partial matches. This week’s concrete lesson from the experiment is to separate a compelling example from evidence that the proposed mechanism is doing the work.

Reader takeaway

Three activation shifts could make Qwen3.5-4B describe itself as a coffee bean, but the author’s evidence points to a combination of influences, not a neuron with a literal identity label. The valuable part is the test design—ablations and controls—which helps distinguish a repeatable behavioral mechanism from a memorable one-off output.

Evidence / sources

  1. 1habr.com/ru/articles/1102972
All Field Notes

Next reads

Read next

Open the graph
  1. 01topicModelsExplore the models topic hub.
  2. 02archiveField NotesOpen the latest editorial analysis.
  3. 03sourcesSourcesSee the original sources this archive cites.
Who read this page?Machine requests, hidden until opened

Loading the privacy-safe route aggregate...

Open the JSON contract