On September 10, 2026, Redwood Research published "An operationalization of opaque serial depth," by Nathan Sheffield, Alek Westover, Lukas Finnveden, Alexa Pan and others. It turns a concept from Google DeepMind (Brown-Cohen, Lindner and Shah, arXiv:2603.09786) into a metric that labs could disclose. The metric, "NLS depth," is the longest chain of serial computation in a model that never passes through a natural-language-rooted token. A token qualifies if its meaning comes from a pretrained language prior, the output space has not been expanded, tokens have not been reinterpreted as another kind of data, and no gradients flow back through tokens. Ordinary chain-of-thought (CoT) post-training, including RL, distillation and length penalties, leaves depth unchanged. Latent reasoning in the style of COCONUT, looped transformers and opaque memory multiply it or add to it.
This matters because CoT monitoring is a central oversight tool, and the post came out a week after reports that OpenAI's GPT-6 Astra uses "opaque recurrence" and is less monitorable. Unlike monitorability evaluations, a depth figure can be calculated from a design before training. That makes it a candidate for disclosure rules and for internal architecture review. The open question is whether the number predicts anything about real monitorability.
What we checked
- If the mechanism described by C4 generalizes beyond the reported setting, comparable primary studies should show the same direction under different models or tasks. partly confirmed. Two primary sources apply the depth measure across many architectures and get consistent results. Redwood finds that NLS depth tracks layer count across open models, with none above about 17,000. DeepMind computes upper bounds for Gemma 3, suggests mixture-of-experts models have lower depth than dense ones, and publishes a tool that works on any network written in JAX. No primary study yet shows, across several models or tasks, that higher depth goes with lower monitorability. [1][2][3]
- (Added) Is GPT-6 Astra's documented drop in CoT monitorability caused by greater opaque serial depth (looped or recurrent architecture)? unresolved. OpenAI's system card confirms that monitorability fell compared with GPT-5.6 Sol. It attributes the drop mainly to shorter, less informative chains of thought and says it is "quite confident" the change in CoT controllability is "not differentially due to any architectural changes." Redwood says only that public evidence is consistent with depth being a contributing factor. TechCrunch reports that Astra's use of recurrence is limited. [4][1][5]
What we found
- Redwood gives a closed-form upper bound for a standard transformer: layers × (4⌈log₂ hidden_dim⌉ + ⌈log₂ seq_len⌉ + 17). The bound is measured on a circuit of at most 10^26 gates, each taking two 32-bit inputs. Applied to Gemma3-1B, it gives about 1,954 (22 sliding-attention layers at 73, 4 global layers at 79, plus 32 for decoding). A full CoT rollout counts as no deeper than one forward pass at maximum context length. [1]
- Across open models, NLS depth tracks layer count, and no open CoT model exceeds about 17,000. The authors would be surprised if any closed model exceeded about twice that. They fit depth to active parameters with an exponent of about 0.26, then combine that with Chinchilla scaling and Epoch's compute trends to project growth of about 1.22x per year. That projection is their own extrapolation, not a measured trend. They note that a 2x jump from looped transformers would equal years of scaling under the current CoT approach. [1]
- The method rests on DeepMind's March 2026 paper. That paper defines opaque serial depth as the longest computation possible without interpretable intermediate steps, computes bounds for Gemma 3, and suggests mixture-of-experts models have lower depth than dense ones. The accompanying Apache-2.0 repository, google-deepmind/serial_depth, computes serial depth automatically from JAX computation graphs. Its examples include Gemma3_4B and a small MLP. When opened, the repo showed only one commit. [2][3]
- The authors say the metric has flaws. For DeepSeek-V4-Pro, they attribute about 12,200 of its depth to Sinkhorn-Knopp normalization steps (2 per layer × 100 × 61 layers) and call this a flaw in the definition, since these steps use few FLOPs but add a lot of depth. Excluding them shortens the deepest path by about 25%. The metric also deliberately ignores post-training that makes CoT illegible, and it misses opaque communication between agents. For that, Appendix F proposes a separate 'maximum opaque FLOP' measure. [1]
- The GPT-6 Astra system card says monitorability "has decreased relative to GPT-5.6 Sol." It attributes the drop mainly to shorter, less informative CoTs. It also says OpenAI is "quite confident that changes in CoT controllability are not differentially due to any architectural changes," and that those changes are correlated with higher no-CoT capability. UK AISI measured a no-CoT math time horizon of 30.9 minutes for Astra versus 3.6 for Sol, and cautioned that contamination may inflate the figure. [4]
- TechCrunch reported that Astra uses 'recurrent depth' or 'opaque recurrence,' but only to a limited extent. Redwood's Buck Shlegeris and Ryan Greenblatt said they were concerned. Redwood's post estimates Astra could have 2–3x the depth of GPT-5.6 Sol, citing reporting about a looped transformer, but says the cause of its lower monitorability is unclear. This is an estimate, not a disclosed figure. [5][1]
What it means
For builders, NLS depth is best used as a cheap check at design time rather than as a safety guarantee. If you are considering latent reasoning, loops, recurrent layers such as Gated DeltaNets, test-time training, or passing continuous state between agents, you can estimate how much the change multiplies your model's opaque depth before training. DeepMind's serial_depth tool does this for JAX models. A change that doubles depth is, by Redwood's own comparison, a bigger shift than years of ordinary scaling, and it is worth flagging in architecture reviews and system cards. Watch for artifacts like the Sinkhorn case. Tie your threat model to the definitional choices, such as no backpropagation through tokens and no expanded output space, because they decide what counts as monitorable.
Do not read a low depth figure as evidence that your CoT is monitorable. By design, the metric does not register illegible CoT produced by RL, and Astra shows monitorability can fall through behaviour (shorter, more controlled CoTs) that OpenAI says is not driven by architecture. Run behavioural monitorability and no-CoT capability evaluations alongside the depth figure, as the authors themselves recommend. Until several models show depth and measured monitorability moving together, treat the metric as a plausible, disclosable proxy whose predictive value has not been demonstrated.
Still open
- Does higher NLS depth predict lower measured CoT monitorability across several models and tasks, controlling for no-CoT capability and RL post-training?
- What is Astra's actual architecture and depth? Redwood's 2–3x figure is an estimate, and OpenAI's system card denies an architectural cause for the change in controllability.
- Will any lab disclose NLS depth or opaque serial depth figures, and how will they handle cases like test-time training or text diffusion?
- Can the definition be fixed so that low-FLOP serial steps such as Sinkhorn normalization do not inflate depth?