Anthropic has never released any technical paper on Claude Mythos. Weights were not published, the architecture was undisclosed, and the company has remained silent about its internal workings. Nonetheless, this did not stop the research community from conducting a bold intellectual experiment — Kye Gomez, the 22‑year‑old founder of the Swarms agent framework, published the OpenMythos project on GitHub: a PyTorch implementation that reconstructs a theoretical model from public research and from scratch, asking a core question: if Claude Mythos really exists, what might its architecture look like?
Core hypothesis: Mythos is a "Recurrent‑Depth Transformer"
The central bet of OpenMythos is the Recurrent‑Depth Transformer (RDT), also called the Looped Transformer. This differs fundamentally from mainstream feedforward architectures such as GPT and LLaMA.
In a conventional Transformer, input tokens flow through dozens to hundreds of distinct layers, each with independent weights; capability scales with layer count and parameter volume. RDT divides computation into three stages: Prelude → Recurrent Block (run T times) → Coda. Prelude and Coda are standard Transformer layers executed once, while the middle recurrent block iterates using the same weights repeatedly, feeding each output back as the next input.
Gomez summarizes the design philosophy in one sentence: "Stack loops, not parameters." Concretely, the recurrent block updates its hidden state each iteration as:
h_{t+1} = A·h_t + B·e + Transformer(h_t, e)
where h_t is the hidden state after the t‑th loop, e is the Prelude‑encoded input (re‑injected each loop to prevent signal drift), and A and B are learnable injection parameters.
The theoretical advantage is that inference depth is determined by the number of iterations at runtime rather than the number of stored parameters. Saunshi et al.'s 2025 paper formalizes that each RDT iteration is functionally equivalent to a single chain‑of‑thought step, with all "thinking" occurring in a continuous latent space without generating intermediate tokens.
Key components: MoE, MLA and stability mechanisms
OpenMythos is not merely a recurrent Transformer; it integrates several cutting‑edge components:
Mixture‑of‑Experts (MoE): The feedforward networks in the recurrent block are replaced by MoE layers using a DeepSeekMoE design — many fine‑grained routing experts plus a few always‑online shared experts. Crucially, the router picks different expert subsets across loop iterations so the same weights perform different computations at different iterations. Gomez summarizes: "MoE provides breadth of domain knowledge; recurrence provides depth of reasoning."
Multi‑Latent Attention (MLA): By default it uses DeepSeek‑V2's MLA, caching compressed low‑rank KV latent representations instead of full key‑value tensors, which can reduce KV memory by 10–20× at production scale.
LTI stability constraints: Training recurrent models has historically been notoriously unstable, with hidden states potentially growing unbounded (residual explosion). OpenMythos adopts the Parcae approach from a 2026 UCSD and Together AI paper by parameterizing constraints to ensure the spectral radius of the injection matrix A is strictly less than 1, guaranteeing stability. The concrete implementation is:
A = exp(-exp(log_dt + log_A))
With log_A and log_dt initialized to zero, A = exp(-1) ≈ 0.3679, which naturally satisfies the stability condition.
Adaptive Computation Time (ACT): To avoid "overthinking" (loops driving hidden states beyond the solution into noise), ACT learns a scalar per position to decide dynamically when to stop looping — simple tokens exit early, difficult tokens receive more computation.
Deep LoRA adapters: Small low‑rank adaptation matrices are added at each loop iteration so different iterations can express fine‑grained differences while sharing the main weights, bridging pure weight sharing and fully independent layers.
Empirical validation: from theory to silicon
OpenMythos is not just paper talk. The author at TinyComputers.io ran a 1B‑parameter version successfully on a device with an AMD Ryzen AI MAX+ 395 APU (integrated Radeon 8060S GPU, 60GB unified memory).
Measured results: with batch=1, sequence length 128, 16 loops, and bfloat16 mixed precision, forward inference took 2.07 seconds with peak memory 6.44GB. The author performed five key validations:
- Spectral radius verification: At initialization the measured
ρ(A) = 0.367879, matching theory to six decimal places and confirming Parcae's stability constraint was correctly implemented. - Loop effectiveness: Comparing outputs across different loop counts, argmax agreement between 1 and 16 loops was only 35.2% with KL divergence 0.19 nats; distributions monotonically converged as loops increased, showing loops perform meaningful computation rather than idling.
- Stability ablation: Removing the LTI constraint caused training to collapse (gradient norms exploding to machine zero within 20 steps and loss freezing at uniform‑distribution entropy), while the stable version sustained effective gradients, reproducing the failure modes predicted by the Parcae paper.
- Hidden‑state growth: Forcing
ρ=2.0made hidden states expand from 91 to 23,232 after 8 loops, approximately following theoretical doubling per iteration. - Throughput testing: Latency scales linearly with loop count (62 tokens/sec at 16 loops); ACT did not show speedups on randomly initialized models because every position in the batch must stop simultaneously to exit early.
Parameter efficiency: 770M matches 1.3B
The Parcae paper provides key efficiency evidence: when trained on the same data, a 770M RDT can reach downstream quality comparable to a 1.3B standard Transformer, nearly halving parameter count. Optimal loop counts and optimal token counts follow power‑law scaling, establishing the first predictable scaling law for recurrent training depth.
OpenMythos supplies full configurations from 1B to 1T parameters. The 1B configuration uses dimension 2048, 64 experts, expert dimension 2048, 16 loops, and 4K context. The project is MIT‑licensed, about 1000 lines of code, and supports single‑GPU and multi‑GPU distributed training.
Epistemic limits: reconstruction is not restoration
After validation, the TinyComputers.io author wrote a pointed conclusion: "Architecture Verified, Mythology Intact."
This highlights OpenMythos's fundamental limit: it verifies "whether recurrent Transformers as a class are viable," not "whether Claude Mythos actually uses that architecture." The author emphasizes that a speculative reconstruction can only tell you whether the rebuild is internally consistent and consistent with cited public research, not whether it maps to the object being reconstructed.
An unexpected finding supports this: when the author attempted 24 loops they encountered an IndexError because per‑loop adapter components were implemented as a fixed‑size lookup table with a max_loop_iters. This means depth extrapolation — training on 5‑step inference chains and testing with 10 or 20 steps — is limited by a preset maximum loop count in this implementation and cannot be simply dialed up at inference time. This detail disappears in the theoretical claims but surfaces in concrete implementation.
Conclusion
OpenMythos's value is not in "cracking" Anthropic's trade secrets but in providing a runnable pedagogical artifact and a modifiable research baseline. It distills pages of math from the Parcae paper into a line of Python and turns the recurrent Transformer from an abstract concept into a physical object measurable in a home lab.
As the project states: "This is an independent, community‑driven theoretical reconstruction based on public research and speculation. It has no affiliation, endorsement, or association with Anthropic or its proprietary systems." In an era where frontier AI models increasingly become opaque endpoints — exposing only input‑output contracts and hiding all internal mechanisms — OpenMythos represents a reverse, open spirit: even if the black box itself is unreachable, the community can use first‑principles reasoning and fragments of public literature to assemble a runnable hypothesis and make it operate on silicon.
Whether Mythos's true architecture is like this or not, OpenMythos has shown that recurrent‑depth Transformers are a viable, parameter‑efficient path with scalable reasoning depth — and that may be more enduring than guessing Anthropic's "myth" itself.
▶️ Enjoyed this article? Creating great content takes effort!
Subscribe to our YouTube channel:
AI Rosetta
Your likes, subscriptions and shares mean the world to us! 🙏
For more AI news, visit:
https://www.airosetta.com