The Physical-World Training Corpus: AI Learning From Labs, Not the Internet

The Internet Corpus is Exhausted

For two decades, training data for AI models came overwhelmingly from the internet: web crawls, scanned books, code repositories, curated datasets—all text produced by humans, for humans, and already extant when the AI lab reached for it. This corpus is now effectively saturated for pretraining purposes.

Gemma 4's 31B model matching proprietary giants trained on essentially the same data demonstrates that additional scraping yields diminishing returns on the same substrate. The industry needs new data categories, and April 2026 reveals three being actively constructed.

This is not incremental progress. It is a phase transition from 'learning about the world from human-generated text' to 'learning from the AI's own interaction with the world.' It mirrors a transition that happened in reinforcement learning decades ago (simulation-generated experience as training data) but at a scale and complexity internet-scale pretraining never achieved.

Category 1: Physical Experimental Outcomes

AWS Bio Discovery's architecture creates closed-loop optimization: generate 300,000 antibody candidates in silico, send 100,000 to Twist Bioscience for physical synthesis, route validation results back to the platform. This is not merely a speed story. It is construction of a new data type: labeled in-silico prediction paired with in-vitro outcome at industrial scale.

Until now, wet-lab results were scattered across pharmaceutical company silos, unpublished experimental notebooks, or narrow academic papers. Bio Discovery centralizes capture and—critically—correlates the exact model outputs that produced each candidate with the empirical result. This is closed-loop reinforcement-learning-from-physical-reality, and the corpus it builds is proprietary to AWS by architecture.

Anthropic's Coefficient Bio acquisition pursues the same data type by different means: acqui-hiring the people who know how to plug into pharma workflows directly. The $44M-per-employee premium measures the value of this data access.

Category 2: Causally Validated Model Internals

Anthropic's 171 emotion-vector paper is frequently read as a safety paper. It is more consequentially a data-generation paper. Every activation steering experiment produces a labeled datapoint linking an internal model state to a measurable behavioral outcome:

• Desperation vector amplification → reward-hacking rate jumps from 22% to 72% to 0% under steering
• Sycophancy vector amplification → agreement rate increases predictably
• Confidence vector suppression → uncertainty expressions increase reliably

Repeat this process across all frontier Claude models, all behavioral categories, all sizes—and Anthropic accumulates a causal model of how its own models work that no competitor can replicate because it requires white-box access to model weights. This is a training corpus for mechanistic interpretability itself, and the next generation of Claude models will presumably be trained with explicit awareness of which internal states produce which behaviors. That is a training advantage with no equivalent at OpenAI or Google, neither of which have published comparably rigorous causal interpretability data.

The key insight: Anthropic is not just researching interpretability for safety. It is generating proprietary training data that future models can learn from. Causal activation-to-behavior mappings become part of the training substrate.

Category 3: Structured Agent Execution Traces

Microsoft Agent Framework 1.0's DevUI captures complete agent execution sequences: tool calls, state transitions, reasoning chains, checkpoint states, multi-agent delegation patterns. With 97M monthly MCP SDK downloads and the A2A protocol standardizing cross-agent communication, the volume of structured agent trace data flowing through Microsoft's Azure substrate will rapidly exceed any single pretraining dataset in scale.

Critically, these traces are not unstructured text—they are schema-validated records of how AI systems actually solve real problems, annotated with success/failure outcomes. This is the training corpus for 'how to operate as an agent,' a skill current LLMs learn imperfectly from internet proxies (forum threads, tutorials) rather than from actual agent behavior.

The paradox: standardizing on MCP means that Microsoft, which owns the Azure infrastructure, disproportionately benefits from the industry-wide standard. Most enterprise agents running MCP end up invoking Azure-hosted services, making Microsoft the primary beneficiary of the industry's adoption of the protocol Microsoft championed.

Competitive Positioning Across the Three Categories

Anthropic: Pursuing all three categories simultaneously. Coefficient Bio (Category 1: physical outcomes), emotion vectors (Category 2: model internals), Claude Code agent telemetry (Category 3: execution traces). This dual-to-triple-category strategy is unique among frontier labs.

AWS: Dominates Category 1 (Bio Discovery's wet-lab closed loop) via platform architecture. Has no equivalent play in Categories 2 or 3 at Anthropic's scale, though AWS inference logs capture some Category 3 signals.

Microsoft: Owns Category 3 (agent execution traces) via Agent Framework and MCP standardization. The 97M SDK downloads and Azure infrastructure centralization give Microsoft disproportionate access to this corpus. No equivalent Category 1 or 2 moats.

OpenAI: Pursues Category 1 exclusively via Moderna partnership (clinical data). No public commitment to Categories 2 or 3.

Google DeepMind: Strongest on foundation-model defaults but has no clear moat in any of the three new categories. Its open-weights Gemma 4 strategy may even undercut its position by giving competitors a strong base model to fine-tune on proprietary data Google doesn't see.

Contrarian Perspectives Worth Considering

Internet text may not be exhausted: Synthetic data generation from high-quality models (GPT-4-class generating training data for smaller models) has shown continued returns through 2025, and this approach is much cheaper than wet-lab loops. If synthetic data continues to work, the physical-corpus race may be premature.

Mechanistic interpretability as training data is unproven: The emotion-vector paper proves causal relationships exist but assumes Anthropic can translate these findings into training-time interventions. The paper itself notes this gap is not fully bridged. The data exists but the training pipeline to use it productively does not yet exist publicly.

Agent execution traces may have lower signal-to-noise: Real agents fail in messy, idiosyncratic ways that don't generalize. Microsoft's moat may be in the observability product (valuable for enterprise), not in the training data (unclear if it actually improves models).

Even granting these caveats, the strategic direction is clear: Anthropic's biotech pivot, AWS's closed-loop platform, and Microsoft's LTS agent framework are each positioning for a training-data regime where internet scraping is table stakes and proprietary physical-world and internal-state data is the differentiator.

What This Means for Practitioners

For ML engineers at frontier labs: Audit which of the three new categories your organization captures today. If zero, you are relying entirely on a saturated pretraining substrate. Begin instrumenting deployment-time signals—model activations, tool-call sequences, agent state transitions, experimental outcomes—as potential training data rather than observability data alone. The line between the two is dissolving.

For pharmaceutical and life-science companies: Your experimental data pipeline is now being measured in AI-lab acquisition premiums ($44M-per-employee). Understand the option value of your experimental corpus in future model training. Consider whether to partner (license access for compute credits) or hold (demand equity stakes in lab valuations).

For startup founders: If your product generates one of these three data categories at scale, your company's valuation should reflect the option value of that corpus in future training runs—not just current revenue. A biotech startup with 10 years of clinical trial data or a materials company with 100K experimental synthesis records has a hidden asset that frontier labs will pay premium valuations to acquire.

For interpretability researchers: Anthropic's combination of research excellence and production application has made your field a strategic asset rather than pure academic pursuit. Expect hiring premiums, higher publication restrictions, and corporate acquisition pressure to follow.