27 billion parameters against frontier models

Inherent Laboratories introduced Faraday on August 14. The London AI lab, founded in part by former Google DeepMind researchers, describes the system as a 27-billion-parameter AI Scientist.

On Replica, the benchmark developed alongside the project, Faraday scores above Claude Opus 4.8 and GPT-5.5 on the scientific replication tasks tested by the company.

That is notable because both reference systems belong to a much larger class of model. But the limitation matters immediately: Faraday does not demonstrate that a 27B model is generally better than Claude or GPT.

It shows that a heavily specialized system can outperform general-purpose frontier models on the particular problem it was trained to solve.

Replica is not asking the model to recite a paper

Replica contains 310 tasks derived from 100 machine-learning and AI-for-science papers. The collection spans areas including natural-language processing, materials science and weather forecasting.

The setup is deliberately demanding. A figure or result is removed from a scientific paper and the agent must attempt to reconstruct it from the remaining information without seeing the original plot and while operating under time and compute constraints.

The job therefore goes beyond retrieving a sentence memorized during pretraining. The agent has to understand the methodology, write or request code, run experiments, inspect the results and decide what to try next.

Inherent is specifically trying to measure what it calls research taste: the ability to choose worthwhile experiments rather than simply generate a plausible answer.

Faraday is not actually working alone

This is where the result becomes much more interesting technically. Faraday uses coding agents as tools. In the evaluation presented by Inherent, it can direct GPT-5.5 Codex.

The 27-billion-parameter model therefore does not necessarily perform all of the programming itself. It can decide what needs to be tested, delegate implementation to a much more capable coding agent, inspect what comes back and then choose the next step.

Faraday is consequently closer to a researcher equipped with an excellent coding assistant than to an isolated language model doing every part of the job itself.

Inherent says Faraday was trained using GPT-5.4-mini as its coding tool before being given GPT-5.5 Codex at test time. According to the lab, the agent was able to take advantage of the stronger tool without being specifically retrained around it.

The brain of the system does not have to be the biggest model

That division of labor changes how the result should be interpreted. In many current AI systems, the strongest available model receives the context, the problem, the tools and responsibility for deciding what to do next.

Faraday explores a different architecture. A relatively compact specialized layer remains responsible for strategy while larger models or tools are brought in only for particular operations.

The potential advantage therefore goes beyond parameter count. A specialized model can learn behaviors tailored to its domain: when to run an experiment, when to abandon a path, how to interpret a result and when another tool is worth calling.

That distinction matters. A general-purpose frontier model has an enormous range of abilities, but it is not automatically optimized to orchestrate a coherent hour-long chain of scientific experiments.

Training focuses on decisions, not only answers

Inherent trains Faraday with long-horizon reinforcement learning. Instead of rewarding only a final answer, the team attempts to evaluate the quality of the process followed during a replication attempt.

To do that, it builds task-specific evaluation rubrics and uses an LLM-based judge. Inherent also aggregates multiple evaluations and applies turn-level credit assignment during training.

The aim is to identify which intermediate decisions actually contributed to a successful outcome. For an agent performing many actions over a long session, that is considerably harder than simply marking a final answer right or wrong.

A home-grown benchmark is still a home-grown benchmark

The results should nevertheless be treated carefully. Replica was designed by Inherent and Faraday was trained specifically around this family of tasks.

The paper released on arXiv is a preprint. These results therefore do not provide independent validation that Faraday is superior to competing systems for scientific research in general.

The evaluation system itself also relies on language models and automatically generated rubrics. Inherent says it conducted a human study to test the judge's consistency, but that does not remove all of the limitations associated with this kind of measurement.

The useful conclusion is therefore not that a 27B model has made large models obsolete. It is that enough specialization can change how the capabilities of those large models are used.

The specialized model could become the conductor

Faraday provides a concrete example of an architecture that may become increasingly common. Instead of sending every problem directly to a gigantic universal model, a system can keep a smaller domain-specific layer in control and call heavier models only for operations where their capabilities are actually useful.

For science, that layer learns which experiments to run. In software development, a comparable system could understand a project's architecture and delegate only selected implementation tasks. Other domains could use their own specialized controllers around more general tools.

That does not automatically mean every run becomes cheaper: Faraday itself calls Codex. What changes is where the decision-making intelligence sits inside the system.

And that may ultimately matter more than Inherent's benchmark score. The next major improvement in AI agents may not come only from building a larger universal model. It may also come from a smaller model that knows far better which model to call, when to call it and exactly what to ask.