Skip to content
ResearchField notes
Nov 10, 2026published
Data Gradientauthor
00:00 UTC
← All field notes

What Does It Take to Teach an AI Model Real Work?

New research in agent training, reinforcement learning, and multimodal evaluation is revealing what it takes to develop capabilities that extend beyond familiar tasks.

An AI agent can complete every step of a workflow and still get the job wrong.

Consider an agent configuring a procurement process. It creates the forms, populates the fields, and establishes the approval routes. Everything appears complete. But if a high-value purchase bypasses Finance, or a contract uses the wrong jurisdiction-specific clause, the workflow has failed at something fundamental.

Teaching a model to perform this kind of work requires more than examples of successful outputs. It needs experience with the decisions, constraints, and exceptions that determine whether the work succeeds.

In October 2026, OpenAI published research from its collaboration with Ironclad, turning professional contracting workflows into tasks that AI agents could practice and learn from. Domain specialists helped define 11 representative tasks, researchers developed synthetic training scenarios, and models practiced in hosted software environments with reinforcement-learning feedback.

On the resulting research evaluation, GPT-6 Astra achieved 55.0% of the available assessment credit, compared with 41.6% for GPT-5.6 Sol, while estimated time per attempt fell from 37.0 to 19.2 minutes.

Those results demonstrate progress on the tested workflows, not general readiness to perform contracting work independently. But the approach is significant.

The researchers were not merely improving how an assistant described a procurement process. They were training it to carry out the process within the software, where success depends on a collection of interrelated decisions.

The distinction between producing a convincing output and developing a useful capability is becoming increasingly important in AI training.

The task is becoming part of the training data

Traditionally, much of post-training has relied on examples of desirable model behavior: prompts paired with target responses, comparisons between candidate outputs, and feedback about which responses better satisfy a request.

These remain central techniques. Supervised fine-tuning (SFT) can teach a model how to perform a task through demonstrations. Preference optimization can help it distinguish stronger outputs from weaker ones. Reinforcement learning can improve behavior through repeated attempts and reward.

What changes with more complex AI work is the nature of the experience being provided.

For an agent using specialized software, the training material may need to represent not just what the finished work should look like, but the actions available, the state of the environment, the constraints governing those actions, and the outcomes that count as success.

The Ironclad collaboration illustrates this approach. A model can practice configuring business workflows, receive feedback on what it accomplished, and improve through additional attempts. The environment makes parts of performance observable that a conventional prompt-and-response example could not establish.

This direction extends beyond software.

Ai2's ToyBlocks framework, introduced in November 2026, provides a controlled environment for studying robot learning. It combines automated task generation, manipulation demonstrations, and evaluation, allowing researchers to investigate how learned behavior changes across task configurations.

The two projects address different domains, but they share an important principle: training examples become more informative when they capture the conditions under which a capability must operate.

That does not mean every model needs an interactive environment. A well-constructed demonstration or preference dataset may be entirely appropriate for the target capability. The important question is whether the training experience contains the distinctions the model needs to learn.

A better reward does not necessarily mean better work

Giving a model opportunities to practice is only part of the challenge. The feedback it receives determines which behaviors the training process encourages.

An October 2026 study, Post-Training Frontier Text-to-Image Models by Composing Preference and Rubric Rewards, makes this problem particularly clear.

The researchers trained a reward model using approximately five million human preference votes from Arena. Those comparisons provided a broad signal about which generated images people preferred.

But preference alone could not adequately capture everything that made an image successful.

An image might be visually compelling while omitting an object from the prompt, placing elements incorrectly, or introducing content the user explicitly excluded. Optimizing aesthetic preference could improve one aspect of quality without adequately improving another.

The researchers therefore combined learned preference rewards with more targeted signals for prompt adherence, constraints, and undesirable optimization behaviors. Their approach improved performance in the image models they tested, although the findings do not establish a universally optimal way to combine rewards.

The deeper insight applies well beyond image generation.

A coding agent may produce a working program while introducing avoidable security weaknesses. A research agent may generate a convincing report while overlooking contradictory evidence. A computer-use agent may reach the requested state through an unnecessarily costly sequence of actions.

These are not equivalent failures, and a single training signal may not distinguish them.

This is why the quality of feedback matters as much as its volume. A broad preference judgment can capture qualities that are difficult to specify explicitly. A targeted check can establish whether a particular requirement was met. An outcome signal can verify that an operation succeeded. Each supplies different information.

Effective post-training depends on whether these signals reinforce the behaviors that genuinely matter, rather than simply the behaviors that are easiest to measure.

Learning the examples is not the same as learning the capability

Even when training improves performance, another question remains: how far does that improvement extend?

Two studies presented at ICLR 2026 challenge assumptions about the relationship between training performance and subsequent capability.

In Quagmires in SFT-RL Post-Training, researchers trained hundreds of models to investigate whether stronger supervised fine-tuning performance reliably leads to better results after reinforcement learning.

It did not.

They found cases where models with better SFT scores performed worse after reinforcement learning than models with weaker initial scores. Higher SFT performance could favor easier or more homogeneous training examples without adequately predicting the model's potential for subsequent improvement.

The experiments focused on mathematical reasoning, so their findings should not be treated as a universal rule for all model-training pipelines. Nevertheless, they expose an important limitation: a metric showing that a model has learned its current training examples may not tell us how well it is positioned to learn further.

A second study, Breaking Barriers: Do Reinforcement Post Training Gains Transfer to Unseen Domains?, examined whether reinforcement-learning improvements transferred to different reasoning domains.

The researchers found substantial improvements on tasks resembling the training data, but inconsistent transfer to unfamiliar domains with different reasoning patterns. Some gains disappeared altogether.

These findings do not diminish the value of reinforcement learning. They help establish what its results actually demonstrate.

Improving performance on a defined task distribution is valuable. Developing capabilities that remain effective across new contexts is a further challenge.

For training teams, this changes how we should think about data quality. Useful variation is not merely a matter of adding more prompts or rewriting existing examples. It can involve different constraints, unfamiliar situations, competing requirements, and cases where the obvious approach fails.

A training dataset can be internally consistent, accurately labeled, and still provide a limited picture of the capability it is intended to teach.

Evaluation establishes what the model has actually learned

Training and evaluation are closely connected, but they serve different purposes.

Training shapes model behavior through demonstrations, preferences, rewards, and other forms of supervision. Evaluation measures what the resulting model can do under specified conditions. Those findings can guide further training, but evaluation examples are not automatically training data.

As AI systems take on more varied work, evaluation must increasingly reflect the evidence and decisions involved in that work.

OpenAI's BioBasicsBench, published on November 6, 2026, is a useful example. Developed with biotech company Ligo, it tests AI agents on 100 routine laboratory-analysis tasks using real experimental material, including microscopy images, gel photographs, instrument readouts, and other laboratory records.

Unlike assessments built around carefully selected figures from published papers, these tasks include the ambiguity and imperfections of everyday experimental work.

The model must do more than recognize biological concepts. It may need to detect errors, interpret experimental controls, or establish when the available evidence cannot support a conclusion.

The benchmark shows progress in the models tested, while identifying limitations in practical scientific reasoning. Its value lies partly in making those limitations observable.

This is a crucial distinction for model development. If a model performs well on familiar examples but struggles with unfamiliar evidence, the next training intervention should be informed by that difference. If performance improves on one dimension while deteriorating on another, an aggregate score may conceal the trade-off.

Evaluation is not simply the final stage after training. It is one of the ways researchers discover what the next training stage needs to address.

The role of human expertise is changing, not disappearing

Synthetic training data, automated feedback, and verifiable rewards are allowing model developers to create and assess learning experiences at considerable scale.

But generating more examples does not automatically establish that those examples represent meaningful work, or that the resulting feedback rewards the right behavior.

Human expertise remains particularly valuable where the quality of an outcome depends on context, interpretation, or judgment.

In the Ironclad collaboration, specialists contributed their understanding of contracting workflows. In BioBasicsBench, scientists supplied real experimental evidence and reviewed what constituted correct interpretation. In image-model training, human preferences contributed a broad signal of perceived quality, complemented by more targeted automated assessments.

These are different forms of expertise entering different parts of the development process.

The same principle applies to the human-data work behind everyday model training and evaluation.

A strong supervised example demonstrates the intended behavior without introducing unsupported assumptions. A meaningful preference distinguishes differences that matter to the user's objective. A reliable multimodal annotation preserves the correct entity, event, or temporal boundary. An assessment of an agent trajectory separates what the system attempted from what it actually accomplished.

Such judgments are valuable because they preserve information that would otherwise be lost or misrepresented.

The challenge is not to maximize human involvement in every part of the pipeline. It is to identify where informed human judgment improves the training signal, establishes reliable evidence, or exposes limitations that automated processes may miss.

That requires contributors who understand the task, can interpret evidence precisely, and can make defensible judgments across increasingly diverse forms of AI work.

From better examples to more capable models

The research emerging through 2026 points toward a richer understanding of model training.

Demonstrations can teach useful behaviors. Interactive environments can give models opportunities to practice. Preferences and rewards can encourage improvement. Independent evaluations can establish whether those improvements hold beyond familiar training conditions.

None is sufficient on its own.

More examples do not guarantee better learning. Higher reward does not guarantee better work. Stronger benchmark results do not automatically establish broader capability.

The essential challenge is connecting the behavior we want, the experiences models learn from, and the evidence we use to judge what they have acquired.

As AI systems move further into professional workflows, scientific analysis, multimodal generation, and autonomous action, that connection becomes increasingly consequential.

The goal is not simply to train models that succeed on the examples we give them. It is to develop capabilities that remain useful when the work changes.

And doing that well depends on the quality of the tasks, feedback, and human judgments that make learning possible.

Data Gradient develops practical training for the people contributing to AI model training and evaluation. Through Data Gradient Learn, contributors build the judgment and technical understanding needed to produce reliable human data across increasingly complex AI workflows.

We use cookies to run the site and, if you allow it, to understand how it’s used. Essential cookies are always on. Privacy policy