Data generation via RL environments: how experts create new training data
RL environments create brand-new training data by having experts perform real processes. This guide explains the data they produce and why it beats scraping.
RL environments generate new training data by having experts perform real processes inside a simulated version of their job and recording everything that happens. Each run produces a trajectory, the full sequence of states, actions, and rewards, along with expert demonstrations, reward signals, and evaluations. The result is brand-new, proprietary data that captures how expert work is actually done, which is something you cannot get by scraping finished documents off the internet.
This is the deeper reason labs now pay for environments rather than static datasets. An environment is not just a place to train an agent; it is a factory that manufactures training data on demand. This guide explains the kinds of data it produces, why that data beats scraping, and why the real constraint is the supply of experts. For the foundations, start with our hub on reinforcement learning environments.
From copying data to generating it
The first era of machine learning ran on data that already existed: text, images, and labels scraped or collected from the world. That data is finite, largely used up for the obvious cases, and fundamentally backward-looking. It shows outputs, not the process that produced them.
Generating data is different. Instead of finding examples, you create the conditions for expert work to happen and record it. An RL environment is the mechanism: a realistic sandbox where a task is performed, scored, and retried. Every performance is new data that did not exist before. The classic software interface for this loop, popularized by OpenAI Gym and now maintained as Gymnasium, exposes two operations, reset and step, that let a system run a task over and over and collect the results. For how this differs from buying a fixed dataset, see our companion piece on companies building RL environments in 2026.
The four kinds of data an environment produces
An RL environment manufactures four distinct data types, and each one answers a different question for the model being trained.
Trajectories
A trajectory is the complete record of one task attempt: the starting state, every action taken, the state after each action, and the reward at each step. Trajectories are the raw material of reinforcement learning because they show not just the answer but the path to it, including dead ends and recoveries. A model trained on trajectories learns sequencing and decision-making, not just pattern matching.
Demonstrations
A demonstration is a gold-standard trajectory performed by an expert: the ideal way to do the job, start to finish. Demonstrations teach an agent what good looks like before it starts exploring on its own, which dramatically shortens training. They also encode the judgment calls that never appear in any manual. We cover how to capture these at scale in collecting expert demonstrations for AI at quality.
Reward signals
The reward signal is the score the environment assigns to each attempt, and it is the single most important piece of data because it defines what the agent optimizes for. A reward signal is only as good as the rubric behind it, and that rubric has to come from someone who knows what correct means in the domain. Research hosted at arXiv on reward design shows how sensitive trained behavior is to small changes in how reward is defined.
Evaluations
An evaluation is an expert’s judgment of an agent run: did it pick the right tools, follow a compliant process, and reach a correct and safe outcome? Evaluations catch failures that a numeric score alone misses, such as an answer that is technically scored as passing but would fail in the real world. This is the same expert-judgment problem as model evaluation, scaled to a full workflow, and we explore it in evaluating AI agents with domain experts.
Comparison: scraped static data vs environment-generated data
| Dimension | Scraped static data | Environment-generated data |
|---|---|---|
| What it captures | Finished outputs | The full process: actions, decisions, outcomes |
| Freshness | Goes stale as the world changes | Generated fresh on demand |
| Ownership | Public or shared, rarely proprietary | Proprietary to whoever built the environment |
| Fit to task | Generic | Tuned to the exact job you care about |
| Teaches sequencing | No | Yes, through trajectories and rewards |
| Judgment captured | Only what leaked into the output | Explicit, via expert demonstrations and rubrics |
| Main cost | Collection and cleaning | Expert supply to perform, define, and grade |
The table makes the trade-off clear. Scraped data is cheap to start with and poor at teaching process. Environment-generated data is expensive to produce and unmatched at teaching an agent how to actually do a job. As the easy public data runs out, the second column is where the next gains come from.
The fake company idea
The sharpest version of data generation is the “fake company.” A team builds a realistic replica of a real job or business: the tools, the data, the systems, and the workflow. Experts then perform the actual process inside it, generating trajectories and demonstrations, while the environment scores each attempt. The agent practises the same job and is rewarded for doing it correctly.
It is called a fake company because the environment imitates a real operation closely enough to produce authentic work data without touching live production systems or real customers. A fake company might replicate a SaaS support desk, a law firm’s contract review process, an insurer’s underwriting workflow, or a finance team’s monthly close. The closer the replica is to the real job, and the more real the experts performing inside it, the more valuable the data that comes out. Standards work like the NIST AI Risk Management Framework is relevant here, because a replica that rewards the wrong behavior will train an agent to do the wrong thing with confidence.
Why this beats scraping
Three reasons environment-generated data wins for agent training:
- Process over output. Scraping shows you a finished contract or a closed support ticket. It does not show the fifteen decisions that produced it. An environment records every decision, which is what an agent has to learn.
- Proprietary and fresh. Data you generate is yours and reflects current practice. Scraped data is shared, backward-looking, and increasingly exhausted for high-value tasks.
- Verifiable reward. Recent reasoning models improved sharply by training against tasks where success can be checked. Environments extend that from math and code to real professional workflows, as long as an expert defines what success means. For the market context around this shift, see data for AI agents and expert data for AI training.
The bottleneck: expert supply
None of this works without the right people. The simulation is engineering and can be built. The data is only valuable if real practitioners perform the task to create demonstrations, define the rubric that becomes the reward signal, and grade the agent’s attempts. In regulated or technical domains this has to be a verified professional, not a generalist, because only a domain expert can define and judge correct behavior. A confident wrong answer in medicine, law, or finance looks fine to a non-expert and fails in the real world.
That is the real constraint on data generation in 2026. There is no shortage of engineers who can build a sandbox. There is a shortage of verified experts available to perform, define, and grade the work inside it at the volume frontier training needs. The ceiling on the data, and therefore on the trained agent, is set by the quality and verification of those experts.
The economics make the bottleneck sharper. Domain experts command real rates, roughly 85 to 200 US dollars an hour for most professions, and 250 to 450 for medical and legal specialists, so every hour of expert time has to produce data that earns its cost. That rewards careful selection and verification over volume. A single contract-review demonstration from a practising attorney who understands the edge cases is worth more than a hundred generic annotations, because it teaches the agent the judgment that matters and keeps it from learning a confident but wrong shortcut. Getting the supply right is not a nice-to-have; it is the difference between data that trains a usable agent and data that quietly teaches it to fail.
Where CleverX fits
CleverX supplies the verified domain experts who generate the data inside RL environments. Practising professionals across finance and accounting, healthcare, law, engineering and manufacturing, insurance, and operations perform the task to produce trajectories and demonstrations, write the rubrics that define the reward signal, build benchmarks from their real work, and evaluate agent runs. Every expert is verified through government ID, professional licence confirmation, LinkedIn experience matching, and a recorded interview, drawn from a pool of more than 10 million verified professionals. For the full picture of this layer, see domain experts for AI training and evaluation.
An RL environment is a factory for new training data, and the experts inside it are the workers who make the product worth anything. Build the environment for the engineering, and solve the expert-supply question deliberately, because that is the part that decides whether the data you generate is a genuine asset or an expensive pile of plausible-looking noise.
Frequently asked questions
How do RL environments generate training data?
An RL environment generates training data by having an agent, or an expert, perform a task inside a simulated version of real work and recording everything that happens. Each run produces a trajectory: the sequence of states, actions, and rewards from start to finish. Experts seed the environment with gold demonstrations, define the reward signal that scores each attempt, and evaluate the results. The output is brand-new, proprietary data that captures how the work is actually done, rather than data copied from somewhere else.
What kinds of data come out of an RL environment?
Four main types. Trajectories are full step-by-step records of a task attempt including the reward at each step. Demonstrations are gold-standard runs performed by experts that show the ideal way to do the job. Reward signals are the scores that tell the agent how well it did against an expert-defined rubric. Evaluations are expert judgments of agent runs that measure quality and catch failures a score alone would miss.
Why is environment-generated data better than scraped static data?
Scraped static data shows finished outputs but not the process, decisions, or judgment behind them, and it goes stale as the world changes. Environment-generated data captures the full process: every action, the reasoning behind it, and whether it succeeded, which is exactly what an agent needs to learn a multi-step job. It is also proprietary, fresh, and tuned to the specific task you care about, rather than a generic pile of public examples.
What is the fake company idea in RL training?
The fake company idea is building a realistic replica of a real job or business, the tools, data, systems, and workflow, and having experts perform the actual process inside it. The agent then practises the same job and is rewarded for doing it correctly. It is called a fake company because the environment imitates a real operation closely enough to generate authentic work data, without touching live production systems or real customers.
What is the main bottleneck in generating RL environment data?
The expert supply. The simulation is engineering and can be built, but the data is only valuable if real practitioners perform the task to create demonstrations, define what correct looks like, and grade the agent’s attempts. In regulated or technical domains this must be a verified professional, not a generalist, because only a domain expert can define and judge correct behavior. Finding and verifying enough of these experts is the hard part.
How does CleverX support data generation via RL environments?
CleverX supplies the verified domain experts who generate the data inside RL environments. Practising professionals across finance and accounting, healthcare, law, engineering and manufacturing, insurance, and operations perform the task to produce trajectories and demonstrations, write the rubrics that define the reward signal, build benchmarks from their real work, and evaluate agent runs. Every expert is verified by government ID, professional licence, LinkedIn matching, and a recorded interview.
RL environments do more than train agents: they manufacture the proprietary, process-rich data that static scraping can never provide. The one input they depend on is verified experts to perform, define, and grade the work. That is exactly what CleverX supplies.