CoreWeave Bridges Production Inference and Model Training in Unified Platform
CoreWeave has unveiled tools to connect real-world model failures directly to retraining workflows, letting teams iterate on deployed models without full redeployment cycles.

Real deployments surface problems that lab benchmarks often miss. These failures reveal which edge cases matter most and what users actually need. Acting on that production signal requires a strategic choice: should the model itself be modified, or does the issue lie elsewhere—in retrieval systems, context handling, prompts, tools, or the serving layer?
Adjusting model weights cannot solve problems rooted in other parts of the system. When the model genuinely needs to change, production data can drive both quality gains and cost improvements. A lighter model might match a heavier one's performance; better training examples could boost consistency; outcome-based rewards might align better with the task. Refining an existing capable model can also eliminate the expense and risk of migrating to a newer model version.
Selecting the right training method from observed failures
Model distillation works when a stronger model already demonstrates the desired behavior and you want a smaller, cheaper model to replicate it. Supervised fine-tuning (SFT) applies when high-quality labeled examples can teach more reliable behavior. Reinforcement learning (RL) suits situations where you can measure outcomes reliably but cannot easily specify the ideal response upfront.
Changing weights won't fix a failure elsewhere in the system. When the model is the right place to intervene, production evidence can improve both quality and economics.
Teams often employ all three methods across a model's lifetime. Ground your choice in evidence from your actual workload, even if that means adopting a technique requiring more setup work.
Converting production traces into training data is essential. Post-training demands computational resources and workflow orchestration, and the retrained model must be tested before returning to serving. This completes the feedback loop. In RL scenarios, inference generates training rollouts during the run, and updated weights must flow back to serving continuously.
At Fully Connected 2026, CoreWeave introduced CoreWeave Model Distillation, CoreWeave RL Rollouts on CoreWeave Inference, and a programmable training API. These capabilities integrate within CoreWeave Forge. The programmable API is currently in limited preview, with a full launch targeted for later in 2026.
Refreshing models with new production data while keeping the base unchanged
CoreWeave Model Distillation offers a managed, end-to-end workflow. CoreWeave Agent Lens monitors inference traffic as the application operates. The system transforms this traffic into training data, re-evaluates weak outputs, tests different base models and settings, deploys candidate versions to Serverless Inference, and benchmarks them against the current production model.
If your model already possesses the capabilities your application requires, fresh production data can sharpen its performance without altering the serving configuration you depend on. Test the improved version against your live model before shifting traffic to it.
Method demonstrated this approach by refining a previously deployed model. In 2024, the company used OpenPipe to fine-tune Llama 3.1 8B for navigating bank interactive voice response (IVR) systems, training it on GPT-4o outputs. In August 2026, Method incorporated fresh production data, re-labeled it using GPT-5.6 Sol, and trained a new version. The new candidate won 57.5% of head-to-head evaluations against the deployed version.
The underlying base model remained constant. Fresh production data combined with Model Distillation in Forge enabled another iteration cycle faster than before. The 57.5% figure represents the proportion of head-to-head evaluations won by the new version.
Writing custom training logic in Python
Teams wanting to design their own training process can use the programmable training API for Serverless Training, which decouples training logic from infrastructure management. You define a Python loop; CoreWeave handles compute allocation and the underlying training systems.
Here's an example combining RL updates with periodic supervised updates on verified examples. The code initializes a learner and serving adapter, configures an Adam optimizer with a learning rate of 1e-5, and publishes updated weights to the sampler after each training step.
Python
learner = await training.create_learner(model=model, lora=lora)
sampler = await serving.create_adapter(name="sampler")
optimizer = AdamConfig(learning_rate=1e-5)
for step in range(num_steps):
trajectories = await run_agent(sampler, tasks)
update = learner.batch()
update.forward_backward(
build_rl_batch(trajectories), loss="importance_sampling"
)
update.optim_step(optimizer=optimizer)
if (step + 1) % sft_interval == 0:
update.forward_backward(next_demo_batch(), loss="cross_entropy")
update.optim_step(optimizer=optimizer)
update.publish_weights(to=sampler)
await update.result()
await (await learner.save_state("final")).result()
await learner.close()
The rollout and batch helpers shown are simplified illustrations. Your agent implementation, reward calculations, and data handling logic belong in those helper functions. The loop applies importance-sampling loss for RL and cross-entropy loss for supervised examples, then waits for the update to complete. It persists the final learner state and terminates the learner.
You control how frequently supervised examples enter the training loop, how rewards are computed, and how the training recipe evolves. SFT, RL, and distillation all use the same training primitives, while the platform manages the distributed infrastructure underneath.
Willow started with supervised learning, then incorporated RL scorers to reduce filler language, enhance readability, and eliminate self-corrected phrases. These scorers merged into a unified reward function. Lawrence Liu, Willow's chief technology officer and co-founder, reported improvements in text formatting, emoji handling, and command responsiveness compared to the deployed model.
Eliminating checkpoint reload overhead in reinforcement learning
RL serving requires sufficient rollout generation, alignment with the policy under training, and rapid weight updates. The trainer, reward computation, and environment can remain in place while CoreWeave Inference manages the serving component of the loop.
CoreWeave RL Rollouts builds on Nvidia Dynamo with vLLM integration. When the trainer saves a checkpoint, the service loads it into the active deployment without interrupting ongoing requests or availability. It returns training metadata: token log probabilities (logprobs), token IDs, and expert routing information for mixture-of-experts (MoE) models.
When the trainer writes a checkpoint, the service hot-loads it into a live deployment while preserving in-flight requests and endpoint availability. Serving uses the trainer's current policy weights, removing staleness and redeployment delays that would otherwise hold up the next batch of rollouts.
Checkpoints can be full snapshots or incremental updates, minimizing synchronization overhead after the initial transfer. Serving operates with the trainer's latest policy weights, eliminating staleness and the redeployment delays that would otherwise slow the next rollout batch.
Tracking quality improvements and infrastructure efficiency separately
CoreWeave validated this workflow together with Nvidia and you.com by post-training Nvidia Nemotron 3.5 Lightning for web search. The training run combined SFT and RL, integrating you.com's search tools directly into the training process.
BrowseComp accuracy climbed from a 36.97% starting point to 45.45%, representing a 22.94% relative gain. Tool calls dropped by 30.24% on average. In parallel, loading checkpoints into an active deployment ran approximately 15 times faster than traditional redeployment approaches.
Improved search tooling, post-training optimization, and harness enhancements combined to boost accuracy and reduce tool usage. Hot-loading slashed the time spent synchronizing checkpoints. For teams attempting to make a smaller or more efficient model competitive with models orders of magnitude larger, that additional training velocity becomes critical.
Maintaining the connection between models and experiments
CoreWeave Forge links production inference, training, and serving back together. Model Distillation provides a managed pathway. The programmable API grants researchers control over the training loop without handling infrastructure concerns. RL Rollouts enables teams to maintain their own trainer while leveraging CoreWeave for rollout serving.
Models generated by serverless training emerge as tagged artifacts that transition into Serverless Inference without requiring a separate registry step. Experiments stay logged alongside them, keeping the training run, resulting model, and eventual deployment linked together.
Choose a failure you can reproduce, use it to guide the training method, and compare the candidate with the model serving users.
Identify a failure you can consistently reproduce, let it inform your choice of training method, and evaluate the candidate against the model currently serving users. Preserve the production examples, evaluation results, and resulting artifact together so your team can reference them later.
The CoreWeave Forge blog and announcement press release offer additional context on the capabilities unveiled at Fully Connected 2026. CoreWeave intends to publish further technical explanations covering Model Distillation, the programmable training API, and RL Rollouts.