The task: turn documents into structured data 

Many business workflows need to turn documents into structured data. The model must find the right values, capture the requested records, and return output the next system can use. Long documents and repeated tables make this harder. We tested gpt-oss-20b and gpt-oss-120b to explore lower-cost alternatives to frontier models. In our baseline tests, both fell short on field coverage. gpt-oss-20b also declined the task or returned unparseable output. We wanted to see whether task-specific fine-tuning could improve these results. 

In Figure 1, we show an invoice that we held out from training. We can quickly spot the invoice number, customer, due date, and balance, but our application must map those values to the correct schema fields, preserve all 37 line items, and return complete, valid JSON. A wrong or missing value can trigger downstream errors or manual review, while invalid JSON can break the workflow altogether. 

Figure 1. Reconstructed excerpt from a public held-out invoice. Highlighted spans show representative values the model must find; the full evaluation uses the complete document and schema.

We used schema-guided extraction. We gave the model document text and a JSON Schema defining the required fields, then asked it to return one JSON object. Our grader compared normalized field-path/value pairs with the expected answer, counting repeated occurrences. 

We measure precision as the share of extracted pairs that match the expected answer, and recall as the share of expected pairs we recover correctly. We use F1 to combine the two into one score, using their harmonic mean. F1 ranges from 0 to 1, with higher scores indicating better extraction. A wrong value hurts precision and can also hurt recall if the expected value remains unmatched. A missing value hurts recall. 

What changed after fine-tuning 

We evaluated the base and fine-tuned versions of gpt-oss-20b and gpt-oss-120b on the same 20 documents. Both improved their extraction F1. gpt-oss-20b rose from 0.4379 to 0.8115, and gpt-oss-120b rose from 0.6816 to 0.8573. 

gpt-oss-20b made the larger gain, with an 85.3% relative improvement after 100 GRPO steps. Fine-tuned gpt-oss-120b finished with the higher F1. Fine-tuning helped both models, but the size of the improvement differed. 

We also evaluated GPT-5.5, GPT-5.6-sol, and GPT-6-sol as reference models. They scored 0.9760, 0.9766, and 0.9764, respectively—still higher than our fine-tuned models. gpt-oss-20b reached 83.1% of GPT-5.6-sol’s reference F1. This is a substantial improvement for gpt-oss-20b and a promising starting point for further task-specific tuning. Our base and tuned scores are averaged across repeated evaluations, while the reference models used a different single-pass protocol. These reference scores provide useful context for the progress we made, with the evaluation differences noted above. 

Figure 2. Macro F1 for gpt-oss-20b and gpt-oss-120b before and after fine-tuning on the same 20 documents, with frontier models as references. Base and tuned scores average repeated evaluations; frontier references use a different single-pass protocol.

These improvements make lower inference-cost base models worth a closer look. Table 1 compares published Azure inference prices. The pricing comparison is intended to provide inference-cost context only and should not be interpreted as a complete total-cost-of-ownership analysis, which would also include training, hosting, retries, and operational costs. 

 

Table 1. Extraction macro F1 and published Azure inference prices (USD per 1M tokens); GPT-5.5, GPT-5.6-sol, and GPT-6-sol are included as frontier-model references  

Notes: Prices checked October 9, 2026, in USD; input rates are uncached. gpt-oss-20b prices are published fine-tuned inference rates for Regional deployments. gpt-oss-120b prices are base-model rates; its fine-tuned rates remain unconfirmed. Frontier prices use Global Standard pricing, with the short-context tier for GPT-5.6-sol and GPT-6-sol. These rates do not represent measured deployment costs for our experimental checkpoints. Token rates exclude training and hosting costs. F1 scores come from fine-tuned checkpoints for the two gpt-oss models and reference runs for the frontier models. All use the same 20 documents, but tuned scores average repeated evaluations while frontier references use a different single-pass protocol. 

We observed more parseable output and higher coverage 

We looked through gpt-oss-20b’s responses and found examples where it declined the task or said it could not finish. We tracked this with a “refusal-wording rate”: the share of responses where our grader detected an apology or cannot-complete language. This measures wording, not whether the model made a safety refusal. We tracked JSON parseability and field coverage separately, although these problems can overlap. After fine-tuning, the refusal-wording rate (Refusals) fell from 25% to zero; JSON parseability (Valid JSON) rose from 63.8% to 98.8%, and coverage rose from 47.1% to 94.9%. Valid JSON means our grader could read the response as an object or array.   Coverage compares the number of predicted field/value pairs with the expected count, capped at 100%.  Wrong or extra pairs can raise coverage, so we read it alongside F1 rather than treat it as a measure of correctness.

Figure 3. Extraction quality and output behavior before and after fine-tuning on the same 20 documents. Bars show means across repeated evaluations. “Valid JSON” measures JSON parseability, not schema compliance or value correctness. “Refusal rate” measures detected refusal wording, not safety refusals. Higher is better except for refusal rate.

Figure 4 shows how these measures changed during training.  We repeated the full training run with a second seed.  It reached 0.8101 F1, close to the headline run’s 0.8115. We ran four temperature-0 evaluations for each seed. Across all eight post-training evaluations, our grader detected no refusal wording. 

 

Figure 4. gpt-oss-20b’s training progress, evaluated on 20 documents held out from training. (A) Refusal-wording rate falls as JSON parseability and field coverage improve. (B) Macro F1 rises and then levels off. Error bars show standard deviation across four temperature-0 evaluations.

What we learned from the model comparison 

We saw different starting points and different gains between the two models. gpt-oss-20b had clearer output problems before training: it sometimes declined the task, returned unparseable JSON, or missed required information. After fine-tuning, we saw improvements in extraction F1, JSON parseability, and field coverage, along with less refusal wording. 

gpt-oss-120b started with fewer format problems. We detected no refusal wording, and 96.3% of its baseline responses were parseable JSON. But parseable output did not mean complete or correct extraction. After fine-tuning, its field coverage rose from 74.1% to 94.1%, and its F1 rose from 0.6816 to 0.8573. gpt-oss-20b made the larger F1 gain, while fine-tuned gpt-oss-120b finished with the higher F1. 

For us, the lesson is to measure the problem first. Try simpler, lower-cost changes, such as prompting, before committing to training. Then consider fine-tuning when your grader can reward the improvements your application needs. How we trained with Foundry Interactive Training 

We used the Foundry Interactive Training API [1] with 17 training documents, expected structured answers, and a Python grader.  We followed the cookbook to connect our grader to Foundry’s managed generation and training operations. We ran 100 group relative policy optimization, or GRPO, steps and saved checkpoints for evaluation. 

For each document, we had gpt-oss-20b generates eight candidate answers. Our grader awarded partial credit for correct values and penalized invalid JSON. GRPO used the candidates’ relative scores to reinforce better answers.   
We used the expected answers only for grading. We did not include them in the inputs we gave the model during reinforcement training. The model had to produce its own answers, and the grader supplied the feedback. 

Figure 5. Customers define the application outcome and quality bar; Foundry Interactive Training runs the post-training loop in Microsoft Foundry.

How customers can use Foundry Interactive Training 

To try this approach, bring four ingredients: a supported base model, examples from your workload, expected outcomes, or another way to check the answers, and a Python grader. Your grader should measure what matters to the application—not just whether an answer looks convincing. 

1. Get connected. Request access to the Microsoft Foundry Interactive Training public preview [2] and connect the onboarding cookbook [3][4] to an assigned Foundry project. 


2. Separate your data. Build training, validation, and final test sets. Use validation data to choose checkpoints. Keep the final test set for an independent check, with new sources and document structures where possible.


3. Build your grader. Implement a grader that rewards the desired outcome. For our extraction task, we checked normalized field-path/value pairs and penalized unparseable JSON.


4. Run a bounded experiment. Set a training budget, inspect the training signal, and compare saved checkpoints on the validation set. Once you choose a checkpoint, evaluate it on the untouched final test set.


5. Evaluate and prepare for deployment. Use session sampling to test your LoRA adapter on held-out prompts and compare it with the baseline. Save checkpoints so you can continue experimenting. Before using the model in an application, confirm the supported deployment path, model compatibility, and serving costs.

Takeaway 

With Foundry Interactive Training, we improved extraction F1 for both gpt-oss-20b and gpt-oss-120b. We brought our task examples and a Python grader, and Foundry managed the generation and training loop. gpt-oss-20b made the larger gain, while fine-tuned gpt-oss-120b reached the higher F1. We did not match the frontier references, but our results show why task-specific fine-tuning is worth exploring before switching to a higher-priced model. The final choice should depend on independently tested quality and the full cost of deployment.

Reference 

[1] https://github.com/microsoft-foundry/fine-tuning 
[2] What is interactive training (preview) in Microsoft Foundry? – Microsoft Foundry | Microsoft Learn 
[3] Interactive Training – Public Preview Intake Form 
[4] https://github.com/microsoft-foundry/fine-tuning/tree/main/interactive_training