Ultramarine 1 Flash - a cheap agent model for knowledge work

Ultramarine 1 Flash: Scaling Non-Verifiable RL

Paper Instruments technical report, 14th October 2026

https://www.paperinstruments.com/blog/ultramarine

Ultramarine 1 Flash is an AI model for professional knowledge work: board papers, financial models, legal drafts, consulting decks. Paper Instruments built it on the open model GLM-5.3 Flash from Z.ai, then trained it with reinforcement learning on tasks that have no single right answer.

The headline: on their internal knowledge-work suite it scores 59.3 at $0.13 per task. Claude Fable 5 scores 57.8 on the same suite at $8.19.

The report matters for another reason too. It explains how to train on "non-verifiable" work, where a grader can't check the answer the way a test checks code.

Key facts

Results

Selected scores from the report. Higher is better.

Cost per task on the PI Knowledge Work suite:

How to read it:

• It beats its own base model on all 6 knowledge-work benchmarks, by 4.1 points on average, with 6.5% fewer tokens.
• It's within 3 points of Claude Opus 5.5 and GPT-6 Sol on the internal suite, at about 2% to 10% of their cost.
• PI Knowledge Work is Paper Instruments' own suite. The public benchmarks show a smaller lead, and Gemini 3.8 Flash wins 2 of them.
• It got slightly worse at code. That's expected: it was trained for documents, not software.

How they trained it

Tasks with no right answer

Code has tests. A board memo doesn't. Paper Instruments grades each task with a rubric: a list of weighted yes-or-no criteria, for example whether a board memo separates what the board can approve now from what still needs legal clearance. An LLM judge marks each criterion met or not met, and the weighted sum is the reward.

They considered 3 sources of tasks:

• Real user traces. Rejected: users leave too much unsaid, and rebuilding the context is hard.
• Expert-written tasks. Good, but slow and expensive.
• Synthetic tasks. Chosen. A generator combines practice area, documents, instruction and rubric, which gives 10^7 to 10^10 possible tasks per practice area.

The worked example in the report: update a board pre-read for "Project Aurora" after a revised forecast. The rubric has 31 weighted criteria across factual accuracy, analysis, completeness, source fidelity, communication and instruction following.

Feather, the agent harness

The model trained inside Feather, their own Python agent harness. It has a short tool set: read, edit, write, grep, glob, bash with background tasks, and a todo list. The todo list survives context compaction, so the agent keeps its plan on long tasks.

2 choices stand out:

• A 50-token system prompt in training. Production harnesses use 1,800 to 4,400 tokens. Their view: long prompts limit exploration. Production instructions came in only at the end of training.
• Each task runs in an E2B sandbox, started from a snapshot. The agent loop stays outside the sandbox.

Paper Office

Agents that edit Office files with python-docx, python-pptx or openpyxl hit gaps in the library, drop into raw XML, and corrupt files without an error. Paper Instruments forked all 3:

The forks keep the import names. An edit either completes or raises a typed error and leaves the file unchanged. paper-docx adds a redline: compare 2 versions and get a Word file with tracked changes.

For business people

• What it is: a cheap model trained to do the document work of analysts, lawyers and consultants, not to chat or code.
• Cost: about 1/60th of Claude Fable 5 per task on the maker's own test, for a similar score. Pay as you go from a prepaid balance.
• Where it fits: high-volume drafting and updating, such as first drafts of memos, model updates and diligence summaries, where a human reviews the output.
• Risks:
  • The best results come from the maker's own benchmark. Test on your own documents before you believe the gap.
  • It's a small company's API. No public information on the team, funding or data processing location beyond a SOC 2 Type II report.
  • No image input, so no reading of scanned PDFs or charts.
  • Client documents go to a third-party API. Check the terms before you send anything confidential.

For technical people

Call it

OpenAI SDK, base URL https://api.paperinstruments.com/v1:

import os
from openai import OpenAI

client = OpenAI(
    base_url="https://api.paperinstruments.com/v1",
    api_key=os.environ["PAPERINSTRUMENTS_API_KEY"],
)
response = client.chat.completions.create(
    model="ultramarine-1-flash",
    messages=[{"role": "user", "content": "Explain EBITDA in 2 sentences."}],
    reasoning_effort="max",
    max_completion_tokens=8192,
)
print(response.choices[0].message.content)

The Anthropic SDK works too, with base URL https://api.paperinstruments.com.

In Claude Code

Ultramarine speaks the Anthropic Messages format, so Claude Code can run on it:

export ANTHROPIC_BASE_URL=https://api.paperinstruments.com
export ANTHROPIC_API_KEY=$PAPERINSTRUMENTS_API_KEY
export ANTHROPIC_MODEL=ultramarine-1-flash
export ANTHROPIC_DEFAULT_OPUS_MODEL=ultramarine-1-flash
export ANTHROPIC_DEFAULT_SONNET_MODEL=ultramarine-1-flash
export ANTHROPIC_DEFAULT_HAIKU_MODEL=ultramarine-1-flash
export CLAUDE_CODE_SUBAGENT_MODEL=ultramarine-1-flash
export CLAUDE_CODE_DISABLE_EXPERIMENTAL_BETAS=1
export ENABLE_TOOL_SEARCH=false
claude

Gotchas

• The API reserves the maximum cost before it answers. Without max_completion_tokens, it reserves funds for the full output ceiling and can return 429 insufficient_quota on a small balance.
• Claude Code settings that return 400: effort medium or xhigh (use low, high or max), MAX_THINKING_TOKENS, more than 32,768 output tokens, tool search, and the 1-hour prompt-cache TTL.
• Any other model name returns 404.
• Paper Office replaces the originals. paper-docx and python-docx both install as docx. Never install both in 1 environment.
• GET https://api.paperinstruments.com/v1/pricing returns the current prices without a key.

My take

The Paper Office forks are the part I'd use first. python-docx and python-pptx sit under my proposals and decks, and silent corruption after an agent edit is exactly the bug that costs an evening. A drop-in fork that refuses an unsafe edit instead of guessing is worth a test on its own, see Python Library: openpyxl and Convert Markdown files to Microsoft Word for where those libraries already sit in my stack.

The model is a cheap second opinion, not a replacement. At $0.13 a task, it's worth a run on high-volume drafts, like call-prep summaries or a first pass on a long RFP. The proof would be my own documents, graded against Claude, not their benchmark.

The rubric method is the real lesson. Weighted yes-or-no criteria are how I'd check NicAI's own outputs too, and their open-source rubric package does exactly that.

Further reading

Paper Instruments docs

Ultramarine API, Paper Office packages and Rubric

https://docs.paperinstruments.com/

paper-instruments/feather-train

The Feather harness, training integrations and evaluation tools

https://github.com/paper-instruments/feather-train

• paper-docx on GitHub, the agent-safe python-docx fork
• rubric on GitHub, LLM grading with weighted rubrics
• Paper Instruments console, for API keys and billing