Independent research · Reasoning models
Reality is a line of open reasoning models trained on olympiad-level competitive programming. The work targets a specific failure: models that derive a correct algorithm and then never commit it to working code.
Large models are good at the hard part of an olympiad problem — finding the algorithm. They are unreliable at the last mile. A model will reason its way to the right recurrence, state the invariant correctly in prose, and then write code that contradicts it. Or it will keep reasoning until it runs out of budget and submits nothing at all.
That gap is not fixed by making the model larger. It shows up in the transition from reasoning to output, and it is measurable: a submission either passes a judge's hidden tests or it does not.
Reality models are parameter-efficient adapters over open base models, trained in three stages: distillation of verified reasoning traces, reinforcement learning against real test-case outcomes, then a second distillation pass on hard, verified solutions to encourage direct and committed answers.
The reward is not a preference score. A rollout is graded by executing it against a strengthened hidden test suite, which makes the training signal falsifiable and makes dataset quality the binding constraint on everything downstream.
An adapter for an open 27B base model, specialized for olympiad-style algorithmic reasoning. Emits a full reasoning trace followed by a Python solution. Weights on Hugging Face.
1,005 competitive-programming problems rebuilt into verifiable reward data. Each carries a reference solution checked against the original tests, plus a regenerated suite of twelve adversarial cases built to separate correct solutions from ones that merely look correct.
A streaming cleaner for JSONL fine-tuning corpora — deduplication, malformed-record handling, and filtering at dataset scale without loading the file into memory. Install from PyPI.
A paper documenting what the fine-tuning did and did not buy, including reasoning-to-output failures, non-termination on hard problems, and the decoding-temperature confound that makes such results easy to misreport.
Training models to use execution feedback. Given its own failing submission and the exact judge verdict, a model should localize the fault and make a minimal correct edit. In practice it usually rewrites from scratch, or repairs the speed of a solution while quietly breaking its correctness. Closing that loop is the current line of work.