Optimization over language: an optimizer that reads its rules in plain English

Every optimizer has the same blind spot. Routing, scheduling, matching, allocation — the math is powerful and well understood, but it runs on numbers, and half of what actually decides a good allocation is not a number. It's a sentence: a customer's note, a reviewer's preference, a policy line, a proposal's pitch. For decades the options were to throw that text away, hand-code it into brittle rules, or train a bespoke model on labels you didn't have. You could only optimize what you could quantify — so the qualitative half got dropped.

A model that returns a calibrated probability from text changes what an objective — and a constraint — can be. The number is trustworthy enough to put inside a solver (an uncalibrated LLM score isn't — feed it in and it corrupts the solution). To see whether that's real, I built the textbook case of “optimization over language” and pushed on it.

The setup: reviewer assignment, the honest way

Assigning papers to reviewers under load limits is a clean, exactly-solvable assignment problem (min-cost flow). The hard part was never the solver — it's the objective: “how well does this reviewer fit this paper?” lives in the abstract. The standard system, the Toronto Paper Matching System, approximates it with text similarity — a heuristic, not a judgment. The modern version of that heuristic is a semantic embedding, and that is the baseline to beat.²

The instance is real: 16 reviewers and 72 papers pulled from arXiv (cs.LG, cs.CL, cs.CV, cs.CR), with co-authorships as real conflicts.³ A calibrated decision model — TypeSafe's Jev¹ — reads each abstract against each reviewer's expertise and returns a calibrated expertise-fit, the objective coefficient. And it reads something an embedding cannot: a reviewer's rule written in plain English.

First, is the reading even valid?

Before optimizing a model's judgment you should check the judgment is real — otherwise you're just optimizing a vibe. So, independent of the solver: how well does each signal identify a reviewer who is genuinely in the paper's field (using arXiv's own categories as the answer key)?

The calibrated fit identifies true experts better than a real embedding model does (AUC 0.753 vs 0.681). That matters: it means the number we're about to optimize is a more valid measure of expertise than the incumbent's, before the optimizer touches it.

The thing no similarity metric can do

Now the constraints. Give some reviewers a rule in plain English — “I only review papers with a concrete security threat model,” “no purely theoretical papers,” “not a fit for computer-vision-only work.” A similarity metric cannot represent any of these; it has no notion of what a reviewer refuses. Jev scores each rule against every paper as a calibrated compliance, so a rule becomes a hard constraint the optimizer must honor. Solve the assignment four ways and count how many assignments break a stated rule:

The embedding baseline breaks 35 rules — not because it's careless, but because the rule is invisible to it. Turning the calibrated rules into constraints takes violations to zero, and — this is the part that matters — it costs nothing: expert coverage is actually the best of the four (69 of 72 papers get a same-area reviewer) and mean expertise-fit is the highest. You don't trade quality for compliance; you get both, because both come from reading the same text.

And it holds as the problem changes shape. Vary how many reviewers each paper needs and the embedding keeps breaking more rules while the calibrated-plus-constraints solve stays at zero.

Why this is new

None of the optimization here is novel — assignment problems are a solved art. What's new is the input. For the first time you can put, into a rigorous optimizer, an objective and constraints that exist only as language — cheaply (this whole study cost under a nickel of model calls), calibrated (so the solver isn't optimizing noise), and validated (the reading beats the incumbent's own signal). That unlocks the qualitative half of problems optimization has always had to ignore: matching, scheduling, routing, and allocation where the priority, the fit, or the rule was written down in words a spreadsheet couldn't hold.

You can drive the whole thing yourself — toggle a rule, switch the objective, and watch the assignment re-solve exactly in your browser.⁴ The model reads the text into numbers; the solver does the math; you own the constraints. That division of labour — a model for judgment, a solver for the optimum — is what we mean, at BIS, by decisions engineered.

Built in the open — with TypeSafe

An independent experiment on TypeSafe's System One model, Jev¹ — not sponsored, not affiliated. Honest caveats: the instance is small; the expertise-fit is a calibrated reading, not ground truth (which is why the independent validity check matters); and the rules were scored offline so the live demo offers a fixed menu rather than free-text entry. The mechanism, though, is clear — and cheap enough to run at real scale.