The model that won't write: reviewing Jev from outside the waitlist

What I tried

I usually review a product by using it. TypeSafe AI made that impossible this week: Jev, the model it launched on September 15 after two years in stealth, sits behind a waitlist, and even the public share link to a recorded playground query redirects to a login wall. The company is still worth reviewing — a $40M seed led by DCVC, a reported valuation around $200M, and a CEO, Diogo Almeida, who co-authored the InstructGPT paper that taught language models to follow instructions. The man who helped teach models to talk is now selling one that refuses to. Jev is what TypeSafe calls a System One model. You send it a state — an email, a log line, a JSON blob, up to about 32K tokens — plus typed questions, and it returns exactly three primitives: Choice picks an option from your list, Score rates the state on your rubric, and Noul returns the probability that a yes/no claim is true. Every answer carries a probability distribution and a confidence; there is no string output to parse. All questions in a call are evaluated in a single parallel pass, which is why input is priced at $0.042 per million tokens and output is unmetered — 'too cheap to meter,' the company says. So I reviewed the evidence instead of the model. My desk: TypeSafe's launch post, documentation, public evals dashboard, manifesto and its pre-launch essay against benchmarks; the only independent hands-on anyone has published — Every's head of evals, Mike Taylor, plus a second test by CEO Dan Shipper; three exported channels of TypeSafe's own Discord, where early users post their numbers; and the ~1,600-point Hacker News thread where Almeida fielded the objections himself. Every figure below carries its source's name. No API key was issued for this review, so nothing here is first-hand testing of the model itself.

Where I got stuck

Where I got stuck is literal: console.typesafe.ai bounced me to /login. The more interesting wall sits behind it. Jev's product is a promise — that its confidence is calibrated, that a 0.9 answer is right about 90% of the time — and the only sanctioned way to check that is on your own labeled data, with a key most people don't have. What the evidence does support is the speed and the price. TypeSafe claims 70–500ms end-to-end and advertises a best case of '193.6x faster, 444.6x cheaper.' Every's test survived contact: 777 judgments across 37 documents in under 0.7 seconds for roughly a quarter of a cent; 1,709 judgments across 11 experiments for under a cent. In Shipper's sharper test — four writing checks over twelve synthetic passages — Jev ran a 0.35-second median against Claude Fable 5.1's 8.83 seconds at roughly 1/580th the cost, and caught six of seven planted defects. The one it missed, in all three runs, was the defect that needed a beat of reasoning. TypeSafe's own evals dashboard makes the same concession: Jev aggregates 67.8% against the best comparator's 74.1%, on reference labels that are averages of GPT-6 Astra and Fable 5.1 outputs — agreement with the frontier, not ground truth. The Discord fills in the texture. A prompt-injection corpus: 96.5% over 662 messages, no tuning, 325ms median. A local-events validator reporting 5.7x faster, 98% cheaper and twelve points more accurate than tuned small models. Six million tokens of word-game judging for $0.14 — alongside real limits: weak on idioms, blind to images, no code, a 32K state, and nothing that needs a plan. One tester fed it 2,900 trading days for under a cent and found no predictive signal at all. As for 'can't hallucinate': the claim is structural. Every output is a value you declared in advance, so a malformed response is mathematically impossible — an entire class of JSON-parsing failures, gone. It says nothing about the other class: Jev can be confidently wrong, which Almeida himself conceded on HN. 'Type safety is not factual correctness,' as the thread's sharpest reply put it. What I could not verify: accuracy beyond Every's small sample; calibration on messy, genuinely ambiguous input — the number this product actually stands on; the architecture, which is unpublished; and the moat. Within hours of launch, two Hugging Face projects had rebuilt the interface on stock Qwen models — inference-layer tricks, not trained reproductions — which supports Sean Goedecke's argument that most of the trick is 'prefill, then sample one token.'

One thing I'd change

Make the one claim everything rides on checkable without permission. TypeSafe's own documentation rightly tells users to bucket their labeled examples by confidence — advice that currently requires a key. Publishing the workflow harness and its reference labels, or a no-login playground with canned queries, would let skeptics try to falsify 'calibrated' before the waitlist clears. A launch that asks for trust should not gate the audit.

Would I pay

Maybe. The job I'd hire it for is real: a calibrated judge consulted on every turn of an agent loop, priced so low the cost line effectively disappears — and the speed and price have survived third-party contact. What hasn't is the confidence number itself; nobody has published calibration on messy data, which is exactly where automation breaks. If my own labeled set confirms the 0.9 bucket is right nine times out of ten, Jev becomes the cheapest line item in the stack. Until then the waitlist isn't just keeping me out — it's keeping the claim unverified.

■

Roger HE is building PersonWise — The presentation presents itself.

Respond to this review →