LLMs are amazing at writing software. Deterministic software.
But actually using LLMs within software.. well, that still sucks.
Which is too bad, because LLMs are great at making sense of unstructured data. They can make judgment calls like “read this customer complaint and figure out what to do about it.”
That use case is so freaking valuable. Just ask Sierra, with its $15B valuation and long list of well-known customers.
Why do LLM calls embedded within deterministic software suck? It’s not just that it’s slow and expensive. which is true...
The big concern is the type signature. An LLM call is a function typed String -> String. If you’ve ever worked with a statically typed language, this is like the least informative signature in software. For example, if the LLM is expected to return a number, you’d rather the output signature be Integer.
When the function just says “I’ll take anything and will return anything”… lol. It’s very difficult for the caller to reason about it, so they end up having to write a runtime validator to parse the output and check the schema.
Sure, you can use Structured Outputs, which makes sure model-generated outputs match JSON schemas provided by the developer. That’s an improvement. Before, asking “Is this customer asking for a refund? Reply yes or no” to the LLM might give back “Yes, it looks like they want a refund, although...” And your code then has to parse that string and hope to extract the correct “yes”.
But with structured outputs, you send a schema like
{ “refund_requested”: “boolean”, “category”: “billing | technical | account” }and the reply is guaranteed to be valid JSON in that shape, e.g.
{”refund_requested”: true, “category”: “billing”}This is obviously much better for the developer.
Yet under the hood it works through constrained decoding, which is still slow. The model still generates the answer token by token, and at each step the server blocks any token that would break the schema. So if the next valid character has to be t or f for a boolean, every other token gets zeroed out. This works, but it’s still slow and expensive...
There’s also no confidence signal. The input might be a long, rambling paragraph from a customer. How confident is the LLM that the customer actually asked for a refund? Shouldn’t the developer act differently if the confidence is 95% versus 52%?
Yes, you can ask the LLM to include a confidence field in its response, but that number is just another generated token.
Then there’s hallucination. What if the answer that comes back was made up? It looks fine to the engineer, and it may even parse cleanly. But it’s not real! A trustworthy confidence level would help here too.
So what do we need to call an LLM over and over, deep in the bowels of a program, with confidence? A fixed set of possible answers, a guaranteed response time, and an honest confidence attached to every answer.
In other words, intelligence with a function signature.
With Jev from Typesafe AI, we get just that. A frontier-intelligence function call. Unstructured state in, typed probabilistic decisions out. AND FAST:
Same 27 questions, one request each, started at the same time. Jev returns all 27 typed answers with probabilities in 0.114 seconds for $0.000081. GPT-5.6 Terra is still generating its answer token by token. Source: TypeSafe.
With Jev, you send the state and a set of questions, each with a type. A Noul is a yes/no question, and it returns the probability the answer is yes. A Choice picks one option from a list you define. A Score places the input on a scale you define, like calm to very angry:a
respon se = client.system_one(
state={
"ticket": "My flight was cancelled. Can I get a refund?",
"refund_policy": "Cancelled flights are eligible for a full refund.",
},
questions={
"refund_requested": Noul(instructions="Does the customer request a refund?"),
"request_type": Choice(
instructions="What is the main request?",
criteria={"refund": "...", "rebooking": "...", "information": "..."},
),
"frustration": Score(
instructions="How frustrated does the customer appear?",
criteria=["Calm and neutral", "Concerned but civil", "Very angry"],
),
},
)
What comes back from the model is typed values. refund_requested comes back as a probability, say 0.97. request_type comes back as “refund”, along with a probability for every option and a confidence score. frustration comes back as a position on your scale, say 0.4.
Then your code decides what to do. You focus on the business logic:
a = response.answers
if a["refund_requested"].noul > 0.9 and a["request_type"].choice == "refund":
issue_refund(ticket_id)
elif a["frustration"].score > 1.5:
route_to_human(ticket_id)Dude! So simple!
TypeSafe calls it a “smart if-statement”, and in their manifesto they say
“Computers can do so much by just branching on bits, imagine if they could also branch on common sense, understanding, and intent.”
Jev also can’t hallucinate an answer, because it can only answer with the options you gave it! Jev can still be wrong, of course. It can pick rebooking when the customer wanted a refund. But it comes with a confidence signal too, which should help clue you in.
TypeSafe trains Jev with a new post-training method it calls RLCD, reinforcement learning for calibrated decisions. RLHF trained models to write answers people prefer. RLVR trained models to get checkable answers right, like math. RLCD trains the model’s probabilities to be honest.
Calibrated means that when Jev says 0.8, it’s right about 80% of the time. That’s across many answers, not a promise about any one of them.
Calibrated probabilities are something you can program against.
And Jev is fast.
Why?
It answers every question in a single pass through the model. An LLM writes its answer one token at a time, and each token waits on the one before it. Jev has no token-by-token loop! That’s why output tokens are free btw.
What is Jev good for? Quick judgment calls. Classifying, routing, scoring, extracting, verifying, and ranking.
Just think of Jev as a tool for developers. Soo… for agents? :)
It’s meant to put fast and cheap intelligence into software.
And it has implications for AI infrastructure!
Let’s dig into some questions you might have:
Decode is the reason AI accelerators have HBM. What does a model with no decode phase need instead?
Is this a slice of today’s LLM inference market, or a new one?
What happens to fast decode solutions like Groq and Cerebras?
Who wins?
Can a humanoid or a car run frontier-class judgment on an edge SoC?
What about a phone?
Let’s get into it.