We Tested Jev on 277 Real Public Tenders. Here Is What a Decision Model Can and Cannot See.

The short version:

• ✓Jev, the new decision model from TypeSafe AI, read 277 real Romanian public tender notices.
• ✓It flagged all 5 notices that were genuinely our kind of work.
• ✓It agreed with our own labels 93% of the time.
• ✓It answered in about 0.6 seconds per notice, and the whole test cost less than one cent.
• ✓It is a great first filter, and a poor decision maker. The reason why is the most useful part of this post.

The problem: 98% of the feed is noise

We are a small software company in Romania. Every day, the national procurement platform (SEAP) publishes small direct-award notices. A few of them are work we can actually deliver: a custom platform, a web portal, some IT consulting, an automation project.

Most of them are not. When we sorted 210 fresh notices from the IT-related codes, this is what we found:

What the notice buys	Notices
Hardware (laptops, toner, network gear, consumables)	77
On-site or physical work (repairs, maintenance, guarding)	71
Other services (audit, events, studies, telecom)	29
Software licenses and subscriptions	20
Support for one vendor's existing system	4
Unclear from the title	4
Genuinely our kind of work	5

Five out of 210. Reading all of them every day is exactly the kind of job a fast, cheap model should take off a person's plate.

What Jev is, in one paragraph

Jev does not write text. You give it a situation and a question with your own answer options, and it picks one, with probabilities. Its makers call it a "System One" model: quick, intuitive judgment, the kind a person makes in a second, rather than the slow step-by-step reasoning of a chatbot. That makes it a natural fit for sorting, and a questionable fit for anything that needs real reasoning.

How we tested it

We built a test set of 277 real notices, all read live from SEAP's public API:

• ✓67 we had already rejected, each for a recorded reason.
• ✓210 fresh ones, which I labeled by title into the categories in the table above.

For each notice, Jev got the title, the estimated value and the buyer, plus two questions:

1. ✓Which category does this notice fall into? (a choice between the six options)
2. ✓Would a small remote software company realistically bid on it? (a yes/no answered with a probability)

Two honest caveats: the reference labels are ours, made from titles only (which is also all Jev saw), and this was a single run on a single day.

Results at a glance

Measure	Result
Real opportunities caught	5 of 5
Agreement with our labels (clear cases)	192 of 206 (93%)
"Would we bid?" score, real opportunities	0.31 to 0.57
"Would we bid?" score, hardware and on-site work	median 0.03, never above 0.25
Median response time	0.6 seconds
Cost of the whole run	about $0.007

The yes/no score is the most useful number. A threshold of 0.3 separates the real opportunities from the noise.

One real example

Here are two notices from the same day, as Jev scored them:

• ✓A city hall buying software modules for collecting local taxes: score 0.37, category "in our lane". It stays at the top of the list.
• ✓An army unit buying SSDs, memory and solder wire: score 0.07, category "hardware". It drops into the "low score" group at the bottom.

Both calls are right, as far as a title can tell you. Which brings us to the interesting part.

What it cannot see

Jev called 21 of our 67 rejected notices "in our lane". That looks like a failure. It is not.

By title, those notices really are our kind of work: web services, a learning platform, software development. We rejected them because of details that only appear in the attached tender documents:

• ✓ten years of experience with one specific product,
• ✓a maintenance obligation running years past the build,
• ✓a requirement to plug into a system only its current vendor controls.

The city hall notice above is one of them. The title says "software modules". The documents say the modules must work inside the existing tax application, which only its current vendor can realistically touch.

No model reading a title can see that. Neither can a keyword filter, or a person skimming the feed. The fix is not a better model. It is a rule: the attachment is always read before anyone decides.

How we use it now

We connected Jev to our daily tender alert, with three rules:

1. ✓It never removes a notice. Each one gets a score, and anything under 0.3 moves to a "low score" group at the bottom, still listed in full.
2. ✓It fails open. If the call fails for any reason, the alert goes out exactly as before, with one line saying the score could not be checked. A failed score is never shown as a low score.
3. ✓It only sees public data. Public notice text, nothing else. No client email, no personal data.

The decision stays with a person who has read the documents. Jev just makes sure that person spends their attention on the few notices worth reading.

Try it on your own data

The request is small. This is the shape we used, with our own categories:

{
  "state": "Public tender notice. Object: Software modules for collecting local taxes. Estimated value: 65,289 lei. Buyer: a city hall.",
  "model": "jev-latest",
  "questions": {
    "lane": {
      "type": "choice",
      "instructions": "Which category best describes what this notice buys",
      "criteria": {
        "in_lane": "Software, web or data work a small remote software company can deliver",
        "hardware_resale": "Physical goods: computers, printers, network equipment, consumables",
        "onsite_or_physical": "On-site work: repairs, maintenance, guarding"
      }
    },
    "bid": {
      "type": "noul",
      "instructions": "A small remote software company would realistically bid on this notice"
    }
  }
}

Our advice: collect a few hundred real examples with known answers, run them, and look first at what it misses, not at its accuracy. For triage, a missed item is expensive and a false alarm is cheap.

Should you use it?

Yes, if your problem is triage: a large stream of text where most items are clearly irrelevant and the few relevant ones must not be missed.

No, if your decision depends on details it cannot see. Put the model in front of the reading, not instead of it.

Frequently Asked Questions

What is Jev? Jev is a decision model from TypeSafe AI. Instead of generating text, it answers structured questions (a choice between your options, a score, or a yes/no probability) and returns calibrated probabilities that software can act on directly.

How accurate was Jev in this test? On 206 clearly labeled notices it agreed with our labels 93% of the time, and it flagged all 5 genuine opportunities. Most disagreements were other kinds of IT services labeled as ours. The reference labels were our own, made from titles only.

How much did the test cost? About 176,000 input tokens for 277 notices, roughly $0.007 at TypeSafe's published price of $0.042 per million input tokens. Output tokens are not charged.

Can a decision model replace reading the tender documents? No. A third of the notices we had rejected looked right by title and were disqualified by details in the attached documents. A model that only sees the title cannot know that, so it works as a filter before the reading, not as a replacement for it.

Is it safe to send business data to a new AI provider? Read the provider's data terms first. In our test we sent only public procurement notices, so no client or personal data left our systems.