01
No made-up score
The “ATS score” became 13 readability checks. Each one shows the lines it is about and a fix, so a user can see why, not just how much.
Cut: the single score
← All work
AI product · rebuild of my own v1
Shortlist
Show your real experience, clearly.
A resume assistant that judges a resume against a job ad requirement by requirement, quotes the resume as evidence for every verdict, and suggests rewrites that are checked for invented facts.
- Role
- Product direction, UX and interface, design system, evaluation design
- Status
- Live working prototype · evaluations run in CI
- Stack
- Python, FastAPI, Gemini, hybrid RAG, Next.js, TypeScript
1The problem
Resume tools hand out a mysterious score and rewrite bullets with numbers you never achieved.
2My role
Product direction, UX and UI, design system and evaluation design. Built with Claude Code under my direction.
3Outcome
Live. Matching accuracy went from 0.22 to 0.91 on 74 labelled requirements, with no missing requirement marked as met.
01
The problem
Resume tools hand out a mysterious “ATS score” and rewrite bullets with numbers you never achieved.
My own first version was guilty of both: its score gave 10 points to any resume, and its job matching was 37 hard-coded substrings, so “ai” matched inside “maintain”.
So the rebuild started from one question: what would a resume tool look like if every judgement had to show its evidence?
Who it's for
Job seekers tailoring one resume to a specific job ad, who want to know what a recruiter will actually find, not a number.
02
Research
Status, honestly
Desk research and my own v1 audit. No usability testing yet; a study plan is written in the case study.
- Audited my own first version line by line: what it claimed, what it actually computed
- Six synthetic resumes in different formats (one two-column PDF, one Word file) and four job ads as test material
- 74 hand-labelled requirement judgements to measure matching against
03 · User flow
From a file to a tailored resume
Step 1 / 6
04
Key decisions
02
The model judges, code scores
The AI decides met, partly or not shown for each requirement, with quotes. Coverage is computed in code: required items count double, partial evidence half.
03
Quotes or it didn't happen
Every verdict quotes the resume, and each quote is checked against the file. A “met” whose quote can't be found is downgraded to “partial”, in front of the user.
04
Rewrites that keep to the facts
Suggestions are flagged beside the text when they add a number the original doesn't contain or a skill that appears nowhere on the resume.
05
Deterministic first
Parsing, layout facts, checks and the keyword view are instant, free and tested, and still work with the AI switched off. The model is only called where judgement is needed.
05
How it evolved
Three looks before it felt like its own product
Real screens from the project's history, in order. Every stage was live.
1 / 3 · First interfaceThe rebuilt product's first screens: an editorial serif look, with the match report beside the resume. It worked, but it looked almost identical to Margin.
06
Final interface
Real screens from the live product.
07
Technical architecture
Deterministic first, one AI call where judgement is needed
On upload · no AI
On match · one AI call
On upload · no AI · step 1
PDF / DOCX
File type detected from bytes; Word and PDF supported.
- The model judges, code scores
- Guardrails on rewrites: new numbers and unlisted skills are flagged
- Shared, tested core with Margin: ingestion, retrieval, quote verification
08
AI implementation
How the AI is kept honest
01
Structured outputs
Gemini returns typed JSON per requirement: verdict, quotes, reasoning. Prompts are versioned.
02
Grounding
Quotes must exist in the resume. Unverifiable evidence is downgraded in front of the user.
03
Guardrails on rewrites
Numbers not in the original and skills that appear nowhere on the resume are flagged beside each suggestion.
04
Adversarial tests
A fake model that misbehaves on purpose: invented evidence must be downgraded, inflated rewrites must be flagged.
09
Before & after
v1 against v2, on the same labelled set
Requirement accuracy (3-class)0.905from 0.216
Evidence F10.965from 0.174
Can assess the requirement1from 0.5
Section F1 (parsing)1from 0.732
From the README's evaluation tables. Six synthetic resumes, four job ads, 74 labels written by me: the change on known cases, not accuracy on unseen resumes.
What v1 actually did
v1v2
An “ATS score” that gave 10 points to any resume13 explainable checks with the lines and a fix
37 hard-coded substrings (“ai” matched “maintain”)Requirements judged with quoted, verified evidence
One global resume and chat shared by every visitorPrivate workspaces, exportable and deletable
Parser hard-coded to the author's own job titlesRules tested on six resume formats, 6/6 parsed
10
Challenges
Free hosting sleeps
The backend sleeps after idle minutes, so the first request can take up to a minute. The app shows a “waking the server” notice instead of failing silently.
Evaluating honestly
The test set was written by the same person who built v2, so the README says exactly that next to the numbers.
Resumes aren't text
Two-column PDFs and Word files read out of order; parsing had to be column-aware before any AI could help.
11
Outcomes
0.22 → 0.91
Requirement-matching accuracy, v1 to v2 with AI
0.97
Evidence F1: the right resume lines quoted
0
Missing requirements marked as met
6 / 6
Resume files parsed, including a two-column PDF and Word
Measured on six synthetic resumes and four job ads that I labelled myself while building v2. They show the change from v1 on known cases, not accuracy on unseen resumes.
12
What's next
- →Real, consented resumes labelled by someone else
- →A usability study (the plan is in the case study)
- →OCR for image-only resumes
Built with Claude Code under my direction: I set the product direction and requirements, and reviewed the design, code and evaluations.
Next project
Margin →
A document assistant you can check.