Systems with a model inside — and the checks that keep it honest.

I build software where a language model is one component, not the whole product. That means the part an AI-training role is actually about: deciding whether an output is good, and being able to say why. Every screenshot on this page was taken by running the software — the apps were launched, the questions were typed, the agent really browsed python.org, and the reports came out of the generator. Every number is measured.

7projects
43klines, excl. dependencies
4languages in production use
323automated tests
01Rust · TypeScript · local LLM

Compass

A local-first assistant that answers only from your own memory — and cannot cite something you can't open.

Rust workspaceTauri 2Svelte 5 + TSSQLCiphersqlite-vecOllama / Groq14.5k lines235 tests

Writing a journal entry returns in 0 ms — the text hits disk first. Chunking, embedding, entity extraction and the graph commit all happen in the background, so a capture never depends on the model succeeding. If extraction fails the entry is still there and still re-ingestible. The same person mentioned twice resolves to one node; two mornings' runs stay two events.

The Compass journal tab, showing an entry box and a list of extracted memories with types, domains and confidence
live The Journal tab. Below the entry box is what Compass has extracted so far — events, entities and insights, each with a confidence and the entry behind it. “Keep on this device” is not decoration: it is a property that travels with the memory for the rest of its life.

Then you ask a question in plain language. Read the next screenshot carefully, because it is the whole thesis.

A Compass answer with citation marker 1, a Why panel expanded showing the source insight and the original journal entry
live The question was typed and answered by local llama3 in 10.8 s. The claim carries the marker [1]; “1 of 8 memories” says how much of the retrieved context was used; the provenance line names the model and where it ran; and opening Why? shows the memory, why it was surfaced, and the original entry from 8/19/2026 underneath it.

There is no step in that chain where you have to take the model's word for anything. What stops an ungrounded answer is not a line in the prompt — it is three properties of the code path:

No retrieval, no generation.An empty context pack means the model is never called at all. A question with no supporting memory returns nothing, not a plausible paragraph.
Citations are resolved, not trusted.Every marker is looked up against the pack it was built from. One that cannot be opened is stripped before the answer is shown.
Privacy travels with the context.A single local-only memory in the pack pins the entire turn on-device — the cloud provider is not deprioritised, it is unavailable.
Three green self-check results: LLM gateway answered by ollama in 886ms, encrypted store round-tripped under SQLCipher AES-256, embedding model separation 0.774 versus 0.180
live All three self-checks, clicked while these screenshots were being taken. The app proves itself on demand rather than in a changelog.
LLM gateway — one real completion, routed local-first886 ms · 21 tok
Encrypted store — write, read, delete round-tripSQLCipher AES-256
On-device MiniLM — similar pair vs unrelated pair0.774 / 0.180
Hybrid retrieval via sqlite-vec vs brute-force scan17.9× @ 100% recall
One journal entry → memories and connections5 + 3 in 3.3 s

Four crates sit behind a typed IPC boundary: the domain model with typed ids and secrets that wipe on drop, a provider gateway with local-first routing and OS-keychain storage, the encrypted store, and the engine. The project was migrated from Flutter to Tauri 2 against a written go/no-go gate — cleared only once the app built, installed and launched on Android with a working IPC boundary — and the Dart tree was then deleted rather than left to rot.

Why it matters here

Deciding whether an answer is actually supported by its sources is the core skill in RAG evaluation. This is that judgement compiled into an architecture, plus the habit of producing a number instead of an adjective.

02Python · JavaScript

AutoApply

A job-application workspace where the LLM is not allowed to make anything up.

FastAPIPostgreSQL + Alembicbge-small, localGroqVanilla ES modules11.2k lines88 tests
The Discover view listing ranked job matches with percentage match bars and Tailor and Track buttons
live The running app against a live Postgres, seeded with a fictional profile. Jobs are embedded locally with bge-small — no API key, no per-job cost — and the percentages are that cosine similarity. They behave: a platform backend role scores 81%, a frontend-only role a clearly lower 72%.

The interesting part is what happens after Tailor. The model is asked to select, reorder and rephrase the user's real bullets for a target job — never to invent experience — and every tailored bullet must cite the source_bullet_id it came from. The output is then verified against the master profile before it can be attached to an application:

Every bullet must reference a real row.An experience or bullet the profile doesn't contain fails the document.
Skills must be a subset.The model cannot add a technology to make the match look better.
Every number must be traceable to its source bullet.The guard against a quietly invented “improved throughput by 40%”.

A document that fails is still stored so it can be inspected and edited — but flagged factual_check_passed = false with notes naming the check that failed.

Documents list showing a tailored resume and cover letter, each with a green fact-checked badge and a scrubbed tag
live Real Groq output that passed the gate. The badges are earned, not printed.
"experiences": [{ "company": "Northwind Logistics",
  "bullets": ["Reduced p99 dispatch latency from 1.8 s to 240 ms by swapping a
               synchronous fan-out for an async, queue-backed worker pool.", …] }],
"ats_keywords_covered": ["Python","FastAPI","PostgreSQL","Kafka","async","queue", …],
"factual_check_passed": true,
"factual_check_notes": "Verified against master profile: all bullets, skills and
                        numbers traceable.",
"metadata_scrubbed": true

Every number in that output traces back to a bullet the user actually wrote. The rephrasing is the model's; the facts are not.

A Kanban application board with columns for discovered, applied, screening, interviewing and offered
live The application board — seven statuses, drag between columns, full status history behind each card. The entire client is vanilla ES modules with no build step: no bundler, no framework, served by the API itself, installable as a PWA with an offline shell.

Behind it: 82 Python modules, 20 service modules covering discovery across sources, matching, tailoring, PDF/DOCX rendering, an IMAP sync that parses recruiter replies, a scheduler and notifications — and 88 tests across 13 modules. When a key is unset the endpoint returns 503 and the interface says so in words, rather than failing silently.

Why it matters here

Two things at once: idiomatic Python and dependency-free browser JavaScript in one codebase, and a fabrication detector for model output built from citation requirements and numeric traceability — the same task as judging groundedness, expressed as code.

03Python · speech · rendering

Vani-Drishti

Recite a Sanskrit or Awadhi verse; get a video where each word lights up as you say it.

faster-whisperPyQt6 desktopFastAPI webPillow + ffmpeg5.9k linesfully local path

Two front ends over one engine. The corpus parser, aligner and renderer are a plain Python package; a desktop app and a web app both sit on top, so the work that matters is written once.

The Vani-Drishti desktop application with corpus controls on the left and a rendered preview frame of a Devanagari verse on the right
live The desktop app, launched. Corpus loaded: 1,935 verses. The right pane is a real preview frame with the word हरनी highlighted at that timestamp.
The Vani-Drishti web interface with a scripture picker, verse list and a parchment verse panel
live The web build, running locally. It adds provider choice — speech-to-text through Gemini or Groq, or a local GPU chant engine — behind one interface.

The hard part is alignment. ASR output for chanted Sanskrit does not match the written corpus token for token: sandhi joins words, elision drops them, and chant-style vowel lengthening moves every boundary. A naive “display what Whisper heard” approach produces a screen full of subtly wrong verse. So the transcript supplies timing only, the corpus supplies the text, and fuzzy alignment joins the two. The screen always shows the correct verse, and the highlight still lands on the right word.

Rendered 1920x1080 frame on a parchment background with one Devanagari word highlighted
output Parchment Classic.
The same renderer in a dark Temple Stone theme
output Temple Stone Dark — same pipeline, themes are data rather than branching code.

Frames stream to ffmpeg one at a time so memory stays flat however long the recitation runs. Devanagari typesetting is handled explicitly — the app manages its own Noto Serif Devanagari and lays out conjunct-heavy lines without clipping, which is where most naive text rendering falls over.

Why it matters here

Assessing recognition in a non-English, non-Latin script means deciding what “wrong” even means before you can score it — a transcript that differs from the corpus may be a correct hearing of an alternate sandhi rather than an error.

04Python · vision agent

Visual browser agent

A vision-language model that drives a real browser by looking at it.

Set-of-marks promptingPlaywrightPydantic action schemasubprocess sandboxGemini / Groq
The python.org homepage with every interactive element outlined in red and labelled with a numbered badge
live Exactly what the model is shown. Asking a model for pixel coordinates is asking it to do the one thing it is worst at; asking for a badge number is asking for something it is good at. Note badge 13, then read step 1 below.
Terminal output of a four-step agent run ending in the correct answer 3.14.7
live One task, four steps, verbatim terminal output from the same run.
StepWhat happened
125 elements detected. The model reasons that Downloads is where the answer lives and picks element 13 — the badge in the screenshot above.
2It sees “Python 3.14.7” on the downloads page and declines to conclude, scrolling to the Active Releases table to check whether something newer is listed.
3It decides a table is a text problem rather than a vision problem and switches tools to GET_PAGE_TEXT. The runtime also downscales the image to a 512px single patch to save tokens.
4It reads the table, explicitly rules out 3.15 as a pre-release, and answers 3.14.7 — correctly.

Step 2 is the interesting one. A weaker loop answers from the first thing that looks like a version number.

The response is a Pydantic model, so the schema is the contract: a required thought field first, then one of thirteen actions — click, type, scroll, navigate, hover, drag-and-drop, keypress, select, download, find-in-page, get-page-text, execute-code, done — each field documented for the model to read, including the warning that generated code must print its output or the agent will never see it.

The parts that stop bad trajectories: code runs in an isolated subprocess with a timeout, so a hanging snippet costs one step rather than the run; the screenshot is skipped on the turn after code execution because the page hasn't changed; and a consecutive-failure counter aborts after three unparseable responses instead of spinning against a rate limit.

Why it matters here

Agent-trajectory evaluation — reading a chain of thought → action → observation and judging whether each step was reasonable — is a large share of current AI-training work. This is that loop from the inside, including the failure modes that produce the bad trajectories you get asked to grade.

05JavaScript

curbcut

The scan is the commodity. The translation into something a client can act on is the product.

Node 20 ESMPlaywrightaxe-corePDF generation3.7k lines
Cover page of a generated WCAG audit report
output Page 1 of a real report for developer.mozilla.org, branded from a demo brand.json.
Executive summary page showing a score of 77 grade C and a severity breakdown
output Score weighted by severity and prevalence, so “fix these first” is ordered by real risk rather than raw count.
Detailed findings page showing who an issue affects, why it matters, how to fix it, estimated effort, and where it occurs
output Every finding rendered from a hand-written knowledge layer: who it hurts, why it costs money, how to fix it, and minutes of effort per occurrence. The file's own docstring sets the bar — “every entry is written to be read aloud in a client meeting without embarrassment.”

Three modules exist because the naive scan is silently wrong, which is worse than incomplete — an audit that looks fine and is worthless is the worst possible failure mode for a document handed to a paying client.

consent.jsWhen a cookie overlay covers the page, axe measures the overlay and reports a nearly clean site underneath it. So the banner is audited first — a consent wall nobody can operate by keyboard is itself a failure — then dismissed, then the real page is measured.
viewport.jsA desktop-only scan misses everything that breaks at 320px and not at 1440px. Target size (2.5.8) is deliberately not checked: it is WCAG 2.2, and reporting against a standard the document never claimed to test would undermine every other number in it.
keyboard.jsThe part axe cannot do, because keyboard access is behaviour rather than DOM. Tuned to miss real faults rather than assert false ones: a finding an agency cannot reproduce in front of their client destroys the credibility of the whole report.

The report also states its own limits in print: automated testing reliably detects roughly a third of WCAG success criteria, cannot judge whether alt text is accurate, and is not a certification.

Why it matters here

Rubric writing and calibrated explanation is most of the job — and that deliberate bias toward missing a finding over asserting a false one is exactly the instinct a good annotation guideline encodes.

06TypeScript · Claude

Proofpage

Paste messy project notes; get a case-study page you can send to a client.

Cloudflare WorkersD1 + R2Anthropic APIno frameworkdeployed
The Proofpage intake page asking for messy notes and screenshots
production Captured from the deployed Worker at proofpage.ujjawalkumar.dev. “The messier the better” is the whole product promise — freelancers do not have tidy notes, and demanding them is why every competing tool goes unused.
notes + screenshots
   └─► INTAKE  (claude-haiku-4-5)  → title, client guess, ≤4 gap-filling questions
        └─► answers
             └─► CASE STUDY (claude-opus-5) → structured study → D1 + R2 → public page

Two models, chosen per job, each labelled in one line:

/** Model for the cheap, high-volume intake pass. */
export const INTAKE_MODEL = "claude-haiku-4-5";

/** Model for the thing the freelancer puts their name on. Do not downgrade. */
export const CASE_STUDY_MODEL = "claude-opus-5";

The intake prompt is the product. It encodes a theory rather than a persona —

“Freelancers write down what they built; prospects want to know what changed.”

— and then constrains the ways the model would go wrong. Ask at most four questions, because every extra one is a chance for the freelancer to abandon the form. Never ask about something already answered in the notes or in an attached screenshot: “if a chart shows the completion rate going 41.2% to 63.8%, the completion rate is answered, and asking for it anyway tells the freelancer you did not look at what they gave you.” Ask for specifics, not essays. Give every question a one-line “why” so the field doesn't get a shrug. And, the rule most prompts are missing — if the notes are genuinely complete, return an empty list; do not invent questions to fill space.

Caching is treated as an economic decision. Every prompt is a frozen constant, because caching is a prefix match and one byte of drift before the breakpoint invalidates the entry — “the difference between ~87% and ~40% gross margin, and expensive to retrofit.” Cache minimums are documented as non-monotonic across model generations (512 tokens for Opus, 4,096 for Haiku) and then deliberately not gamed: padding the intake prompt purely to earn a cache hit would cost more than it saves. Images go before text in the user turn, both because that is the documented ordering and because it keeps the varying text where caching wants it. Token usage is logged per generation, so margin per study stays visible instead of arriving on a bill.

Failure is typed. A ConfigError — bad key, dead model id — is loud in the log and never invites a retry; a GenerationError is written for the user to read. Without that split, “a 401 reads as ‘try again in a moment’, which is how you end up retrying a misconfigured API key forever.”

That behaviour is not theoretical: it fired while this page was being written. The Anthropic account is out of credit, so a live generation returned a 400 — and the Worker classified it as a deployment fault, logged CONFIG ERROR — needs a fix, not a retry, and declined to suggest trying again. That is why there is no generated-page screenshot here; the error path did exactly what it was built to do.

Why it matters here

Model selection justified per call site, a prompt that constrains behaviour rather than describing a character, an explicit licence to return nothing, and an error taxonomy that separates “the model failed” from “we are misconfigured.” Those are the same distinctions that separate a useful model-behaviour report from a useless one.

07Rust · privacy design

lifelg

A local-first search engine for your own screen — and a design document that argues with its own spec.

Rust daemonTauriAT-SPI2SQLCipherworking spike
Terminal output showing an AWS key and card number redacted, a denylisted window dropped, and the panic pause engaging
live The spike's capture-time controls, executed on synthetic input. An AWS key and a card number are redacted in place; a window matching the denylist is dropped entirely — before embedding, before storage, “as if it never happened”.

Note what that output also shows: the 2FA code was not caught. The denylist and pattern set are a starting point, not a finished secret detector, and the project's own notes say so. Naming what a control does not yet cover is the difference between a security story and a security claim.

forget is the other half — it VACUUMs after deleting, because rows removed from SQLite stay on disk and forensically recoverable otherwise. That detail is what separates a real erase button from a checkbox.

Behind the CLI is a design document that takes the original specification and states where it is wrong:

The spec saidThe correction
OCR everythingOCR last. The cheapest and most accurate text comes from the accessibility trees apps already expose — an order of magnitude less CPU, and better search.
Find “the green and blue chart”Text search cannot. Visual recall needs multimodal image embeddings; the spec promised a feature its own architecture could not deliver.
Vector search over everythingOver millions of tiny fragments it returns garbage. “Last month”, “Monday”, “Mark said” are filters, not semantics.
SQLCipher secures the dataThe vector index is a plaintext hole — embeddings can be inverted back toward their source text.
Storage is the riskCapture is the risk: other people's audio, passwords, 2FA codes. Redact at capture time, show the recording state, ship a panic pause.
Negligible CPUNot while running OCR, Whisper and embeddings continuously. Event-driven capture, heavy work deferred to idle windows, on the NPU/GPU.

The query planner is rule-based on purpose. The schema was designed first, so swapping the heuristic parser for a small local model later is a drop-in replacement for one function body — callers only ever see a query plan, never how it was produced. Its known limitations are listed in the file rather than discovered afterwards. The ask command already goes further: retrieve evidence, then synthesize a cited answer through a local model on Ollama, with a deterministic fallback when Ollama isn't reachable.

Why it matters here

Knowing when not to use a model, being able to write down why — and reporting the case your method missed instead of only the ones it caught.

Skills matrix

AreaEvidence on this page
PythonFastAPI, SQLAlchemy + Alembic, pytest (88 tests), asyncio, Pydantic schemas, PyQt6, Pillow, Playwright, faster-whisper
JavaScript / TypeScriptNode 20 ESM CLI, Cloudflare Workers with D1 and R2, Svelte 5, and a dependency-free ES-module PWA with a service worker — no framework, no build step
RustTwo Tauri 2 applications; typed IPC, SQLCipher, sqlite-vec, trait-based provider ports, AT-SPI2 capture, 235 tests under clippy -D warnings
LLM engineeringCitation-bound generation, source-id traceability gates, two-model routing, prompt caching as a margin decision, typed agent action spaces, local inference via Ollama, provider failover, typed error taxonomies
EvaluationCosine-separation checks, recall and latency measurement, factual verification against a source of truth, and honest reporting of what a method misses
Data & storagePostgreSQL, pgvector, SQLite, SQLCipher, sqlite-vec, D1, R2, migrations, real foreign keys
AlsoKotlin / Android, WCAG 2.1 AA, Devanagari and IAST text processing, Docker Compose, CI across Linux, macOS and Windows

Also in the workshop

EZRIDE

A voice-first driving codriver for Android Auto, 6.3k lines of Kotlin. Design thesis: the LLM is not in the realtime loop — deterministic guidance drives speech, the model handles questions and phrasing off the critical path.

uniqtrader

An LLM review panel deliberately A/B-tested against a deterministic checklist implementing the same protocol over identical inputs, because “does the model help” is not the same question as “does it beat the checklist you'd have written instead”.

ORBIT

Career decisions turned into falsifiable predictions — Next.js, Prisma, and schema-constrained generation against the Anthropic API, with the check-back date required up front.

phychess

Play chess.com by physically picking up pieces: MediaPipe hand tracking over a phone camera feed, pinch to grab, board calibration, synthetic input to execute the move.

Contact

Open to AI-training, AI-tutoring and applied-LLM roles. The fastest way to reach me is email — I read and reply from it directly.

Colophon

Every image on this page is a real artifact. The Compass, AutoApply and Vani-Drishti screenshots are the applications running — launched, driven, and captured. The agent trace and annotated view are one real run against python.org, which it got right. The curbcut pages come from a PDF the tool generated. The Proofpage shot is production. Nothing here is a mockup, and where a screenshot could not be produced the reason is stated rather than papered over.

Where a screenshot needed data, the data is fictional: AutoApply is populated with a demo profile so no personal information appears. Line counts exclude dependencies, virtual environments and build output. Six of these ship end to end; lifelg is a working spike whose design document is deliberately ahead of its code.

Two repositories in the same folder — a 3D chess game and a Sanskrit TTS system — are clones of other people's work and are deliberately excluded.

github.com/ujjwal7789 · ukumar951.gm@gmail.com