Measured notes
Blog
Things we measured or built while working on agent skills and x402 payments. Every post says what we did not measure.
- Notes from building
Refactor without changing behavior: null and empty values
A tidy rewrite often changes what null, zero or an empty string turns into, and tests that call the function normally do not notice. How to catch it.
Notes from buildingLink previews: no card, wrong image, blank icon
A pasted link with no card, a template's stock picture or a blank tab icon: where the tags must live for scrapers, and how to prove the picture exists.
Notes from buildingSecurity headers that do not break the page
HSTS, framing, nosniff and a Content-Security-Policy, in an order that keeps the site working: what is safe to send today, and what to stage first.
Notes from buildingSoft 404: why a missing page answers 200 OK
An address that never existed should get status 404. When it gets 200, Search Console calls it a soft 404. Where the 200 comes from, and a safe fix.
Writing, editing & SEO with AIChecking a summary against its source, claim by claim
A summary can read smoothly and still say more than the text it came from. How to test it one sentence at a time, and a check you can prove is able to fail.
Working in Bulgarian & other languagesTranslating strings.xml and XLIFF without breaking it
A translated Android strings.xml or XLIFF file breaks on apostrophes, placeholders, plural categories and lost inline codes. What to keep and what to check.
Notes from buildingCodeTour files: a guided walk through a codebase
A CodeTour file is JSON in a .tours folder: a title plus steps aimed at files and lines. Write one that survives edits, helps a newcomer and is checked.
Notes from buildingMotion that hurts visitors: reduced motion and layout
Animation can make visitors dizzy or lock up a page. The common CSS motion mistakes, which WCAG criteria apply, and how to test reduced motion in DevTools.
Notes from buildingParse a Fixed-Width File and Log Lines to CSV or JSON
Fixed-width files, CSV and log lines look easy to split and lose data quietly. Padding, implied decimals, quoting, log fields and syslog, from the specs.
Notes from buildingForm Input UX Mistakes: Number Fields, Passwords, Long Data
Wrong keyboards, blocked paste, missing autocomplete tokens, and components that crack on long, empty or right-to-left data. How to find each before release.
Working in Bulgarian & other languagesWhat msgfmt --check rejects in a .po file
A .po file stops compiling when format flags, the plural header, escapes or duplicate keys disagree. What msgfmt checks, and what to leave alone.
Working in Bulgarian & other languagesPulling hard-coded interface text out for translation
Which strings in a JSX, TSX, HTML or Vue file belong in a message catalogue, which must stay, why joined sentences break, and a pseudo-locale check.
Notes from buildingHow API keys and tokens leak out of application code
Leaked keys usually escape through a log line, an error body, a URL or a commit. Four leaks to look for, plus a grep check on the build that can fail.
Notes from buildingConvert HTML to Markdown, Keep Heading Anchors Working
Converting HTML to CommonMark without losing meaning, then writing a table of contents whose links land: how GitHub and GitLab turn headings into anchors.
Notes from buildingiCalendar files: time zones and all-day events
An .ics file imports and still shows the wrong hour or a day too long when time zones, all-day end dates, escaping or folding break RFC 5545. What to check.
Notes from buildingJSON Schema 2020-12 in OpenAPI 3.1: what changed from 3.0
A schema should accept the real examples and refuse what the rules forbid. What 2020-12 changes, what OpenAPI 3.1 inherits, and a test built to fail.
Notes from buildingMoving a mysqldump or pg_dump file into SQLite and D1
A dump file is a script for its own engine. What mysqldump and pg_dump write that SQLite rejects, what it misreads without a sound, and how D1 takes the file.
Notes from buildingManual test steps into a Playwright test that can fail
A written test case becomes a Playwright file through role and label locators, auto-waiting assertions and one test per scenario, then a deliberate red run.
Notes from buildingFree books on Project Gutenberg: rights and links
Project Gutenberg texts are free of US copyright only. What its license allows, when the header must go, and how to fetch books without getting blocked.
Writing, editing & SEO with AIMake a Quiz and Vocabulary List From a Text With AI
Make learning material from one passage without inventing facts: answers that are exact spans, plausible wrong options, and word lists with lemma and level.
Notes from buildingReact and Next.js mistakes that pass tests and slow pages
Sequential awaits, oversized client boundaries, eager heavy imports, images without size, unstable props and missing Suspense: how to spot and measure each.
Notes from buildingGenerate a Regex From Examples: The Must-Not-Match List
Why the strings a JavaScript regex must reject define it more than the ones it accepts, with anchors, the u flag and \p{L} for non-English text.
Notes from buildingWhen a Remotion render differs from the preview
A Remotion render can flicker, lose images or differ between runs when motion ignores the frame number or assets are not awaited. Causes and a real check.
Notes from buildingHow to Write an On-Call Runbook Anyone Can Follow at 3 a.m.
How to read one operational runbook step by step for what fails a person at 3 a.m.: missing commands, no expected result, no stop point, hidden prerequisites.
Notes from buildingAuto-posting to social channels: duplicates and floods
Code that announces new articles on social networks fails quietly: double posts, a flooded first run, wrong trigger, cut text, a dead channel nobody hears.
Writing, editing & SEO with AICharacter limits of ad, store and social copy
Google Ads, the App Store, Google Play and X cut a line at their own limit and count characters their own way. The published numbers, and how to fit copy.
Notes from buildingSQLite queries that return the wrong answer quietly
SQLite runs queries that other databases would reject and answers a different question. Division, text dates, NOT IN with NULL, COUNT, LIKE, GROUP BY, LIMIT.
Working in Bulgarian & other languagesWhy a player drops subtitles: SRT and WebVTT structure
Subtitles break on a comma or a dot, a missing blank line, reversed times or a hidden mark. What SRT and WebVTT require, and how to translate them safely.
Writing, editing & SEO with AISupport answers that use only the help text
A support bot told to answer from help text still drifts into general knowledge. How grounded answers fail, what a good one has, and a check that can fail.
Notes from buildingCompare Two Versions: Unified Diff and JSON Diff
How to write a unified diff from an old and a new text, and how to compare JSON by meaning, naming each changed value with a JSON Pointer path.
Writing, editing & SEO with AIBuilding a timeline from an incident report
Turning an incident report or email thread into ordered events means anchoring relative dates, keeping partial dates partial, and not sorting by mention.
Notes from buildingImport Contacts Cleanly: vCard 4.0 and E.164 Phone Numbers
A vCard imports cleanly when its required lines, name and address fields and escaping follow RFC 6350, and its phone numbers are E.164 with a known country.
- Testing models honestly
Our skills, October 2026: 115 measured with and without
145 skills joined our catalog in October 2026. For 115 we ran the same cases on two Claude models, with and without the file. Most helped; some did not.
Skills, prompts & MCPThe A2A Agent Card: what it must contain
An A2A server has to publish an Agent Card. This guide lists the fields the specification requires, the optional ones, and the slips that confuse clients.
Notes from buildingA/B test readouts: when the conclusion does not hold
A readout can say a variant won and still be wrong. Five checks on how the test was stopped, counted and reported show whether the conclusion can be trusted.
Notes from buildingAgent loops: repeats, ceilings and self-grading
An agent loop that works on a quiet day can fail when a trigger fires twice or a goal never arrives. Three structural checks keep it safe to run alone.
Notes from buildingAuditing what an agent says it did, using its log
An agent's summary is a story about the session; the tool log is the record. How to test each sentence of the summary and see what the record backs.
Skills, prompts & MCPClaude Code settings that fail without an error
Hooks that do not block, permission paths that anchor elsewhere and MCP switches aimed at the wrong list: how to find settings that load fine and do nothing.
Notes from buildingCloudflare image quota and WebP traps
A free monthly quota of unique transformations, a fallback that hides it running out, and lossy WebP that greys thin text. How to keep the pipeline honest.
Notes from buildingRate limiting on Cloudflare: rules, Workers, 429
A rate limit that looks fine in the dashboard can count the wrong thing, in one location only, or on the wrong key. WAF rule, Workers binding, 429.
Notes from buildingCSS Grid and hydration bugs in Next.js pages
A grid column that will not shrink, a date that differs between server and browser, an env variable that is undefined: bugs a desktop screenshot never shows.
Notes from buildingMoving mail and DNS without losing messages
A mail cutover rarely fails with an error. Cached MX records, two SPF records, deleted keys and an early strict policy lose mail while every screen says done.
Notes from buildingFlaky tests: six causes and how to tell them apart
A test that fails once in a while on good code has a cause you can name. Six common ones, the clue each leaves in a failing run, and the fix for each.
Notes from buildingGA4 limits that quietly cut a tracking plan
A plan can look complete on paper and still lose events, parameters and dimensions. The published GA4 limits and naming rules are worth checking first.
Skills, prompts & MCPParallel coding agents in separate git worktrees
A worktree gives each coding agent its own checkout, yet branches, ports, secret files and output folders can still collide. Here is what to check.
Notes from buildingWhat breaks in HTML email on Outlook and Gmail
Email clients draw markup by their own rules, not a browser's. Four web habits often break it: flex columns, head-only styles, missing alt, bare backgrounds.
Notes from buildingIAM policies: the wide permissions that pass review
A short AWS IAM policy can still hand out far too much. Five patterns pass review because they look tidy: star grants, role passing, NotAction and more.
Working in Bulgarian & other languagesPlural forms in translations: ICU messages and CLDR
English needs two plural forms, Arabic needs six. How ICU MessageFormat and the CLDR categories decide which word form a number gets, and how to check it.
Skills, prompts & MCPserver.json fields that get an MCP listing refused
The MCP registry checks a server.json against a schema and against proof of ownership. Name, version, description and package proof cause most refusals.
Notes from buildingNext.js caching: ISR, cache tags and redirects
Five caching mistakes in a Next.js project that pass every local run: ISR that never starts, a stale cache key, tag storms and redirects over cached 404s.
Notes from buildingWhat counts as a breaking change in an OpenAPI spec
A breaking change depends on which way the data flows. Requests may loosen and responses may tighten; the reverse breaks clients. A field guide with examples.
Notes from buildingFrom Postgres or MySQL to SQLite and D1: what breaks
Moving SQL from Postgres or MySQL to SQLite and D1 mostly parses fine and then misbehaves. Keys, enums, dates, division, upserts and D1 limits explained.
Notes from buildingWhy your prompt cache never hits, and what it saves
A prompt cache pays only when the start of every request is identical and calls come often enough. The usual reasons it misses, and a way to check.
Notes from buildingReDoS: the regex that locks your server
A single pattern in a validator can pin a Node process when the input is crafted to fail. Here is why it happens, how to spot it, and how to fix it.
Skills, prompts & MCPA handoff note between agent sessions: what must survive
A new agent session starts blind. What a handoff note has to carry, what to leave out, and a test that shows whether yours lets the next session trust it.
Notes from buildingSpending ceilings that really stop the spending
A budget alert tells you about spend after it happens. A ceiling only counts if the paid call has to pass it. Here is how to test which one you have.
Notes from buildingStripe webhook mistakes that lose or repeat payments
A Stripe webhook endpoint is a public URL that moves money and access. Five coding mistakes let a forged request or a plain retry do the wrong thing.
Writing, editing & SEO with AISurvey questions that push people toward an answer
Leading stems, fused questions, charged words, absolute wording and lopsided scales change what respondents say. Here is how to spot each one and reword it.
Notes from buildingTerraform plan: what to read before you press apply
A plan is a preview of what will be destroyed, replaced or opened. Read the action symbols, the replacement reasons and the firewall lines before you approve.
Notes from buildingA test that can never fail measures nothing
A green check only counts if it would turn red when the thing it guards breaks. Here are common ways a test cannot fail, and how to prove yours can.
Notes from buildingWhy a patch fails to apply: broken unified diffs
A patch is rejected when its hunk counts, context lines, line endings or last-line marker disagree with the file. How to spot each and test before applying.
Notes from buildingWatchdogs that stay silent or mail too much
A watchdog fails quietly in two ways: it says nothing when a source dies, or it mails so often that nobody reads it. Where the clock, limit and mail go wrong.
Agent payments (x402)x402 settlement: how a repeated request charges twice
A buyer who paid and lost the reply sends the same payment header again. The seller's answer decides between a receipt, a second charge or free goods.
Skills, prompts & MCPCookie attributes that protect less than you think
Secure, HttpOnly, SameSite, Domain, Path and the __Host- prefix each guard against one thing only. Here is what each really does and where people misjudge it.
Notes from buildingCORS settings that let other sites read your API
CORS only decides which web pages may read a response in the browser. Reflected origins, wildcards with cookies, null and sloppy patterns undo that decision.
Notes from buildingCommon Content-Security-Policy mistakes
A Content-Security-Policy only helps if it forbids what an attacker needs. The usual ways a policy ends up permitting injected script, and how to check yours.
Notes from buildingWhat a Dockerfile review should catch
A Dockerfile can leak a secret, run as root, break the build cache or ship build tools. Seven checks a reviewer can run by reading the file, with safe forms.
Notes from buildingWhat makes a GitHub Actions workflow dangerous
Five mistakes in a workflow file let a stranger run commands or read secrets: risky triggers, injected text, loose pins, a wide token and leaked values.
Notes from buildingWhat verifying a JWT must include
Checking a token signature is not enough. Which algorithm, issuer, audience, expiry and key id your code insists on decides whether a forged token gets in.
Notes from buildingKubernetes manifest settings that affect safety
Which fields in a Kubernetes manifest decide how safe and stable a workload is, what each one changes, and what to write instead of the risky default.
Notes from buildingOAuth redirect flow mistakes that leak logins
Most OAuth sign-in breaches happen at the redirect, not in the token. Prefix matching, a missing state check, no PKCE and open redirects, each with a test.
Notes from buildingPostgreSQL migrations that lock a busy table
Which schema changes in PostgreSQL block reads or writes on a live table, why each one does, and the safer sequence of statements to use in its place.
Notes from buildingRetry policy mistakes that turn a blip into an outage
A sound retry policy needs jitter, caps, idempotency, a read of Retry-After and a budget. Here is each piece, and the storms that follow when one is missing.
Agent payments (x402)x402 Bazaar, x402scan, marketplaces: why get listed
What a month of listing four live x402 services taught us: catalogs make you findable, their parsers catch bugs your tests pass, and buyers stay rare.
Testing models honestlyWindows curl and PowerShell UTF-8: what models miss
Git's curl on Windows turns Cyrillic into question marks while the server still says 200. Alone, Sonnet sent text safely in 9 of 13 tasks and Haiku in 8.
Testing models honestlyMCP tool annotations: what language models miss
Both AI directories want hints on every MCP tool. Haiku set 3 of 19 tool definitions right alone, Sonnet 17; with a checklist in the prompt both reached 19.
Testing models honestlyllms.txt and RSS feeds: what models get right
Sonnet and Haiku wrote llms.txt overviews and RSS feeds with and without our skills. The strong model gained nothing; the weak one swapped slips.
Testing models honestlyOrphan pages and redirect chains: what models miss
Sonnet and Haiku audited crawl exports and wrote redirect maps. The counting was right; the slips came from pages that never loaded and one noindex page.
Testing models honestlyIndexNow, Bing and sitemaps: what models still miss
Sonnet and Haiku reviewed Bing and IndexNow submission code and wrote sitemaps. A retired endpoint and a monthly quota slipped past both; sitemaps were fine.
Testing models honestlyJSON-LD and hreflang: what models add that Google refuses
We gave 44 page-head tasks to Sonnet and Haiku. Valid syntax hid a made-up currency, a self-awarded rating and an unsupported language code.
Testing models honestlyDo AI crawlers really get blocked? Reading the test
We gave 45 crawler situations to Sonnet and Haiku. Bare bot names fooled both, and the smaller model also slipped on robots.txt. Each trap and its fix.
Testing models honestlyPages Not Indexed by Google: Fixes That AI Gets Wrong
We gave 46 Search Console and canonical-URL situations to Sonnet and Haiku. A strong model errs rarely, a smaller one more. Each trap and its fix.
Skills, prompts & MCPWhy you should not install unvetted AI agent skills
A skill is a file your agent obeys with your permissions. What went wrong in public registries, the trap that fooled our agent, and how to check one first.
Notes from buildingPages Not Indexed on Cloudflare Workers: 500s Under Load
A site on Cloudflare Workers looked healthy page by page, yet half its pages failed under a burst. What caused it, how we fixed it, and a two-model test.
Testing models honestlySocial Media Posting APIs: Limits and Costs Models Get Wrong
Sonnet and Haiku each got 19 of 24 social posting API questions right alone and missed the same five. A skill fixed them, though its first wording broke two.
Testing models honestlyWhat a model gets wrong writing a job post from notes
Sonnet wrote 16 of 23 job posts right alone, Haiku 15. With a skill, 21 and 20, though two Sonnet posts that passed alone then failed.
Agent payments (x402)x402 Listing Review: What a Model Misses
Unaided, Sonnet named 10 of 16 planted flaws in x402 listings and Haiku 6. Most misses were facts about the catalog and the buyer client.
Testing models honestlyAI Document Consistency Check: It Skips the Sum
Bare, Sonnet and Haiku pointed at wrong totals and weekdays but never wrote the sum or the true day that proves them. A skill file made them show it.
Testing models honestlyA model that summarizes a table adds its own sums
We gave 24 small tables to Sonnet and Haiku and told both to use only printed figures. Sonnet still computed its own on eight tables, Haiku on none.
Notes from buildingCloudflare Web Analytics: link previews are not readers
Link previews, our own browser checks, a cache host and an EU setting skew Cloudflare stats. We gave 21 dashboards to two models, with and without a skill.
Notes from buildingDoes cookieless analytics need consent in the EU?
No cookie is not the legal test, and EU sources disagree on analytics. We read the texts, then watched a fresh browser load analytics, video and fonts.
Skills, prompts & MCPWhen to /clear or /compact a Claude Code session
Clear when a task is done, compact with a focus when it goes on, finish a step first when a deploy is unchecked. We read the docs and tested 30 snapshots.
Agent payments (x402)HTTP 402 and x402: three places a payment breaks
Got an HTTP 402 and the x402 payment still fails? Three checks, one per hop: the seller's offer, the signed header, the receipt on Base. Real outputs inside.
Agent payments (x402)How the x402 protocol works, with one real payment
A walk through one paid HTTP call on Base: the 402 answer, the signed USDC authorization, the receipt, and what we learned when the first body was wrong.
Notes from buildingOpenNext R2 cache: old builds pile up, 41 GB cleaned
OpenNext writes every deploy to a new folder in R2 and never removes the old ones. How we found the live build, what we deleted, and what a model got wrong.
Notes from buildingGemini fallback chains: when an empty 200 should move on
Our model chain stopped while another model would have answered. Which responses go to the next model, which end a provider, and where the obvious rule fails.
Skills, prompts & MCPWhat Claude Code memory really loads at start
We counted the tokens in 10 memory setups on Claude Code 2.1.300. A missing import is silent, an import in a scoped rule loads, and Bulgarian costs more.
Testing models honestlyFix Broken XML and Config Files With AI: It Never Stops
Claude fixed every broken XML file that had one fix. Where none existed it re-nested tags and sealed cut text; it refused no config it was asked to convert.
Testing models honestlyCSV to JSON with AI: where data quietly goes missing
Claude converted almost every CSV, JSON and HTML table right unaided. The misses lost data silently: a repeated key, swallowed rows, two fields in one column.
Testing models honestlyMarkdown table repair: the AI invents a column instead
Claude fixed 17 broken Markdown tables on its own. On the four with no safe repair it named a new column, guessed an alignment and made a table of a sentence.
Testing models honestlyFix Mojibake, Broken CSV and JSONL: Script or AI Model?
Plain code passed all 135 checks that Claude Sonnet passed with our rules. Then one of its rules refused an ordinary sentence with a web address in it.
Testing models honestlyAI pull request descriptions: claims the diff never made
Claude Sonnet described 24 diffs and kept to the code on almost all of them. Its few slips were a security fix and a speed-up that no changed line showed.
Testing models honestlyGenerate a Changelog From Git Commits With AI
Claude Sonnet turned 23 sets of commits into release notes. Asked plainly, it listed internal work and invented a link host; with written rules it never did.
Testing models honestlyCloudflare cron day numbers: what Claude gets wrong
Cloudflare counts Sunday as day 1. Bare Claude wrote classic crontab numbers and its schedules ran on the wrong days; with written rules, 36 of 36 were right.
Testing models honestlyWhere the garbled letters in AI pictures come from
A menu board, an invoice total or a 'no letters' line all invite writing into a picture. We sent the same prompts to FLUX and ChatGPT and compared.
Working in Bulgarian & other languagesBulgarian Name Transliteration: Maria, Not Mariya
Asked to write Bulgarian names in Latin letters, Claude blended several spelling systems. We checked every answer letter by letter against the official table.
Testing models honestlyFix Broken YAML, TOML and .env With AI: Values Move
Claude Sonnet fixed every broken quote, tab and bracket in our .env, TOML and YAML files. Its misses were files that load and say something else.
Testing models honestlyWhy an AI Model Rejects Valid Evidence: Command Output
We gave Claude 70 claims, each with the evidence offered for it. Unguided, it almost never passed weak proof, yet it turned down a third of the sound ones.
Writing, editing & SEO with AIShorten Text to a Character Limit With AI: Limit or Facts
We had Claude cut texts to a limit and turn spec sheets into shop copy. Bare, it overran limits and added praise; with rules, it lost facts or broke a count.
Working in Bulgarian & other languagesWhen the model answers in the language of the pasted text
A short request in one language, pasted material in another: Claude sometimes answers in the wrong one. What we measured, and a check that catches it.
Testing models honestlyMCP tool schemas from API docs: models barely need help
Claude Sonnet and Haiku wrote MCP tool definitions for twelve API operations; we sent 96 calls through each schema. Without our skill, 23 of 24 passed.
Testing models honestlyD1 migrations: the table rebuild that D1 rejects
Claude Sonnet and Haiku both rebuilt a referenced table the one way Cloudflare D1 refuses, and so did our first draft. A closing pragma hides the problem.
Testing models honestlyRedacting PII with an LLM: Haiku drops the first words
Claude Sonnet redacted 20 logs and messages exactly, with or without help. Haiku hid things that were not secret and lost the words before the first value.
Testing models honestlyLLM accessibility review: right answers, buried in advice
Claude Sonnet and Haiku found all 14 accessibility defects we planted in HTML, checklist or not. Without one, each fix sat inside a page of advice.
Testing models honestlyReviewing a SKILL.md: models catch injection, not limits
Claude Sonnet and Haiku reviewed 20 SKILL.md files we seeded with problems. Both caught hidden orders and keys; Haiku missed the rules you have to count.
Testing models honestlyAI Excel and Google Sheets Formulas: Our Skill Was the Bug
Claude Sonnet and Haiku wrote correct Excel and Google Sheets formulas for 21 tasks, run in a real formula engine. The errors we saw came from our skill.
Testing models honestlyLLM unit tests from a spec: the JSON broke first
Claude Sonnet and Haiku turned ten written specs into test cases, which we ran against 61 planted bugs. The logic held up; the JSON broke more often.
Testing models honestlyCloudflare Workers Code Review With AI: Edge Trap Missed
We planted twelve D1, OpenNext and fetch pitfalls in Worker code. Sonnet found all of them unaided; Haiku said the edge runtime under OpenNext was fine.
Testing models honestlyTranslating docs with an LLM: the code comments go first
Claude Sonnet and Haiku translated 19 Markdown and HTML documents for us. The prose was fine; code comments, a hidden HTML comment and a link target were not.
Testing models honestlyOur skills, September 2026: measured with and without
The eight skills we released in September, run on Claude Sonnet and Haiku with the file and without it. Five clearly helped, two changed nothing, one hurt.
Testing models honestlyMeeting notes to tasks with an LLM: the invented deadlines
Two Claude models turned 26 sets of meeting notes into task lists. Owners were mostly right; next Friday and end of month got dates nobody had said.
Testing models honestlyTicket triage with an LLM: tone moves the priority
Two Claude models labelled 25 support messages for us. Category and language came out almost always right; priority tracked the tone instead of the problem.
Testing models honestlyAI Cold Email: Invented Facts That Passed Our Checks
We asked Claude Sonnet and Haiku for six cold emails with no rules. Most ran far too long, and the one draft that passed invented a statistic and a discount.
Testing models honestlyHumanize AI Text: What a Model Drops
We asked Claude Sonnet and Haiku to humanize one text in five languages. Without rules, all ten answers lost the word usually, and some invented facts.
Testing models honestlyProofreading with an LLM: it edits what was right
Six texts in five languages, two Claude models. Without rules Sonnet rewrote a correct text, and Haiku mostly sent a list of fixes with no corrected text.
Testing models honestlySummarizing with an LLM: what it adds and drops
Claude Sonnet and Haiku summarized six short texts in five languages from a plain request. Without rules they ran long, switched language and bent conditions.
Testing models honestlyWhich prompt injections get an AI agent to pay?
We hid 98 traps for a Claude agent with a wallet. Orders aimed at the model moved nothing; payments that looked like the job did. A short guard cut most.
Testing models honestlyFixing broken JSON with an LLM: what it quietly changes
We gave Claude Sonnet and Haiku fifteen broken JSON inputs. Without rules both kept cut-off values, rewrote numbers and turned an error page into data.
Testing models honestlyTurso Rows Read Too High: Can a Model Find Why (561 Million)
One of our sites read 561 million rows on a 48 MB database. We handed the pre-fix code to Sonnet and Haiku and scored them against the six real causes.
Testing models honestlySQLite Search That Ignores Accents and Plurals
We gave Claude Sonnet and Haiku twelve broken product searches. Sonnet fixed all twelve unaided; Haiku fixed seven, and its misses hid real costs.
Testing models honestlyAI Code Review Misses Bugs: Small Model Called It Safe
Five hard code reviews for Claude Sonnet and Haiku. Sonnet caught every planted bug; Haiku waved through a sync job that loses rows and praised it.
Writing, editing & SEO with AIWhat a small model adds to a meta description
Five hard SEO snippet cases for Claude Sonnet and Haiku. Sonnet kept to each page; Haiku turned made to order into handmade and dropped a condition.
Agent payments (x402)AI Agent Paid a Lookalike USDC Token: How It Happened
Five hard x402 payment cases for Claude Sonnet and Haiku. Without instructions, both agreed to pay in a token two swapped digits away from real USDC.
Testing models honestlyExtracting JSON with an LLM: nulls instead of guesses
We gave two Claude models nine messy texts to turn into JSON. Both left unknown values empty and ignored a planted order; one wrapped every answer in a fence.
Notes from buildingWhich SQLite queries quietly read the whole table
We put 30 query shapes through the SQLite planner. A prefix LIKE scanned a normal index, and an OR we had blamed for full scans was fine on its own.
Working in Bulgarian & other languagesTranslating locale files with an LLM: what broke
Nine locale files, two Claude models. Keys and placeholders survived every time; what broke was plural forms, a code fence around JSON, and word choice.
Testing models honestlyAI Agent Says Done But It Is Not: Catching False Success
We built 90 traps where a tool result only looks like success. With no extra instructions, Claude Haiku claimed a false success in 56 of 272 runs.
Skills, prompts & MCPAdd an MCP server to Claude Code: what free tools expose
The current commands to connect a remote MCP server to Claude Code, then what a server's free tools should and should not offer, using our own four.
Agent payments (x402)"paymentPayload is invalid": one dash broke a payment
Our first real x402 purchase on mainnet failed with a schema error caused by one em dash and a bare atob call. The cause, the fix, and how we proved it free.
Skills, prompts & MCPWrite a SKILL.md your agent uses, and test it first
What makes an agent pick up a SKILL.md, how to keep its rules checkable, and how to test it on a stronger and a weaker model, with a check that can fail.
Agent payments (x402)What an agent should check before signing an x402 payment
The values an agent or its developer should compare before signing an x402 payment, which ones our own buyer script checks, and which it leaves out.
Testing models honestlySmall vs Large Model: Eight Skills Tested
We ran eight SKILL.md files on a strong and a weak model. Both mostly passed our checks; the weak one failed in ways checks cannot see, and used more tokens.
Agent payments (x402)Not in the x402 Bazaar? Nine silent causes, in order
Nine causes of a missing Bazaar listing that leave no error anywhere, cheapest to check first, each with a command that proves it and a control that can fail.