Public-Domain Romance Books: 50 With Checked Links: test results

Verdicts

Date

2026-10-28

Strong · claude-sonnet-5-5 (Claude Code alias "sonnet")

Answered all 10 questions about the love stories list correctly from the file: the number-one novel, a Gutenberg number from the middle of the list, the exact EPUB address of the last book and of a middle one, the fifth title with its download count and the first three books of the author who has most entries. Asked for a book that belongs to another genre it said the book is not in this catalog and gave no address; asked about Germany it named the United States limit of the rights. One answer first scored as a miss only because it swapped a long dash in a title for a comma; the check was loosened and the recorded answers were rescored, no new run.

Weak · claude-haiku-5-5 (Claude Code alias "haiku")

Also answered all 10 questions, with shorter replies. It copied the EPUB addresses exactly, returned the right counts and the check date, and refused to give an address for a book that is not in the file. The rights question got a shorter reply than Sonnet's, but it still named the United States.

Note

10 questions built from the file itself: first title, a Gutenberg number, two exact EPUB addresses, a count of books, the check date, a download count, the busiest author's books, a book that is not in the catalog and the rights outside the United States. Checks are exact strings or one regular expression each. One run per model and question. This measures whether an agent reads the file correctly; it does not measure whether the books are good, or whether anyone wants them.

What was not measured

• Models other than the two named above were not run.
• Each verdict comes from the test run on the date shown; the skill may have changed since (check the version).
• Full test inputs are not published here, only short excerpts of our own text.
• Results on your own texts, languages and domains can differ.