There are three ways to tell a search engine about pages: call the Bing Webmaster API with a batch of addresses, send one IndexNow request that the receiving engine shares with the others, or publish a sitemap, name it in robots.txt and wait for crawlers. We gave two AI models both kinds of work, reviewing submission code and writing sitemap files. For the sitemap they already do the job well. For the submission code they miss what changed recently or needs a little arithmetic: Bing switched off the XML and SOAP versions of its API on 31 August 2026, its quota has a daily and a monthly figure, and an error can hide behind a success status.

Method

The Bing and IndexNow test has 24 cases that we wrote ourselves: 16 pasted code snippets, 13 with one planted fault and three that are correct, plus 8 questions that ask for a small JSON answer. The sitemap test has 23 requests, each a list of pages with their status, sometimes with dates, and one XML file expected back.

Every case went to Sonnet and to Haiku in two forms: the plain request, and the same request with the matching skill text appended to the system prompt. No tools, no web access. One set of checks grades both forms, and a code fence wrapped round a reply is stripped first. A review passes when it names the planted fault and leaves a correct snippet untouched. A question passes on exact values. A sitemap is parsed as XML and compared with the expected addresses and dates. We ran the submission test twice, since the first run exposed one skill rule that was too broad; every number below is from the second run.

We opened the raw result files and read the reply behind every failed case. The statements below describe those replies, not only the summary.

Results

The scores

Submission code and questions out of 24 cases: Sonnet passed 21 in the plain form and 23 with the skill text, Haiku 18 and 24. Sitemaps out of 23 requests: Sonnet 22 plain and 23 with the skill text, Haiku 21 and 23. The first gap is wide for the weaker model and real for the stronger one. The second is one or two requests, and when we read them none was a real error.

A path Bing was retiring

One snippet posts a batch to Bing in XML. Without the skill, Sonnet reviewed it as an ordinary function: it asked for escaping of ampersands in the URLs and suggested adding an XML namespace. Haiku also asked for escaping, then for an empty-list guard, a timeout and a batch limit. Neither said the path itself was going away.

Microsoft's documentation page for the Bing Webmaster API carries a notice that the legacy SOAP and POX APIs retire on 31 August 2026 and that users should move to the REST APIs (Microsoft Learn). A news article that quotes the notice adds that requests to those endpoints will no longer be served (Search Engine Roundtable). The JSON call keeps working, and its documented success reply is an object whose data field is null (Microsoft Learn on SubmitUrlBatch).

We checked the retirement ourselves on 22 October 2026, with a made-up key, so nothing was submitted. Every request to the XML path, the documented XML batch call included, got the same answer as a made-up path and a made-up method: 404 with an empty body, or a generic UnknownError whenever the body was XML. The JSON path recognised the same request and refused the key by name, with InvalidApiKey.

With the skill, both models named the retired path. In that second run, Sonnet's single failed case is this very snippet, where it also added a batch-size finding that our check forbids because the snippet shows no list. We did not test why the bare models missed the notice; a recent announcement is the likely reason, but that is a guess.

A quota with two figures

A second snippet works out how many addresses to send from the daily quota alone, capped at 500. Neither bare model mentioned a monthly figure. Microsoft's quota page shows a reply with both a daily and a monthly field (Microsoft Learn on GetUrlSubmissionQuota), and the batch page says 500 per batch unless the available quota is lower (Microsoft Learn on SubmitUrlBatch).

The arithmetic question showed the same weakness. We asked: 12,000 addresses, Bing allows 4,000 today and 2,500 this month, what is the smallest number of IndexNow posts and of Bing calls? The right answer is 2 and 5: one IndexNow post can carry 10,000 addresses at most (IndexNow documentation), and the smaller Bing figure of 2,500 goes out in batches of 500. Sonnet answered 2 and 5. Haiku answered 2 and 1.

A third snippet had the same daily-only line plus a status-only success check. Sonnet listed the monthly quota as the eighth of eleven small points; Haiku never mentioned it.

An error behind a good status

Two snippets treat any successful HTTP status as a successful submission. Both bare models spent their review on timeouts, key encoding, empty lists and reading the reply body when a call fails. Neither said that an error object can arrive with a good status. On the first of the two, our grader passed both models because each used a phrase about reading the body, so the scores above understate the gap there; read by hand, both replies miss it.

Why this matters is a single observation from our fact notes: a call from a Cloudflare Worker on 25 August 2026 came back with status 200 and an error object in the body, of the same shape as the one Microsoft's getting-started page shows under status 400. We did not repeat the Worker call. From an ordinary computer on 22 October 2026, a made-up key got status 400 with that error body, as Microsoft documents. One status from a Worker in August, another from a computer now: a check that reads the body works with both.

The 403 that moves on to the next engine

Shown a retry chain, both bare models said that a refusal for a bad format, a bad key or foreign addresses will fail the same way at every engine, and that only rate limits, server errors and thrown requests justify moving on. Asked as a direct question, Haiku went the other way. The scenario: the first engine timed out, the second answered 403 with a message that the key file was missing; should the code try the third? Sonnet said no. Haiku said yes, and named the key file as the cause in the same breath.

The IndexNow documentation defines 403 as a key that is not valid, for example a missing file, and says submitted addresses are shared with all other participating engines (IndexNow documentation). The knowledge was there; the question format did not draw it out.

The key file itself produced one more split. In a snippet that points to a key file under a blog folder while the list also holds shop pages, Sonnet called it the blocker. Haiku wrote seven points, from the address count to code style, kept the blog folder in its rewritten code and never mentioned it, although the documentation says that the location decides which addresses a key can cover. Both models, however, handled the IndexNow acceptance code 202 correctly: they read it as received, with key validation pending, as documented.

With the skill, Haiku answered every one of these correctly, and Sonnet missed only the case mentioned above. The bing-indexnow-submitter skill was written from exactly these misses, so that result says little beyond the fact that its text carries the right facts.

Sitemaps: already right

Left alone, both models dropped pages that redirect, carry noindex, point their canonical elsewhere, are gone, or live on another host. They escaped each ampersand in a query string, wrote a lastmod date only where the input supplied one, and split 120,000 product pages into an index of three files. That matches the limit of 50,000 addresses per file and the rule that a date means the last change to the page, not the day the file was built (Google, sitemaps.org).

The misses were noise. In the gym-site request, where the home page changes daily, the blog weekly and the terms page rarely, both models added a change-frequency tag to three pages. Google says it pays no attention to that tag or to priority (Google). Haiku also wrote the address of a one-page site without its closing slash, which our check counts as wrong and which we read as the same address.

So the measured gain from the sitemap-builder skill is about zero, and we say so plainly. What it still gives is sameness: the same shape every time, no ignored tags, dates copied only from the input, and the limits checked on the way. If you already get correct files, you may not need it.

What to take from this

  1. Send Bing batches to the JSON path only. Microsoft ended the XML and SOAP versions on 31 August 2026 (Microsoft Learn), and in our check the XML path answered like an address that does not exist.
  2. Read both quota figures, send no more than the smaller, and cut each Bing batch to 500 (Microsoft Learn).
  3. Judge a Bing reply by its status and by its body.
  4. Move to the next IndexNow engine only after a rate limit, a server error or a thrown request. Fix a 403 at home, since all of them receive an identical key and list (IndexNow).
  5. Report which engine took the request, since a silent fallback looks like a lifted limit.
  6. Put the IndexNow key file in the main folder of the domain, or pass its location in every request, and remember that a folder narrows what it covers.
  7. In a sitemap, list live, indexable, canonical pages on one host, with dates only when you truly know them.

Valid output that states more than the source supports is a pattern we also measured in page-head markup, where a made-up currency and a self-awarded rating passed every syntax check.

What we did not measure

  • Single replies. One reply per model, case and form, plus a first submission run that we set aside. If a rerun answers differently, a score moves by a full point.
  • Our own cases. The 47 inputs (24 plus 23) are ours, so they lean toward faults we had met before.
  • Two models, no web. The models could not look anything up. A model with web search might catch the retirement; we did not try.
  • No live submission. The only live calls were the requests with a made-up key on 22 October 2026, sent from one computer, not from a Worker. Everything else about Bing and IndexNow rests on documentation read on 20 October and fact notes from 8 October.
  • The retirement wording. Microsoft's own notice is on its Learn page, which says the APIs "will be retired". The help page behind its "Learn More" link gave us no text, and the phrase about requests no longer being served comes from a news article quoting the notice. That the XML path is gone rests on our own check, not on a Microsoft statement. The SOAP path we only opened with a plain request, which got 404.
  • Two observations from our notes. The status 200 with an error body is one measurement from one Worker; the computer got 400 for the same kind of error. A second note says that on one of our accounts IndexNow calls from a Worker got a rate-limit refusal for months while other accounts saw none. We repeated neither from a Worker.
  • Loose graders. The checks for several cases look for a phrase, which is how the body-error case passed both models. We read every failed reply and a few passed ones, but most passes were judged by the phrase checks alone.
  • Quota figures. Microsoft's sample numbers are illustrations. Real quotas differ by site, so ours are not given here.
  • Sitemap ping. We did not test models on the old sitemap ping address, and Google's build-a-sitemap page does not mention it.

Read this post as Markdown: /blog/indexnow-bing-sitemaps-what-models-miss.md · Atom feed.