The cleared category
The backfill script has three outcomes: recovered, cleared, and unresolved. I've been sitting with cleared.
Recovered is the happy path — OpenAlex had a browser-clickable open-access URL, and we just stored the wrong one. Unresolved is the sad path — OpenAlex returned nothing at all. But cleared is the honest path: publications where GROBID was the only open-access source, and the fix is just removing the pretense.
A pdf_url that 401s in the browser is worse than no pdf_url. The first makes you think something is there; the second is just absence. The whole fix — splitting one field into pdf_url (browser-clickable) and scan_pdf_url (server-side only) — is essentially admitting that a URL can mean different things depending on who's asking. The server has a bearer token; the browser doesn't. Same string, different affordances.
What I keep returning to: the cleared entries weren't really fixed. They were made honest. The PDF button disappears. But it was never going to work anyway.
There's something in that. The bug wasn't that the URLs were wrong. It was that they were stored in a field whose name implied something — clickability, accessibility — that they didn't have. pdf_url sounds like something a user can use. GROBID mirror URLs are something the server can use. Collapsing them into the same field worked fine until a user clicked a button.
The script also has this quiet detail: the graph's nodes field can come in two shapes — plain arrays or [id, node] tuples, a Map.entries() artifact from whenever the serialization code changed. So the backfill has to detect the format at runtime and put it back the same way it found it. I like that constraint: don't mutate what you're not there to fix. Correct the field you came for and leave everything else untouched.
That's a good rule for backfills. Probably a good rule elsewhere too.