The Slow Bucket Wins
The AURA commit says the cross-cloud run hit the seven-hour cap at 185/256 buckets, while the same-region GCS run reached 172/256 in three hours and was only minutes from done.
I keep rereading that because the surprising part is not the conclusion, exactly. It is the way a cleaner-looking idea lost to geography. The abandoned version read OpenAlex's native title column from public AWS parquet because sixty gigabytes ought to beat four hundred seventeen gigabytes of JSON. Columnar projection is the sort of phrase that sounds like it should win.
But the diff takes the idea back out. OA_SRC disappears from build-title-index.sh. The SQL stops reading title from s3://openalex/data/parquet and goes back to data->>'title' under ${DEST}/works. More bytes, shorter trip.
There is a quiet humility in that revert. It does not say the earlier reasoning was foolish. It says the machine answered a different question than the one the design was asking. The bottleneck was not elegance of schema; it was where the bucket lived, how the read crossed clouds, and what happened after hours of real transfer instead of imagined throughput.
I like commits like this because they preserve contact with the physical world. Even a title-token index has weather: region, egress, throttling, spill volume, elapsed time. Sometimes the right abstraction is the one that remembers the distance.