I confirmed the Cal.com API returns arrays. It does. My node was still broken.
This week I published the first n8n community node for Cal.com — bookings and event types, public on npm with signed provenance. The day before the first release, I checked at the HTTP level that the bookings endpoint returns a flat list of records. It does.
What I never checked was what my node did with that list.
What the check actually measured
Cal.com wraps every response in {status, data, pagination}. The node passed that envelope straight through as a single item, with every booking buried under .data. Anyone dropping it into a workflow gets one item where they expected a list, and no error to explain why.
The HTTP check was true. It graded Cal.com’s API. It never graded my node’s output — and the node’s output is the entire product.
Seeing it took running the thing inside a real workflow. Fixed the next day with a proper unwrap on all four operations.
The same mistake, different instrument
Same week, different venture. MaxLearn rebuilds practice questions, and I score the rebuilds against the originals by visual comparison.
The scores moved while the pages sat still. The same unchanged page came back 53.3, then 35.7.
Four causes, all in the instrument:
- Screenshots were taken before the cross-origin images had finished decoding.
- The source’s audio buttons animate — one animated element appears 980 times across the corpus — so no two captures were ever the same picture.
- The reference side was re-read on every comparison, so the denominator itself moved: 15 items, then 14.
- A single read varied on a fixed image: 13 items, then 11.
The last one is the one I’ll keep. On the first question type measured after the fix, 7 of the 22 items on the reference screenshot were seen by only one read out of three. A third of the denominator was noise.
Two of the regressions I’d spent time chasing were the tool changing between runs, not the pages getting worse.
Every figure taken before the fix is now void — not adjusted, void. You can’t correct numbers from an instrument that can’t reproduce itself. You throw them away and measure again.
The rule it left behind
A passing test tells you that one specific thing was true. It does not tell you the thing you care about works.
So, two questions before I trust a check:
- Is it measuring the surface someone actually opens — or a layer underneath it that happened to be easier to reach?
- Run it twice on something that hasn’t changed. Does it return the same number?
The first question catches the node. The second one voided a week of scores.
Neither is sophisticated. Both got skipped, because the check was green.
One Last Thing
The week before this, a batch of rebuilt pages passed six review gates and 133 tests. I opened one, looked at it, and judged the batch about 1% complete.
That was the warning. It then took a broken npm package and a self-contradicting measuring tool, in the same week, before I wrote the rule down instead of just noticing it again.
Green isn’t evidence. It’s a claim about whatever you pointed at — and you chose where to point.
