In the first week of running a newly built pipeline, one step reported success. It hadn't worked.
We build a lot of our own workflows in-house, including the one that produces the content you're reading right now. Eight days after we finished building the biweekly version, one step pulled the wrong source. It's the step that turns a finished blog post into shorter social pieces. Instead of the current, human-edited post, it read an earlier draft, written before that week's edits had landed. It generated a full batch of social content from the stale draft. It reported success. There was no error, no warning, no flag of any kind.
That's the part worth pausing on. New systems make mistakes while they're being tested, that's normal. What's worth noticing is what this one looked like from the inside: from the system's own point of view, nothing had gone wrong.
What "succeeded" actually meant
This wasn't a hallucination, and it wasn't the model getting something factually wrong. The step did exactly what it was told: fetch the post, generate derivatives from it. It just fetched the wrong version, and had no way of knowing that. The code executed, the output looked plausible, and the batch finished on schedule. The only thing wrong with it was invisible to the step itself, because the step had no concept of "current" versus "stale" in the first place. It just had a location to read from, and it read from it.
Why "it ran" isn't the same as "it ran correctly"
Automated systems are generally built to catch the failures their designers thought to imagine: a missing file, a timeout, an API error. What they're much worse at catching is a step that completes cleanly while operating on a false premise. This is the part worth sitting with if you're testing, or trusting, any process where one step's output silently becomes another step's input, whether that's a content pipeline like ours, a CRM update that's supposed to trigger a billing step, or a dashboard that's supposed to reflect this week's numbers. A system can execute a broken assumption flawlessly, with full confidence, and never once flag that anything is off. Confidence and correctness are not the same signal, and most automation only reports the first one. That distinction doesn't show up in a status log. It shows up when someone compares the output to what it should have been, which is a manual step until you build it into the process itself.
What actually caught it, and why it didn't go further
A person did, during the review step already built into the process, before that batch went anywhere near publish. Looking at the derivatives against the actual post, the mismatch was obvious: phrasing and structure that had been changed during editing were still showing up in the social drafts, because the social drafts had been built from the pre-edit version. Nothing generated from the stale draft went out publicly. That review step is exactly why we test a new pipeline against real work before trusting it with anything that leaves the building.
The fix wasn't a smarter model or a better prompt. It was structural: the derivative step now pulls directly from the same, current version the human editors are working in, instead of relying on its own memory of an earlier stage. That closed this specific gap. We haven't found the same pattern in the pipeline's other steps, and we're not calling that settled. Checking is its own ongoing task, not a box ticked once. The review step didn't get more sophisticated because of this, it just got one more question added to it: before anything moves forward, does the source it just used match the source a human would recognize as current? That's a small addition. It's also the only thing that would have caught this specific failure sooner than it was caught.
Where this doesn't apply
Not every automated tool has this risk. A single-step process, one that starts and finishes a task in one pass with no dependency on another step's output, doesn't have a handoff to get wrong. This specifically shows up where a later step assumes an earlier step's output is still accurate by the time it runs, and the more steps you chain together, the more places that assumption can quietly be wrong.
Before you trust the next automated handoff
Pick one place in your business where a second step depends on what a first step produced: a CRM update that triggers an email, a report that feeds a dashboard, an approval that clears a next stage. Ask one question about it: does the second step pull the current, approved version of that thing, or does it work from its own earlier memory of it?
You don't need to audit the whole system to find out. Pick one handoff, trace it back one step, and look at what it's actually reading from right now. If the answer isn't obvious in under a minute, that's the same gap we found in ours.
If you're not sure which one is true, that's the gap. It won't announce itself, and it won't fail loudly. It'll report success right up until someone happens to check, which is exactly why we check before we trust, not after.
