Feedback Loops: When Output Becomes the Next Input

So far, every mechanic covered traces bias back to data collected before a model was ever deployed. But once a system is actually in use, something else can happen: its own outputs start becoming part of the data that shapes its future. That's a feedback loop, and it's one of the more overlooked ways bias gets worse over time rather than staying constant.

A concrete version: a system that recommends which job listings to show which candidates starts out with a small skew — it shows one category of listing slightly more often to one group. People can only apply to jobs they're shown, so that group ends up applying, and getting hired, into that category at a higher rate. If that outcome data is later used to retrain or fine-tune the system, it now sees even stronger evidence that this group "belongs" in that category — a pattern the system itself helped create.

The loop doesn't require any intent. Nobody has to decide to make the system more biased — the ordinary act of using real-world outcomes as future training signal is enough, if those outcomes were already shaped by the system's own earlier behavior. That's what makes it a compounding problem rather than a one-time skew.

What defines a feedback loop in this context?

In a feedback loop, a system's earlier output can shape the real-world ____ that later becomes its own training data.

A feedback loop requires someone to deliberately decide to make the system more biased over time.