Narrow Data In, Narrow Results Out

A model's outputs are shaped entirely by its training data. If that data is narrow — say, text from only one topic, one style, or one narrow slice of the world — the model tends to perform well only on that narrow slice and struggle or produce skewed results outside it. It never saw anything else to learn from.

The same idea applies to flawed or unrepresentative data, not just narrow data. If a dataset contains factual errors, skewed viewpoints, or systematically underrepresents certain groups or situations, the model tends to absorb those same issues into its patterns and reproduce them in its outputs — without knowing it's doing anything wrong.

This is why the people building a model care so much about what goes into training, not just how much of it there is. A model trained on a huge pile of low-quality or narrow data isn't automatically better than one trained on a smaller, more carefully chosen dataset — quality and coverage matter alongside sheer scale.

What tends to happen when a model is trained on narrow or flawed data?

A model's outputs are shaped by its training data, so data that is narrow or flawed tends to produce results that are similarly ____.

A bigger training dataset is always better than a smaller one, regardless of quality.