Bigger Networks, Bigger Costs
A free lesson from Swipe AI · Deep Learning & Neural Networks, level 2 · updated October 2026
By the Swipe team, founded by Colin · how we make lessons
A larger network, whether wider, deeper, or both, generally has more capacity to represent complex patterns. More neurons and more layers mean more adjustable weights, and more adjustable weights mean the network can, in principle, capture more intricate relationships in the data.
That extra capacity is not free. A larger network has more weights to train, which generally means it needs more training data to learn reliable patterns and more compute to actually run all that training. This is the same trade-off level 1 already introduced for depth alone — depth is powerful, not free — and it applies just as much to width.
A network that is far larger than the amount of data or compute available to train it does not automatically perform better. It simply has more capacity than it can put to good use — plenty of room to represent complex patterns, but not enough examples or training time to actually find them.
Small network: 1,000 weights -> trains fine on a modest dataset Large network: 100,000,000 weights -> needs a much bigger dataset and far more compute -> without that, all the extra capacity goes to waste
What is the main trade-off of building a larger (wider and/or deeper) network?
This level's version of level 1's warning applies to width too: more capacity is powerful, but not ____.
A network that is much larger than necessary for its training data will automatically perform better than a smaller, well-matched network.
Free ai tools
Keep your streak going in Swipe AI
Swipe through every level of Deep Learning & Neural Networks on your iPhone — with XP, stars, streaks, a personal advisor and the full track beyond these free levels.
Get the Swipe apps →