Sizing a Network for the Job
A free lesson from Swipe AI · Deep Learning & Neural Networks, level 2 · updated October 2026
By the Swipe team, founded by Colin · how we make lessons
Picking a network's width and depth is really about matching the network's size to the job: how complex the underlying pattern is, and how much data and compute is realistically available to train it. A network that is too small may not have enough capacity to represent the pattern well at all. A network that is too large for its resources may have plenty of capacity but never get trained well enough to use it.
There is no single ideal size that works for every task. A simple, well-defined task with a modest amount of data usually pairs well with a smaller network. A far more complex task, backed by a large and high-quality dataset and plenty of compute, can justify a much larger one. Sizing a network is a judgment call based on the specific job, not a fixed formula.
The practical takeaway from this path is simple: width and depth are levers you pull before training starts, and pulling them further always has a cost. The goal is not the biggest possible network — it is a network sized to actually fit the problem you are solving and the resources you actually have.
What determines the right size for a network's width and depth?
A network that is too small for a task may lack the ____ to represent the pattern well.
There is one universally correct network size that works best for every possible task.
How Many Neurons, How Many Layers
10 questions · pass with 60% — just like in the app.
What does a network's depth refer to?
Width and depth are decided before training starts, not something the network chooses for itself.
During training, a network adjusts its ____ but does not add new layers or neurons on its own.
What is the main benefit of a larger (wider and/or deeper) network?
A larger network's extra capacity always comes at some cost in data and compute needed to train it well.
This track's earlier lesson that depth is powerful, not ____, applies to width too.
What can happen if a network is far larger than the data and compute available to train it?
There is one single ideal network size that works best for every possible task.
What should mainly guide the choice of a network's width and depth?
A network that is too small for its task may lack the ____ to represent the pattern well.
Free ai tools
Keep your streak going in Swipe AI
Swipe through every level of Deep Learning & Neural Networks on your iPhone — with XP, stars, streaks, a personal advisor and the full track beyond these free levels.
Get the Swipe apps →