There's also copy-on-write to consider. If you actually had to allocate real memory-backed pages for every allocation a process made, had the right to, but never modified, the size of each process would be a lot larger. For example, if a process consumes 1GB of memory and forks a child, the child has access to the same 1GB of memory, but it doesn't really consume 1GB of memory. It has mapped a bunch of copy-on-write pages from its parent, and when either of them next modify those pages, the real memory is allocated.
If they never modify those CoW pages, they can both happily keep using the same copy of the page in memory, and you can have two 1GB processes using a total of e.g. 1.01GB of real memory.
This is very useful in practice, but it means that the system needs the ability to over-commit memory allocations (allow CoW allocations etc., when there is no actual memory available to back it), and over-commit currently, and probably should, requires swap (some place to dump pages in case an over-committed allocation comes calling).