Of course Huang will lean on the software being key because he sees the hardware competition catching up.
Google, Meta, Amazon do “shallow and broad” software. They are quite fast at capturing new markets swiftly, they frequently repackage OpenSource core and add the large amount of business logic to make it work, but essentially follow the market cycles - they hire and layoff on a few year cycle, and the people who work there typically also will jump around industries due to both transferable skills and relatively competitive competitors.
NVDA is roughly in the same bucket as HFT vendors. They retain talent on a 5-10y timescales. They build software stacks that range from complex kernel drivers and hardware simulators all the way to optimizing compilers and acceleration libraries.
This means they can build more integrated, more optimal and more coherent solutions. Just like Tesla can build a more integrated vehicle than Ford.
They're building it for themselves and employ world-class experts across the entire stack.
How can NVIDIA develop "more integrated" solutions when they are primarily building for these companies, as well as many others?
Examples of these companies doing things you mention as being somehow unique to or characteristic of NVIDIA:
Complex kernel drivers or modules:
- AWS: Nitro, ENA/EFA, Firecracker, NKI, bottlerocket
- Google: gasket/apex, gve, binder
- Meta: Katran, bpfilter, cgroup2, oomd, btrfs
Hardware simulators:
- AWS: Neuron, Annapurna builds simulations for nitro, graviton, inferentia and validates aws instances built for EDA services
- Google: Goldfish, Ranchu, Cuttlefish
- Meta: Arcadia, MTIA, CFD for thermal management
Optimizing Compilers:
- Amazon: NNVM, Neo-AI
- Google: MLIR, XLA, IREE
- Meta: Glow, Triton, LLM Compiler
Acceleration Libraries:
- Amazon: NeuronX, aws-ofi-nccl
- Google: Jax, TF
- Meta: FBGEMM, QNNPACK
You must have an amazing CV to think these are shallow projects.
Meta builds hardware from chip to cluster to datacenter scale, and drives research into simulation at every scale, all the way to CFD simulation of datacenter thermal management.
