This week I used to be at NVIDIA GTC in San Jose, California. It was loopy. The final large GTC was in 2019, simply as NVIDIA was finishing its Mellanox acquisition. This 12 months, that acquisition become a key part of its new DGX techniques. The larger takeaway although, was that GTC was rocking. That’s stated coming from a perspective of getting been to tons of of business reveals over time. As such, now we have 5 key takeaways from the occasion.
AI is Rocking 5 Key Takeaways from NVIDIA GTC 2024
Listed here are the highest 5 key takeaways from GTC 2024:
- One brutally obvious statement is that AI isn’t part of HPC. GTC 2024 had over 17,000 individuals. Of us have been enthusiastic about SC23 hitting 14,295 attendees in Denver, and the thought was that was swelling due to AI. After all, each reveals have off-site, non-badged attendees doing briefings, displaying NDA merchandise, and shutting offers. On the similar time, GTC felt full, even on the FOURTH day when normally present flooring are barren. Here’s a take a look at the hallway outdoors of the present ground after it closed, and that was regular visitors to see throughout the 4 day occasion.
- Develop energy techniques right this moment. The NVIDIA DGX GB200 NVL72 is a mouthful, however it is usually a robust system that makes use of 120kW. Eight of these, plus mandatory storage, switching, and so forth is probably going over 1MW of energy in a single information middle aisle. Liquid cooling goes to be necessary. We now have checked out generations now of top-end NVIDIA liquid cooled servers. Fluid, fittings, metallic blocks, and tubes are usually not the constraints on the way forward for compute. Energy is. Most prospects right this moment can determine 30kW racks. 60kW racks are doable in lots of information facilities. 120kW is a gigantic soar. On the scale of many megawatts, simply sourcing sufficient low-cost and dependable energy is a worldwide want.
- NVIDIA isn’t a gaming firm. We focus on NVIDIA GPUs as graphics processing models. The heritage was, certainly, the GeForce (and Riva?) strains, skilled graphics strains, and so forth. GTC 2024 talked a bit about OVX and Omniverse. The main target was clearly on information middle AI (Blackwell), networking (NVLink/ NVSwitch and ConnectX-8 Infiniband), and Grace. Robotics (Jetson and Drive) have been actually there. The keynote, present ground, and so forth have been dominated by information middle, robotics, and automotive. GTC has all the time had a slant on this route however after two days I used to be reflecting that if I didn’t use NVIDIA GPUs frequently and this was my first GTC, I might have nearly no concept that NVIDIA sells playing cards for gaming after two days within the convention.
- Robotics is the darkish horse. NVIDIA may discuss so much about AI right this moment, however one in all its large bets for the long run is that the identical LLM expertise used to flood the world with textual content, photos, and, more and more, video goes to make its solution to robotics. Encoding motion and extra into LLMs seems like a promising solution to extra effectively program robots. If we wish Rosie the Robotic from The Jetsons, or one thing much more human-like, {hardware} that may benefit from the huge information middle investments in LLMs is required. Jetson Thor could be an enormous one when it launches within the first half of 2025. Till then, now we have Jetson Nano.
- NVIDIA has a wild set of interconnects. Simply holding observe of those is difficult. For scale out there may be Infiniband. For administration, Ethernet. GPU-to-GPU is NVLink. GPU-to- Grace CPU is Chip-2-Chip (C2C). Blackwell half to Blackwell half is through NV-HBI. NVIDIA is designing an enormous variety of interconnects simply to maneuver information round. That’s one thing that a number of people overlook when accelerator FLOPS/ TFLOPS. A startup not solely must get the on-package, and in-chassis proper, but it surely wants the scale-out materials as nicely.
Last Phrases
Everybody I spoke to at NVIDIA GTC 2024 commented about how busy it was. NVIDIA had an worker hour on the finish which was nice. There have been many inexperienced badges strolling across the present ground and some stopped to say hello, take selfies, after which speak about what they make, typically pointing to the product.
Here’s a higher solution to put it. I spent 4 days and a night at GTC, and I by no means received to most likely 20% of the present ground nor a couple of displays I needed to see. Usually, I get a minimum of an hour to myself after conferences and already lined many of the ground. Not this journey. GTC was rocking.
