AI Inference

“Training will continue to matter, and Nvidia’s current architecture, including high-speed compute, large amounts of high-bandwidth memory, and high-speed networking, will likely continue to dominate. Answer inference will be a meaningful market, albeit a relatively small one, and speed from chips like Cerebras or Groq … will be very useful. Specifically, agentic inference will be the largest market by far, because that is the market that won’t be limited by humans or time.”