The Big Problem with GPUs and How To Solve It | Randolph and Kandan, MantisGrid AI

We don't want the industry to fall apart, that's why we started this company.

6 months ago, everyone was talking about model deployment. Now everyone has moved on to agentic AI deployment and autonomous agent swarms. But all of this is running on an infrastructure that itself is not automated.

And this underlying infrastructure has massive GPU cluster systems. If even one GPU fails the whole training workload fails. You can't have 100,000 employees managing 100,000 GPUs. But LLMs are not the right solution for managing the infrastructure.

Kandan and Randolph have spent decades operating large-scale telecom and cloud infrastructure, in a world where pagers used to run lives of people managing it. With MantisGrid they have come together to build 'The' model for AI infrastructure.

Watch the episode to understand why making AI infrastructure autonomous is so important, why LLMs are not the answer, and what it takes to build the systems that will keep millions of agents running.

00:00 — Invisible Infrastructure Behind AI
01:05 — The Pager Nightmare
04:01 — The GPU Problem Nobody's Solving
05:38 — The Cloud Infrastructure Story
07:53 — Why Just Watching Isn't Enough
09:35 — Building the Waymo of AI Infrastructure
15:00 — How an Autonomous AI Infra System Works
20:35 — What Happens to DevOps?
22:46 — Why LLMs Shouldn’t Run Production
29:14 — The Path to a Billion Dollars
32:06 — The Problem Hyperscalers Can’t Solve
37:03 — Earning Trust to Run Infrastructure
39:55 — When Agents Start Managing Agents

-------------
India’s talent has built the world’s tech—now it’s time to lead it.
This mission goes beyond startups. It’s about shifting the center of gravity in global tech to include the brilliance rising from India.
What is Neon Fund?
We invest in seed and early-stage founders from India and the diaspora building world-class Enterprise AI companies. We bring capital, conviction, and a community that’s done it before.
Subscribe for real founder stories, investor perspectives, economist breakdowns, and a behind-the-scenes look at how we’re doing it all at Neon.
-------------
Check us out on:
Website: https://neon.fund/
Instagram: https://www.instagram.com/theneonshoww/
LinkedIn: https://www.linkedin.com/company/beneon/
Twitter: https://x.com/TheNeonShoww

Connect with Siddhartha on:
LinkedIn: https://www.linkedin.com/in/siddharthaahluwalia/
Twitter: https://x.com/siddharthaa7
-------------
This video is for informational purposes only. The views expressed are those of the individuals quoted and do not constitute professional advice.

Send us Fan Mail