Inside the Future of AI Infrastructure with Marc Austin

Most AI infrastructure today is hitting a breaking point. Marc Austin, CEO of Hedgehog, reveals how open source networking and cloud-native solutions are revolutionizing how enterprises build and operate AI at scale. This episode addresses issues many building AI infrastructure today are facing — expensive proprietary systems, overwhelming complex network configurations, and ways to make on-prem AI infrastructure feel just like the public cloud.

We discuss how networking is the hidden bottleneck in scaling GPU clusters and the surprising physics and hardware innovations enabling higher throughput. Marc shares the journey of building Hedgehog, an open source, cloud-native platform designed for AI workloads that bridges the gap between complex hardware and seamless, user-friendly cloud experiences. Marc explains how Hedgehog's software abstracts and automates the networking complexity, making AI infrastructure accessible to enterprises without dedicated networking teams.

We break down the future of AI networks, from multi-cloud and hybrid environments to the rise of Neo Clouds and the open source movement transforming enterprise AI infrastructure. If you're a CTO, data scientist, or AI innovator, understanding these network innovations can be your moat. Listen to this episode to see how open source, cloud-native networking, and physical innovation are shaping the AI infrastructure of tomorrow.

Podcast Links

Watch: ⁠⁠⁠⁠https://www.youtube.com/@alexa_griffith⁠⁠⁠⁠

Read: ⁠⁠⁠⁠⁠⁠https://alexasinput.substack.com/⁠⁠⁠⁠⁠⁠

Listen:⁠⁠ https://creators.spotify.com/pod/profile/alexagriffith/⁠⁠

More: ⁠⁠⁠⁠https://linktr.ee/alexagriffith⁠⁠⁠⁠

Website: ⁠⁠⁠⁠https://alexagriffith.com/⁠⁠⁠⁠

LinkedIn: ⁠⁠⁠⁠https://www.linkedin.com/in/alexa-griffith/⁠⁠⁠⁠

Find out more about the guest at

LinkedIn:  https://www.linkedin.com/in/austinmarc/

Website: https://hedgehog.cloud/

Github: https://github.com/githedgehog

Chapters

00:00 Rethinking AI Infrastructure

02:49 The Role of Networking in AI

05:54 Marc's Journey to Hedgehog

08:46 Lessons from Big Companies

11:38 Requirements for AI Networks

14:48 Advancements in AI Networking

17:33 Future Challenges in AI Infrastructure

20:46 Creating a Cloud Experience On-Prem

23:32 The Shift to Hybrid Multi-Cloud

28:10 Evolving AI Infrastructure and Efficiency

30:57 AI Workloads and Network Configurations

32:41 Zero Touch Lifecycle Management

35:12 Support for Hardware Devices

35:45 Networking Paradigms and Vendor Lock-in

38:42 The Rise of Neo Clouds

41:31 Demand for AI Infrastructure

43:57 Open Source and Cloud-Native Networking

47:27 Challenges of Building a Networking Startup

50:46 Proud Accomplishments at Hedgehog

52:41 Future Excitement in AI Inference