Kafka's origins at LinkedIn

Jay Kreps is the co-founder and CEO of Confluent, the company built around Apache Kafka — the open-source data streaming platform he originally built while at LinkedIn. In this conversation, Jay shares his full journey: how Confluent grew from a scrappy group of engineers with no go-to-market experience into a publicly traded enterprise software company. He makes the case that the difference between what a company can do, and what it must do, is one of the most underrated building levers; illustrated through his years spent pushing Confluent towards a cloud product, in the face of widespread opposition.

In this episode, we discuss:

• Why moving from software engineer to CEO requires almost an entirely new skillset
• The product marketing pyramid Jay built to explain Kafka to the world
• How Confluent bludgeoned its way to a cloud-first business when the early product was “embarrassing”
• The critical difference between what a company can do and what it must do
• What keeps scaling companies from becoming "Chipotle”

References:

• Amazon Web Services: https://aws.amazon.com/
• Apache Kafka: https://kafka.apache.org/
• Benchmark: https://www.benchmark.com/
• Confluent: https://www.confluent.io/
• Jun Rao: https://www.linkedin.com/in/junrao
• LinkedIn: https://www.linkedin.com/
• McKinsey & Company: https://www.mckinsey.com/
• MySpace: https://www.myspace.com/
• Neha Narkhede: https://www.linkedin.com/in/nehanarkhede
• Oracle: https://www.oracle.com/
• Red Hat: https://www.redhat.com/
• Snowflake: https://www.snowflake.com/

Where to find Jay:

• LinkedIn: https://www.linkedin.com/in/jaykreps/
• Twitter/X: https://x.com/jaykreps

Where to find Brett:

• LinkedIn: https://www.linkedin.com/in/brett-berson-9986094/
• Twitter/X: https://twitter.com/brettberson

Where to find First Round Capital:

• Website: https://firstround.com/
• First Round Review: https://review.firstround.com/
• Twitter/X: https://twitter.com/firstround
• YouTube: https://www.youtube.com/@FirstRoundCapital
• This podcast on all platforms: https://review.firstround.com/podcast

Timestamps:

01:18 Making the leap from engineer to CEO

03:33 The 80% rule: what a CEO actually needs to know

04:54 Scaling different business disciplines

09:31 How Confluent’s story began in LinkedIn

12:13 The growing need for scalable data tech

13:37 What the early Kafka product looked like

16:38 Kafka’s underwhelming open-source launch

18:38 The blog post that accelerated Kafka’s adoption

20:16 Why so many marketing messages fail

28:08 The decision to build Confluent

34:24 Planning to fundraise before building the product

39:19 Confluent’s early years: Tough product decisions

47:07 The underrated growth lever question for companies

55:46 Why founder optimism is an overrated trait

1:00:29 What should founders give up as they scale?

1:02:47 Why people become trapped in a failure mindset

1:08:33 The Chipotle problem: Losing excellence at scale