From A to B

A/B testing tools - all they do is randomize traffic. So why are we spending thousands of dollars for it? Jonas "Elvis" Alves (of AB Smartly) joined the pod to discuss some of the major challenges in building A/B testing tools. It kinda started in a way I had no idea it would - that you're not buying a testing tool, but buying a... trust platform? Jonas throws some shade at other testing tools, but I bleep it out, don't worry ;) . (And thankfully, he doesn't sales pitch at all. )We got into:- Jonas makes the case for BOTH building in-house and buying an external tool- How to earn back trust once it's broken, through data or experimentation- Booking.com ran A/B tests on A/B testers (it's super meta and super cool)Go follow Jonas on LinkedIn:⁠https://www.linkedin.com/in/jonasalves/Also go follow Shiva Manjunath on LinkedIn:⁠⁠⁠https://www.linkedin.com/in/shiva-manjunath/⁠⁠⁠Subscribe to our newsletter for more memes, clips, and awesome content!⁠⁠https://fromatob.beehiiv.com/

Is 'ethics' something we've undervaluing when it comes to experimentation? Should we actually care about this nerd stuff of 'ethics'?!Well, yes. Obviously. AJ "Does No Wrong" Long joined Shiva to chat about this specifically, as someone who works in the fantasy sports realm of things, ethics can be a contentious subject. We got into:- Why "users first" is really the only way to foolproof ethnical experimentation (and low-key, the only great way to run a business long term) - How PrizePicks fosters 'responsibility' in fantasy sports (really cool things they're doing there)- How to set (and enforce) guardrails in programs. Which is stupidly important in experimentation, when people want to break it down and hack it as much as possibleGo follow AJ on LinkedIn:⁠https://www.linkedin.com/in/angusjlong/Also go follow Shiva Manjunath on LinkedIn:⁠⁠⁠https://www.linkedin.com/in/shiva-manjunath/⁠⁠⁠Subscribe to our newsletter for more memes, clips, and awesome content!⁠⁠https://fromatob.beehiiv.com/

Emotion in data analysis, from what we've been told, should never happen. But the kicker - humans are humans. We use emotions in literally everything we do. So how do we channel it best to make sure we are positively impacting the business?Gerda "The koala" Thomas joined Shiva to chat about this topic. Where does it make sense to channel emotion? In getting EXCITEMENT about testing, and ideas! But maybe we park it when actually analyzing tests, because that can lead to bad analysis. We got into:- The good (and bad) about data. Because sometimes a hyper-fixation on certain data leads to dark patterns and bad UX- Emotion can actually be GOOD in building a culture of experimentation (well, some emotions....sometimes)- Is 'hiring culture' really THAT important to getting people involved in data? (yes....yes it is)Go follow Gerda on LinkedIn:⁠https://www.linkedin.com/in/gerdathomas/Go check out her company, Koalatative:https://koalatative.com/Also go follow Shiva Manjunath on LinkedIn:⁠⁠⁠https://www.linkedin.com/in/shiva-manjunath/⁠⁠⁠Subscribe to our newsletter for more memes, clips, and awesome content!⁠⁠https://fromatob.beehiiv.com/

Joining the discussion are Slobodan Manić, an insightful voice in digital marketing, and Shiva Manjunath, the creator and podcaster of No Hacks. They dive into intriguing conversations about the future of AI in 2026, debating its declining trust and potential impact on marketing. Shiva predicts the fading importance of website-focused CRO as AI takes over decision-making. The duo also critiques the rise of low-quality AI-generated content and emphasizes the need for authenticity to stand out. Their insights offer a thought-provoking glimpse into the evolving digital landscape.

Don't know what the hell CRO and experimentation is? Trying to understand the basic concepts and process? We gotchu. Rommil "The Thrill" Santiago and Shiva Manjunath went into all the common topics, and we can even help you explain this to others who might not fully understand what you do in CRO. Please passive aggressively share this episode to people who don't understand CRO. I will fall on my sword for you. We got into:- Why CRO is seen as a Hail Mary (and why it SHOULDN'T be....)- CROs shouldn't exist. Because everyone should be doing CRO in some way- What the hell MDE really even is? And do you even need it? (Yes....yes you do)Go follow Rommil on LinkedIn:⁠https://www.linkedin.com/in/rommil/Go check out his book:https://www.experimentnation.com/prove-it-or-lose-itSign up for Experiment Nation (I'm there... he's there too I guess): https://www.experimentnation.com/Also go follow Shiva Manjunath on LinkedIn:⁠⁠⁠https://www.linkedin.com/in/shiva-manjunath/⁠⁠⁠Subscribe to our newsletter for more memes, clips, and awesome content!⁠⁠https://fromatob.beehiiv.com/

Kelly Hopping, CMO at Demandbase and host of the NextGen CMO podcast, shares her extensive marketing expertise. She discusses the critical link between marketing efforts and pipeline metrics, emphasizing that senior leaders care more about business impact than details. Kelly shares how to effectively communicate results to gain buy-in and offers career pivot advice. She also highlights the importance of A/B testing versus qualitative research and provides tactics for CRO practitioners to align with executive priorities.

Finding clients is hard enough - finding GOOD ones? Feels damn near impossible. I enlisted the help of Tas Bober to help navigate this complex field of getting client work, and explored topics related to CRO/client work as well!We got into:- Bringing confidence to the client relationship (you don’t always have to cave in to EVERY little freaking thing they want you to do)- Running from bad clients when they don’t pass the vibe check (and how to find shitty clients BEFORE they pay you)- How to talk to clients (and stakeholders) appropriately without talking DOWN at them (hint: treat them like they’re children. It works.)Timestamps:00:00 Episode Start4:12 Detecting Client Red Flags BEFORE Signing A Contract With Them8:46 What To Do When A Client Gives You The ‘Ick’16:07 Importance of Setting (And Enforcing) Boundaries20:30 Setting Success Metrics for Engagement Upfront29:46 When “Leading With Data” Doesn’t Make Make Sense…?46:20 Tas’ BIGGEST Pro Tip (Don’t Just Skip To This Timestamp Ya Leetches)Go follow Tas Bober on LinkedIn:⁠https://www.linkedin.com/in/tasbober/ ⁠Go check out her podcast "Notorious B2B": https://www.linkedin.com/company/notorious-b2b/posts/⁠Also go follow Shiva Manjunath on LinkedIn:⁠⁠⁠https://www.linkedin.com/in/shiva-manjunath/⁠⁠⁠Subscribe to our newsletter for more memes, clips, and awesome content!⁠⁠https://fromatob.beehiiv.com/

People are finding websites more and more manipulative. And we don't mean this word lightly. Why is this the case? In large part, it's because of the shitty tactics people are using on sites. But really, it's because websites care far more about short term sales and metrics instead of focusing on LTV. And that leads to hacks, not long term gains. Made With Intent released a REALLY great report which went into detail on this, and I had the man, myth, and legend David "Zedd" Mannheim on the pod to chop it up to discuss that report (among other things). We got into:- Why People Feel Like Websites are Manipulative- How Popular 'Best Practice' Tactics Are Probably Hurting You Way More Than Helping- Tips and Tricks on Maximizing Your Popups (WITH DATA)Timestamps:00:00 Episode Start5:05  People Feel Websites Are Too Manipulative11:12 Websites Seem to Prefer Sales Over CX14:40 What Are Pop Ups Getting Wrong19:02 Marketers Are More Focused On Numbers Than Humans26:56 Does Social Proof Still Work In 202531:18 Scarcity Messaging35:55 Discounting Isn’t A Strategy42:02 Segmentation Is One Of The Most Important Things To Focus OnGo follow David Mannheim on LinkedIn: https://www.linkedin.com/in/davidleemannheim/ Download the full report here (it's a REALLY good read):https://www.madewithintent.ai/the-intent-gap-report Also go follow Shiva Manjunath on LinkedIn: ⁠⁠https://www.linkedin.com/in/shiva-manjunath/⁠⁠Subscribe to our newsletter for more memes, clips, and awesome content! ⁠https://fromatob.beehiiv.com/

We back out here with the amazing Emma Travis to dig into 'quality vs. quantity' when it comes to experimentation programs. The conversation Ben Labay keeps trying to force, but a bit more nuance to it. There is a time and place for quality. But there is a time and place for velocity and scaling your program up in terms of number of tests run. How do you know when to do it? And what are triggers to get you .... ahem.... From A to B. Emma "Biscuits and Gravy" Travis joined me on this episode to go DEEP into this topic. Emma was sick and still trooped through this episode - give her all the flowers!!!We got into:- When to focus on velocity vs. when to focus on ‘quality’ of experiment- Tips on auditing your program to know if you’re too focused on quality vs. quantity- Smart ways to focus on research when resources are limitedTimestamps:00:00 Episode Start3:57 Is Velocity Appropriate In Tracking Metric For Experimentation Maturity?9:23  Velocity Increase Doesn't Always Mean A Decline In Quality11:51  Experimentation Velocity Is Like Flying A Plane19:19 When To Scale Velocity Vs. When To Focus On Quality24:33 Run A Heuristic Audit Of Your Cro Program (*Bonus Points For Doing It In Miro)28:00 What Defines "Quality" In A Test?36:35 How Do You Communicate A Decline In Velocity Knowing You're Increasing Quality?40:35 Good Ways To Use Ai To Help You Speed Up Research Go follow Emma Travis on LinkedIn: https://www.linkedin.com/in/emma-travis/ Also go follow Shiva Manjunath on LinkedIn: ⁠https://www.linkedin.com/in/shiva-manjunath/⁠Subscribe to our newsletter for more memes, clips, and awesome content! https://fromatob.beehiiv.com/

In this episode, Rishi Rawat, a consultant specializing in copywriting for conversions, shares his insights on heuristic audits. He discusses the essential components that transform a mediocre audit into a valuable one. Rishi highlights the frequent shortcomings in current auditing methods and the crucial distinction between expert and non-expert evaluations. They also explore the importance of empathy in web design, understanding user emotions, and how these factors can significantly influence user experience and conversion rates.