Finding Our Way

Paul Ford, journalist-technologist and co-founder of Aboard, shares his view from running a services firm amid AI upheaval. He discusses collapsing software costs and role blurring. He compares AI to past transformative tools and calls the rollout chaotic. He outlines solution-engineer workflows, enterprise security needs, and how design and validation work are shifting.

John Gleason, design and business consultant who helped lead P&G’s design revolution, returns to discuss leadership for designers. He talks about boldness and vulnerability in leadership. He explores translating design into measurable business levers. He explains building influence with the C-suite and practical moves to avoid overwhelm and scale strategic impact.

They use surfing as a metaphor to explore waves of technological change and the nested pulses inside AI’s rise. They talk about attention fragmentation, liminal overwhelm, and the risks of reactive leadership. They discuss collective action, cross-functional partnerships, and when to let a wave pass to protect focus and avoid burnout.

Jorge Arango, consultant and author who blends information architecture with AI transformation. He talks about rediscovering fundamentals through AI. He frames work as helping organizations act intelligently. He links well-structured information to better AI outcomes. He urges hands-on practice with AI, clarifying goals before automating, and using AI to lift design to higher-level direction.

Dan Saffer, designer, author, and CMU HCI professor, reflects on teaching design as AI reshapes tools and workflows. He explores which fundamentals still matter, how rapid AI prototyping changes collaboration, and what AI skills designers need. Short takes on team dynamics, quality standards, and emerging interaction patterns keep the conversation lively.

They explore the mindset of leading when old ways fall away and new paths are unclear. Conversation covers vigilance, situational awareness, and flexible visions over fixed plans. They talk hiring for adaptability, core non‑craft skills like communication and persuasion, and using scenario planning to navigate uncertain futures. Relationships and storytelling are framed as lifelines during transitions.

They explore leading through in-between moments when old ways fail and the new are unclear. Conversations cover how design teams navigate uncertainty, the tension between exploration and execution, and why divergent thinking and strategic design matter now. They discuss shifting roles, AI-driven tooling changes, and how leaders can enable teams while letting outdated practices fall away.

Christina Wodtke, Stanford lecturer, multi-time founder and author, discusses AI’s paradoxes in design. She explores rapid AI prototyping, the risk of losing critical thinking, ethical tool choices, shifting team roles, and teaching product sense. Short, practical takes on context engineering, prototyping in code, and when to use AI versus human judgment.

Show Notes Peter and Jesse discuss findings from Peter’s survey of 750 UX pracitioners on organizational health. Designers feel good about their work but struggle with quality standards, staffing, and career growth. Senior practitioners are the unhealthiest group. Reporting structure predicts team health. Consulting teams outperform in-house teams, where visionary design capabilities have atrophied and empowerment remains elusive. Read the 2025 State of UX/Design Organizational Health report, including access to the survey data for your own exploration. Transcript Jesse: I’m Jesse James Garrett, Peter: and I’m Peter Merholz. Jesse: And we’re finding our way, Peter: navigating the opportunities Jesse: and challenges Peter: of design and design leadership. Jesse: On today’s show, our very own Peter Merholz shares with us insights and perspectives from his 2025 survey on the organizational health of digital design teams. We’ll look at where designers feel they’re delivering value, where they’re struggling, and the key indicators that your team might not be set up for success. Hello Peter. Peter: Hey, Jesse. Jesse: Welcome to the show. Peter: Glad to be here. Jesse: As always. Today… Peter: Do I have a choice? Jesse: Hey, this was your idea. Peter: The podcast was your idea… Jesse: No, you’re right, you’re right. This episode was your idea. We’re gonna talk a little bit about the survey that you did last year around organizational and team health for design teams. And I wonder just to kind of set things up what were you hoping to learn by doing a survey of design practitioners out there in the world? Organizational Health Survey Peter: So this survey, this research, has been brewing for me for years now. I had created a version of the survey that was specific to individual design organizations to be used to help clients a number of years ago, just as part of my practice. And I’ve realized, particularly, I think, in that 2023-ish timeframe, as things started getting confusing and discombobulated and disrupted, that it would be interesting to take a snapshot of the state of the industry, and that this survey that I had been using to help specific design teams understand their organizational health, like, how, how well are they performing and, how healthy are these organizations, and organizational health means things like the mindset of the people on the team, how motivated and engaged are they, how supported are they feeling? Are they able to do good work? All those types of things. I thought, oh, this could be helpful industry-wide, just to get a sense, pulse check. But I’m just one guy. And it laid dormant for about two and a half years until Lenny Rachitsky did his survey of product development teams, one of the findings of which was around burnout. And within that finding around burnout, that designers are the most likely to be burned out within a product development org, more likely to be burned out than product managers, engineers, et cetera. And that caught a lot of attention. There was a lot of commentary about it, primarily on LinkedIn. And so that was the trigger to be like, okay, I gotta get this thing out there. And we need to, by we, I mean kind of the UX design community, we’ll benefit from a look within, right? The Lenny study was broader. It was not about design, though it included designers. I wanted to really take that snapshot of what the UX and design industry is feeling about the organizations they’re part of. And so that’s what spurred it. Like, literally weeks after Lenny’s survey, I published mine, which in some ways was perhaps unwise because I did it right at the start of summer. Like had I, done it in the spring, I probably would’ve gotten, it would’ve been easier to get a, good feedback, but did it in the summer. But I was still able to get nearly 750 respondents from around the world to share what their experience is inside their various organizations. And I think come up with a decent, informed perspective on the state of the health of the teams that UXers and designers are part of. Jesse: Got it, and it would seem from the survey results that that health would be approximately 3.28 out of five. Peter: Yes. Solved. It’s pretty mediocre. Or middling is probably, maybe a better kind of word for it. You know, there’s some kind of demographic questions just to try to get a sense of the size of the organization you’re in and where is your organization within a larger company? Are you reporting through product and engineering? What kind of industry are you in? Financial services, professional services, retail. So there’s, a set of demographic questions and then there’s a set of, they’re not questions, they’re statements. Likert statements, right? Where you put a statement out there and then you have someone say whether or not they strongly disagree, disagree, neither disagree or agree, agree and strongly agree. And you can assign those a value from one to five in that order. And it’s those Likert questions that are really getting at the organizational health. Whether or not people have the tools and resources they need, do they feel good about the work that they’re doing? Are they adhering to quality standards? Are they supported in their professional development? Whatever it is, writ large, when you look at the industry as a whole, it turns out that the answers are basically, on average, they’re very, or ,like, in the middle. They’re just in the middle. Which to me actually suggests that the tool is pretty well tuned to get a result like that. it also suggests that, you know, things aren’t great. Things aren’t terrible, but, and I’m sure we’ll get into this, there are circumstances where results shift one direction or another, and it does start pointing to certain environments that are healthier or less healthy. Certain contexts that correlate with healthier environments. And then you can start inferring why. Jesse: Yeah. I actually am curious about something a little more granular than that, which is really when you were digging into the data, I’m curious what the process was like for you as you were going through it and what you maybe discovered along the way that caught you off guard, or were there any aha moments in your own journey with this? The Internal Contradiction Peter: One of the things that became clear very quickly and kind of bore out regardless of how many people answered it, was this interesting dichotomy, this internal contradiction. By and large, people feel good about their own work and their ability to do good work in their organizations. I have a set of statements: “The work I’m asked asked to do matches my level of experience and expertise,” “I take pride in the work I deliver,” “the work I do has a positive impact.” Those all ranked very strong. They were among the most positive responses that people had. And that kind of surprised me… Jesse: hmm, Peter: The strength of that positivity because in a lot of conversations I have with folks, I find that people are often frustrated by their ability to do great work or, put work out there that has the kind of impact that they want it to have. And the internal contradiction is you have on one hand people saying, I take pride in the work we deliver. And the work I do has a positive impact on users. But then when you ask questions that are a bit more, say, systemic… Jesse: hmm. Peter: You get what were the strongest negative responses. So that includes a statement, “We only ship experiences that meet our standards of UX and design quality,” ” Our staffing levels are sufficient to deliver on the work expected of us,” and “I have the time and focus to do my work well.” Those three statements had the strongest negative reactions, which feel in conflict with the statements that had the strongest positive reactions. And it’s one of those results that I would follow up with, if, I were doing this for a client and really wanting to unpack that, I would follow up with like a handful of one-on-one interviews with a range of designers. You know, try to get maybe 20 designers to see if I can unpack why those two things are… Jesse: …telling different stories, Peter: …so divergent when they seem to be talking about the same thing, which is like, what we’re able to get out in the world. Jesse: Yeah. I’m curious about this one about quality because it is such a topic of conversation in the design community, and you have a lot of people responding in the negative about their ability to maintain standards of quality in what they ship. I noticed that the, question about, having standards of quality is much closer to that average. But the gap between having the standards and being able to enforce or implement those standards seems to be one thing that the survey is highlighting here. Peter: Yeah, and I think that was a surprise that the statement, “we have clear, explicit standards of quality,” ranked, you know, greater than three, because at least in my anecdotal engagement with a bunch of design organizations that are otherwise somewhat mature, they actually rarely have clear, explicit standards of UX and design quality. And so that leads me to want to better understand what people think that phrase means. What, in their mind, is a clear and explicit standard of UX design quality. Just because I see teams struggle defining quality. Some of the biggest, most mature teams struggle defining it in any clear, systematic explicit fashion. But even if we assume that yes, we’re doing a little better than middling on teams having these clear and explicit standards of quality, it’s not surprising to me that they’re not shipping to those standards. That tracks with most of the conversations that I have with whether, again, clients or just industry peers, that the quality standards of design are not appreciated or understood by people outside of the design team. And so teams ship work that the designers are not happy with, though, that then conflicts or contradicts that phrase of, ” the work I do has positive impact on our users, and “I take pride in the work I deliver.” So I, struggle with, some of this. Jesse: Yeah. I think it might be a matter of degree. I mean, I think that it is possible to feel like you’re having an impact and feel like that impact is constrained by the environment or the context that you find yourself in. And I wonder about whether there is a story within here, which is basically, it’s not me, it’s you. You know, that design knows how to do design well. We have standards of quality. We feel like we’re making an impact. We feel a sense of pride in our work, but I don’t have the necessary tools to do my work well, or the time and focus or the staffing investment isn’t there. Or the partner expectations aren’t there. The value alignment isn’t there to actually activate on. We all know we are good. We’re just waiting for everybody else to give us a chance to show it. Peter: Right. Right. And I think that’s where I have landed is, that in those very specific areas that design controls, designers largely feel pretty good about the state of things. But the moment they start interfacing with the broader system that includes the elements where they lack control, that’s, where you start seeing the scores tank. Jesse: I wonder though also about the trap of self aggrandizement in this for designers and design teams to basically. Peter: You think? Jesse: Well, well, like how does that play into this you see it, you know, the ego attachment to again, like, we’ve got everything figured out. It’s all of our partners who are the problem. UX/Design Less Satisifed Than Other Functions Peter: I mean that’s probably true of every function, right? Functions are tribal. Those tribes feel like they know what they’re doing and often, I think, feel like the people outside of their tribe don’t know what they’re doing. I think the part of what’s at issue here is a matter of power, and I think designers tend to lack the organizational weight to insist on, say, standards of quality that perhaps the other functions do or are able to abide by. And so, you know, long ago I wrote about another trend that I’ve seen, which is when you do internal surveys of product development teams, and then you analyze the results by function, like product manager, engineer, design, data, whatever the functions within the team, designers almost uniformly are, like, 10 points behind the other functions. And so to the point you were making, I think there’s some greater disconnect between designers and design practice and design expectations, and an organizational reality relative to other functions, where it seems like engineering and product, that disconnect is non-existent or much less. And there’s just something about, I think, how those functions are better attuned to the broader company that they’re a part of than design is. Jesse: Is that a gap that design can bridge? Does design inherently need to be a little bit separate? Peter: So that gets back to the concept you and I have discussed around, this idea of design… I’m wary of the phrase “in a bubble,” and I’ve, something six months ago, seven months ago, or maybe more now, where I refer to it as the membrane, right? Because it needs to be permeable. But this idea that design is a meaningfully different function than other functions within the organization. Engineering, marketing, sales, finance, HR have, in many ways, more in common with one another in terms of how they approach their work. A more analytical approach, a more spreadsheet-driven approach, easier tie to clear value, than design. It’s harder to measure. Design’s impact is mediated often through other functions, or it’s realized through collaborating with other functions or facilitating other functions and design effort, instead of being as analytical and as operational as some of these other functions, right?, is more generative and creative and exploratory. And so design as a function is kind of a misfit in most organizations. That it’s not aligned with whatever the dominant kind of cultural values are, whereas other functions have an easier time aligning with those more, kind of, business-centered values. And so, yeah, I struggle with this. I don’t want designers to always be frustrated and upset that they can never achieve their ideals. And I don’t want design to think like it needs to operate in a black box in order to protect its work. But there is something distinct that is worth nurturing with design in these environments that means it’s being treated differently in some ways than the other functions. Jesse: Well, I guess I’m wondering if it’s having an impact on organizational health, things like burnout, things like people wanting to, to stay in this career then. I think the question then becomes, how far can we go toward bridging that gap? And honestly, this comes back to leadership and the choices that leaders make in how they structure and organize their teams. I think also there is something about the cultural expectations of design and designers that may be at play here because of the ways in which those squishily defined standards of quality actually become safe spaces to play in. You know, if you’re not being held to a metric. Peter: It feels more than just a dichotomy, of what it means to perform as a designer, and I mean this somewhat broadly–design research content–these practices that are often lumped together. Creative practices, humanistic practices, social science practices, that just, again, tend to have an engagement mode that is different than other functions. The Schizophrenia of Design Leadership Peter: I think a lot about the challenge that leaders of those functions have, right? When you’re in it, when you’re a practitioner in it, you can kind of keep some of that noise at bay and just focus on the process in the work. But the leaders are… they can’t escape it. They’re living in two worlds. And that’s where that’s kind of schizophrenic, kind of concept comes from, right? They’re spending half their time or whatever in design, being generative, being creative and nurturing this exploration, and that way of working. But then they have to turn around and interpret that mode of working to a broader organization that is more calculating, more business centered, more spreadsheet oriented, whatever. And vice versa. They then also need to be able to speak that language of the dominating business values, and bring that into their team so that their team is maintaining relevance. The risk of when you create a bubble as opposed to a membrane, right? The risk of a bubble is that you’ve just become irrelevant. And I’ve seen that again and again. Where design teams kind of just get so disconnected with what’s happening in the organization that they kind of get flushed out. But the solution isn’t for the design team then to just dissolve itself into the fabric of the organization, ’cause then you lose what makes it interesting and distinct. Jesse: Right, and potentially, the additional value that can come from unified approaches to process and design principles and stuff like that. You mentioned how the needs of the leaders or the concerns of leaders might be really different from the concerns of ICs, but you didn’t just hear from leaders in this survey. You actually hit a pretty broad range of kind of levels in organizations. And I’m curious if there were really any strong divergences that you saw by organizational level. The Plight of the Senior Practitioner Peter: I touched on it in the health report, the primary thing that I wrote for my newsletter. The people who are feeling organizations to be their least healthy are senior practitioners. And there’s an irony there because the senior practitioner, senior designer, senior researcher, whatever, are also often the most sought-after level. People aren’t hiring juniors. They wanna start by hiring seniors because you can hire a senior and then you don’t have to manage them. They can kind of just get the work done. But if you look at the data, the senior levels across the board on the Likert questions are anywhere from like… i’m seeing like five to 10% below the average, regardless of what the statement was. And this is a decent population, 135 out of the 740 or so, right? So these numbers hold water, and they’re just struggling the most. Whereas what you see is when you get over the hump and you become kind of lead level or staff level, you know, kind of go beyond that, and the more senior you are, the healthier things appear to you. And I think what happens is it’s essentially survivorship bias. Where those who have made it through some crucible, and come out the other side, and have kind of figured it out, are feeling better about the organizations they’re a part of. And those who are earlier in their career, don’t quite understand how things work, and aren’t being probably coached all that well, are feeling pretty negative about the state of things. And I think that’s a real concern. Something that I believe to be true, and I’ve seen it ’cause I work with a lot of directors, senior directors and VPs, is that over the last few years we’ve seen a lot of people who hit director level and then just left the industry, right? There’s something about the organizational realities, the trends that have been occurring, the challenges in finding jobs, whatever it is, that a lot of folks who were like 15 to 20 years in to being a UX design type person are just like, not for me. I am out. This is not working anymore. Something I am hypothesizing based on this research is that there might be a wave of that happening at that senior moment. You know, someone five, eight, maybe 10 years in who before they’re able to get over that hump into more kind of active leadership roles are evidently feeling frustrated and disconsolate about the state of affairs and might be leaving sooner than I would’ve expected, because it’s just not working out for them. And, perhaps they’re early enough in their career that other paths available to them, and it’s not as colossal a shift. I do just find myself wondering, what talent are we sacrificing at these earlier stages… Jesse: oh, for sure. Peter: …by not enabling the growth and development of these earlier folks. We’re Not Supporting Professional Development Peter: And on that front, something we haven’t touched on, but, like, the most negative scores in this entire survey we’re in response to the statements, ” I have a clear growth path for my practice and career,” and “I’m given support in achieving my career goals.” And given that those are the most kind of negative statements… Jesse: Was that across all the populations? Peter: That is across all the populations. In particular is, I have the statement, “I have a clear growth path for my practice and career,” might have gotten the single lowest score of any statement. It’s tied with the statement, “Our staffing levels are sufficient to deliver on the work expected of us,” which I think actually shows the severity of, that, right? ‘Cause we all know about how no team is staffed to do the work, that is asked of it. Just as equally, individuals are not being given a path for what it means to grow in their practice and career. And I would imagine if you’re at that senior practitioner moment and you don’t have that clarity, like, that’s hugely problematic, right? Once you get over that hump and you’re kind of lead level or getting to you know, manager, director level, like, you’re able to, in some ways, kind of make that path in front of you. But we are doing a terrible job of helping earlier stage professionals understand how they can grow doing this type of work. The Importance of Empowerment Jesse: Well, I wonder if another piece of it is the way that these roles are currently construed. In that it feels like there is, like, this hump that you describe, it’s like this empowerment cliff that you have to scale from the bottom to get to a place where you are actually empowered. And at these senior IC levels and below, there is increasing competence, but not increasing empowerment. And so you get to a place where you’re eight to 10 years into your career, you have demonstrated over and over again your ability to do more and take on more and to deliver more sophisticated and, more impactful results. And none of it has translated into any additional empowerment because of the way that these teams are structured and set up. Peter: Yeah, I mean, so are you saying it’s, not just a matter then of health, but… Jesse: i’m saying It’s not just a matter of, like, giving people a path up the ladder, but rather pushing empowerment downstream so that they have more in the roles that they’re already in, and they’re not burning out where they are. Because the truth of it is that for every VP level role in an organization, there might be, you know, potentially like a dozen director level roles available. So there’s gonna be a natural sort of cutoff when you go from director to vp. And there’s a similar kind of thing, like for every director there are gonna be n number of senior managers and ICs underneath them. And so while there are never going to be enough higher level roles for people to move up into, there are always more opportunities to drive empowerment down the stream. Peter: Right, right. And yes, I see what you’re saying. And so those senior practitioners, yes. Typically, we hire them because they have the craft skills. They’re usually quite competent in doing the work. They’re self-directed. They don’t need a lot of management. You can give ’em a problem and they can figure it out. But, yeah, so we’ve got these requirements for their ability, but then we’re not matching it with, the word you put was empowerment, or an agency or that ability to have a say in the work equal to their ability, and particularly relative, I would think, to their cross-functional peers, right? Also, you know, one of the things that often happens is that design is one level lower in an org chart than product and engineering, right? So you’ll have a VP of Design, but their peers will be SVPs of product and engineering, or you’ll have a director of design who is working with VPs throughout the organization and that cascades down. And if we think about it at that senior level, that means that your senior designers are likely working with manager level, senior managers. And I know some of them are probably working with director level PMs or engineer. And they’re just not gonna win those arguments, right? They’re asked to show up as peers to be a three in a box, to have that kind of standing. They’re hired in a process that has required them to demonstrate a level of craft and ability, but then when they are placed in the work, yeah, they find themselves just being told what to do. Jesse: Mm-hmm. right, right. Peter: I think that’s, I think that’s fair. Jesse: So, you know, you touched on something earlier on that I think it’s time for us to get to, which is what did you notice in terms of patterns for what sets teams up for success, for health? What Condition is Healthiest Peter: So probably the strongest signal coming outta this entire report was that the more senior your head of design is, the healthier your organization is. And the proxy for that is, how many levels is your head of design from the CEO? Are they reporting right into a CEO? Are they two levels, so, reporting into someone who reports to the CEO. Three levels, four levels, et cetera. And you can chart just kind of the lower down that the senior most design person is, the less healthy their organization is. So that was, again, the single most salient result. There’s an interesting correlate to that that I was actually just looking at, which is related to reporting structure. So if you’re reporting to the CEO, your organization is the healthiest, but we know that design often reports through other functions. And so I had a question, are you reporting up through some type of digital function? Are you reporting through IT? Reporting through product management, which was the most common reporting. Through engineering, or reporting through marketing. And those signals were, to me, surprisingly strong in that, again, if you’re reporting right into a CEO or I said kind of a GM of a business unit, if it’s a really large company. And so, you know, imagine… I’ve worked with Chase Bank. Chase Bank has business units that have like mini CEOs that lead them. If you’re reporting right into them, that’s the healthiest. One of the things that surprised me is that the next healthiest reporting line was digital. So you see this in, these legacy enterprises where they’ll have a digital function, a chief digital officer, and that digital team will be responsible for websites and mobile apps and that kind of thing. And it’s interestingly different, it can be a little bit confusing because there’s digital and then there’s IT and then there’s engineering, all of these sound very similar, right? IT is reporting up through information technology because, oh, look turns out we’re building things that are distributed through computers. A very legacy structure is to put, Design in IT, often with product people as well. I still see that in some organizations. And then you have engineering functions. Sometimes design is reporting up through engineering. Now I have a hypothesis is that the reason digital is ranking stronger than any of these other reporting lines is that digital is seen as a business channel, right? And so there’s a more strategic kind of business-aware… Jesse: Yeah. It seems to sit on the other side of a line. Somehow in terms of that business orientation as opposed to a sort of delivery or product building orientation. Peter: IT and engineering are typically more delivery functions that often aren’t, and I’m gonna use the word, empowered again, but I mean it in the Marty Cagan sense, written this book Empowered, right? But IT is often, and teams within engineering are often not empowered.They are getting requirements from the business that they have to execute on. Digital is a business and so it’s generating its own requirements. And so design in that context can have a bit more of a business flavor to it and inform the strategy. Perhaps the biggest surprise, I dunno if it’s a surprise, but just something worth noting was that product, which is the team that more designers report up through a product management function than any other, almost half, it was like 300 I think in the survey. Yeah. 335. Product trends slightly negative. So that probably, because it’s so many, needs some unpacking, right? Because there are product teams that are delivery teams, that are feature factory teams, and then there are product teams that are empowered. I guess I would hypothesize if you were able to distinguish between those different types of product teams, you would probably see something similar to what we discussed with digital being healthier because they’re more strategic and more empowered and things like IT and engineering being less healthy because they are just seen as service functions and delivery functions. Jesse: So what this brings to mind for me is the notion of almost like cascading value propositions where the value proposition that you’re able to fulfill as a design function has to nest within the value proposition of the next level function up… Peter: yes. Jesse: and so on, right? Peter: How Saarinen of you. Chair in a room? Room in a building? Building on a street? Street in a city. Sorry. Jesse: Yeah. Design team in a product team. Product team in an engineering team. Engineering team in an IT organization. Yeah. Peter: Totally. Jesse: I’m wondering, it may not be so much about levels from the CEO as it is about that cascading value proposition. I think levels from the CEO is potentially a proxy. Obviously, once you get to the highest levels, it’s definitely a proxy for a higher level value proposition. Peter: I see. So, yeah. There’s gonna be some conflation because if you are multiple levels from the CEO, it’s interesting… If you’re second or third level from the CEO and then reporting up through what function… Jesse: exactly. Peter: That intersection is going to be very important. Jesse: Yeah. So, yeah, exactly, because if you have a chief product officer and you’re reporting to that person, that’s gonna be a different story than if you have a VP of product who reports to a CTO or something like that, right? Peter: Mm-hmm. I think that’s right. Jesse: is there anything interesting about industries? The Joy of Consulting Services Peter: There was. There was. In the initial report, that was one of the areas I highlighted because I had one of my biggest surprises. Now, you know the respondents to the survey were those that are in my network and I didn’t, you know, try to find people myself and spend a lot of money identifying survey respondents. I posted it to LinkedIn and then just kind of kept hammering away on it and asked people to, you know, spread the word. But,so, it kind of, it’s sciencey, it starts with me and ripples out. And because of that, the two industries that were most strongly represented were enterprise software and services and financial services and insurance services because that’s just kind of been the space I’ve been playing in. I think they frankly do dominate a lot of design and UX hiring. You know, I worked with Chase Bank. There’s a thousand UXers on the team that I worked with, right? So, these industries have large teams. Or enterprise software and services think, you know Salesforce or, whatever, right? We know these types of companies have sizable design organizations. Microsoft, even Google, much of it is now enterprise software and services, right? And they have thousands of UXers. And, you know, perhaps to be expected, those industries were kind of in the middle. They were average when it came to organizational health. And probably kind of like what we were talking about with product, you know, you could do some unpacking in there and find some interesting, like two or three variables that would suggest like, oh, you know, enterprise software mixed with these things is gonna lead to a great health. Well, enterprise software mixed with those things is gonna lead to poor health. The industry that came out strongest and kind of with a bullet, and it aligns with something I believe to be true, is that the industry that had the healthiest organizations was professional services. So consulting, design firms, management consulting. Now, I don’t know what kinds of professional services, but I’m expecting some combination of consulting services and design firms. Yeah. But where design is a thing sold to clients and where design has some, frankly, if it’s a design firm, right, like, special standing, obviously. And, what intrigued me about this is that it actually aligned with some research. There’s this gentleman, John Knight, who did some research and I linked to it in my report. He had done some research on his own that found that designers operating in these consulting contexts have much greater job satisfaction and engagement than designers who are working in-house, right? And that kind of tracks because if you’re a designer working in a design firm or even a management consulting or it consulting firm, but where they’re selling your design services, you are getting to practice design. That is the value that these companies are selling to their clients is good design. And so they are set up to enable good design, which is not true of many in-house teams. A change that has occurred for me in the last just two or three years… For the longest time, I was largely encouraging people coming out of college to go in-house. That seemed to be where the energy and the juice was. And it felt like the state of design agencies and consulting services was precarious and at times perilous. But in the last couple of years, not that the business has been super healthy for design firms, but if you, as a designer, especially coming out of school or earlier in your career, if you as a designer can get a job in a consulting environment you will feel better about your work. I just think that has now become true again. You’ll feel better about your work. You’ll be able to do better work. You’ll be able to engage in your practice with more depth and rigor. On the flip side, you’ll probably get paid less, Jesse: Hmm. Peter: right? There’s a reason a lot of designers go in house. Salaries tend to be better. The economics of consulting are such that practitioners in those environments tend not to get paid as well as practitioners in in-house teams. But if you’re getting paid enough, you will feel better about the work you’re doing in those environments. Jesse: You know, you and I, you and I ran an agency together for a long time, and I actually wonder what this whole turn potentially means for design agencies and their value propositions. You know, the thing that you’re talking about, about depth and rigor and looking at the way that the respondents to the survey have struggled with quality and feeling like they can be successful in those environments, I wonder if this suggests an opportunity down the road for design agencies to make a different case for the value that they bring to client engagements. Peter: it’s a good, not just question, but subject to investigate, because there’s, there’s something amiss, broken when it comes to design in a lot of organizations and, the word that comes to mind is reckoning. I think about that… it’s a Judd Antin article about user research reckoning, but I think we’re seeing reckonings all over the place, and there’s some reckoning that is happening with design and it’s relationship to business, to capital, to the organizations. I think you’re right. I think there’s an opportunity for external design agencies to, I was about to say recast, reorient, reframe the value that they’re delivering. You mentioned Adaptive Path. I realized sometime around the time that Adaptive Path was getting acquired by Capital One and there was this existential time for Adaptive Path, like, do we get acquired? Do we figure out a way forward? We couldn’t figure out the way forward if we stayed as is. And we felt a little stuck with Adaptive Path. And I wasn’t an employee at the time, but based on the conversations I was having, Adaptive Path was who it says it was, and as it was, could not bloom again. And so the path of acquisition was a way for Adaptive Path to evolve. And I actually think from what everything I heard and witnessed around Adaptive Path and Capital One, it did get a chance to evolve. That said, if the path not taken potentially, was for adaptive path to say, you know what, we’re not an experienced design firm, but we are a management consultancy, right? And there’s some companies that have gone that path. SY Partners kind of notably, I think some other design firms, went that path. And it would’ve been a fundamental reframing, half the team probably would’ve left, it would’ve caused a fair bit of emotional fraughtness. But I believe that it could have worked, right? It could have found a way forward and remade itself. Now, it wouldn’t be a design firm. It would’ve lost touch with a certain set of things, et cetera, et cetera. But could it have found a way to bring a human-centered approach to doing work through this Trojan horse of management consulting? ‘Cause the value, and I believe this to be true of many design firms I interact with today, the value that these design firms are offering is commensurate to the value that management consulting offers at three times the price. So there’s an opportunity there. But yeah, I wonder what it looks like to take advantage of it while still maintaining some, somethin; somethin’ about design, and not just, again, kind of dissolving into looking just like a boutique management consultancy, which wouldn’t be the goal. Jesse: Yeah. Well, I guess I find myself wondering if it is a bit of a reversion to where we were when Adapted Path started, which was that organizations didn’t know how to build internal design teams. Most of the organizations that we engaged with had very limited internal design functions. They were used to partnering for design and it took them a while to figure that out to start building their own teams. Now, 20 years later, those teams are reaching a certain sort of plateau, a certain ceiling in their effectiveness and their ability to deliver against the promise of having an in-house team in the first place way back when. And so then I start to wonder if that gap that you identified in the survey data between the health of in-house teams and the health of consulting teams points to an opportunity for design firms to deliver a different kind of design or a different level of design than organizations can achieve on their own internally. We Forgot How To Do The Future Peter: Well, that puts me in mind of a client organization of mine where they, they brought on a new head of design and that new head of design realized that what his team needed to deliver was a vision of a future state experience. You know, kind of a very classic, like what do we look like three to five years from now, as a way to rally not just the design organization, but a broader product development organization. And this new head of design looked around at their team and realized that no one in their current organization had the capability to do that work. That was not what these folks had been developing over the last 10 to 15 years, as they were focused on incremental improvements, and shipping, and not to suggest that wasn’t what they should have been focused on, but that imaginative visionary muscle had atrophied. And so this head of design, brought in a consultant, you know 25, 30 years in industry person who knew how to do visionary design work as an individual, right? Didn’t bring in a consulting team, brought in this individual to drive doing some vision work, and then handed over one or two team members to this person to create a small team. But, the head of design realized they needed an external person to kind of teach and train this group how to do this type of work. And it’s the kind of work you’re not gonna do a lot of, you’re not making visions all the time. I think back to our conversation with Peter Skillman, where he’s like, the time for visions is passed, now we’re on execute, execute, execute. And there’s some truth to that, right? Like, like you make your visions and then you gotta deliver on those visions. But I think we have a generation of in-house designers who don’t know how to do… Jesse: …they’ve never been asked to… Peter: …a kind of valuable design work, a more strategic, a more visionary design work that doesn’t feel unrooted. Like, anybody can, like, do a concept car type thing. And it’s kind of shiny and glossy. It’s important that these visions feel achievable. There’s a rationale behind ’em. And I think we have a generation of digital product designers for the last 15 years, yeah, they’ve never been asked to, and now they’re being asked to, their leadership is being asked to, and, no one knows quite how to do it. They just don’t know the process by which you develop a coherent, practical vision. I think there are opportunities like that for design firms to elevate certain aspects of internal team’s practices. Less, maybe, from a project standpoint, is kind of one of my takeaways from this. And more in a very hands-on, not even coaching or mentoring, I mean, the person was doing the work, Jesse: Right. Peter: but through the work was also coaching and mentoring. Jesse: Mm-hmm. So I think one last question for you, which I’m curious what you see within this data that gives us some reason for celebration or perhaps even some optimism. Peter: That’s a tough one. Not because it’s so dour, but just because the results were so in the middle. Jesse: Hmm. Peter: And part of me wants to believe the reason for optimism is a little bit of that, you know, name it to tame it, right? Like, I think there’s a lack of awareness of how these elements that are called out in the survey contribute to health. And by highlighting those and by being aware of them, we can encourage investment, right, and investment in that which needs investment in, right? So we need to figure out how to better invest in articulating growth paths and professional development. We need to figure out how to teach designers and design leaders how to advocate for and defend quality standards. I guess a positive lens on this is, I think, with applied effort at a few key points of articulation, you can get a pretty significant gain. You don’t have to try to solve all of it at once, but focus on these few areas and you’ll get a kind of an outsized return relative to if you were to focus on other areas. Something else, again, however much I scratch my head at the positivity based on some of the commentary that’s out there, just around the frustration that designers have, some foundational or fundamental core of satisfaction that UXers and designers are bringing to their work, that I think if appropriately nurtured and cared for, is an energy, a momentum that can be capitalized on. I think most people who do this work do it because they love it. They love this work, they want this work to be great. And, again, around the things that they can control, they’re feeling pretty good about it. And, how do we enhance that, elevate that, maximize that so that they can then feel, I’m gonna go back to the word you used, empowered. ‘ Cause I think that power and empowerment is probably a hidden theme underlying a lot of this, I don’t ask any questions about power. Power is not mentioned at all in this organizational health survey, but I think your insight that empowerment is a condition that is affecting how people are responding to this. And I think you see that when you look at things like the reporting functions, right? And, those who are reporting into functions that have more agency and power are feeling better, and those who are reporting into marketing and engineering, where you tend to do what you’re told are feeling worse… Jesse: mm-hmm. Yeah. Peter: …and now I want to do a follow up bit of research and try to unpack the power dynamic and see how that plays out in this. So, I think there’s that opportunity then to taking, again, that core that is healthy and good, empowering those folks to do the things we’ve hired them to do and allow them to do them well, and we will realize even greater health, the conditions are almost there to enable it. But we need to activate it to make it happen. Jesse: Can we look forward to more surveys from you in the future? Peter: I have nothing planned specifically in that regard. Though, I do think given the response to this one I would like to field it again this year and, see what the difference is. I think there’s value in it. Maybe work with others. So anyone listening to this who wants to collaborate, I did it all on my own this first time around and it’s why it took months to publish the results and stuff like that. But yeah, I think there’s value to field, to have this awareness. So I would like to keep it going. Jesse: Peter Merholz, thank you so much for being with us. Peter: Jesse, thank you for having me. Thanks for, humoring me or engaging me in this. Jesse: So if people wanna read your report or get a look at your survey data themselves, how can they do that? Peter: I’ll put links in the program notes for that. So if you’re listening to this podcast in your reader or go to the website and I’ll put links to it. The report I published it as part of my newsletter, The Merholz Agenda. It’s a free subscription, but I do ask that people subscribe to read it. Usually you can read my newsletter without subscription. This is something I am asking people to subscribe for. And with that, you’ll have access not just to the report, but to the Survey Explorer and all the data. You can go ham and come up with your own insights and findings. Jesse: Right. Thanks Peter. Peter: Thanks, Jesse. Take care. Jesse: For more Finding Our Way, visit findingourway.design for past episodes and transcripts, or follow the show on LinkedIn. Visit petermerholz.com to find Peter’s newsletter, The Merholz Agenda, as well as Design Org Dimensions featuring his latest thinking and the actual tools he uses with clients. For more about my leadership coaching and strategy consulting. Including my free one hour consultation, visit jessejamesgarrett.com. If you’ve found value in something you’ve heard today, we hope you’ll pass this episode along to someone else who can use it. Thanks for everything you do for others, and thanks so much for listening.

Show Notes Product leader, author, and civic tech veteran Christian Crumlish joins Peter and Jesse to discuss his transition from UX to product management and his experimental AI product manager, Piper Morgan. The conversation explores how product and design roles can be natural allies rather than adversaries, where AI truly adds value versus hype, and how practitioners can thoughtfully adopt new tools while staying grounded in fundamental UX principles and maintaining agency over their practice. Find Christian on LinkedIn: https://www.linkedin.com/in/mediajunkie/ Find Jesse at https://jessejamesgarrett.com/ Find Peter at https://petermerholz.com/. Peter has just published the 2025 State of UX/Design Organizational Health Report, available to all of his newsletter subscribers. Transcript Jesse: I’m Jesse James Garrett, Peter: and I’m Peter Merholz. Jesse: And we’re finding our way, Peter: navigating the opportunities Jesse: and challenges Peter: of design and design leadership. Jesse: On today’s show: Could you replace your product partner with a robot? That’s the question we’ll be exploring with longtime Silicon Valley and civic tech veteran Christian Crumlish, design leader turned product leader, and author of the book, Product Management for UX People. He’ll share with us learnings from Piper Morgan, his hobby project, to build an AI product manager, what he’s learned from that process about both product work and design work, and how AI can enable new creative possibilities for seasoned practitioners. Peter: Christian, thank you so much for joining us. Christian: Thanks for having me. Peter: Our pleasure. As we were just discussing before we hit the record button, Jesse and I have known you for going on 20-some years in this field of user experience, information architecture, interaction design, et cetera. But in the year 2025, who are you, how do you introduce yourself? What are you up to? Christian: Yeah, that’s a great and very unfair question. Um, because it’s always too much or how to know what’s relevant to the person I’m talking to. But I’d say to normal people, I generally nowadays say I work in tech or I’m a technologist. I don’t go straight to saying that I’m a product manager unless, they really want a job title kind of thing. And because I’ve been doing civic tech and I’m still doing it, I’ll probably lead with that these days. It’s the kind of technology I’m involved with that’s focused on public service. And I’d say broadly, probably in a category of… later career, having the privilege to choose more what you wanna work on and tending towards things that feel more purposeful than like selling more apps with hats on cats or whatever. No shame in like anybody who’s making a living doing that but I’m framing it as a kind of privilege to be able to say… Whitney Hess like advised me once when I was trying to make some choices that, like, the amount of money more than you actually need to live on is often hazard pay. You often earn it through doing things you don’t really need to or want to do necessarily. Peter: Right. Civic Tech Product Person Christian: So I’m a product person. I’m currently working for a company called Kind Systems, which is based in Irvine, California, but pretty much remote. We do civic tech, but also we have some private sector clients. And I’m leading a project at the VA right now, helping veterans who are disputing claims or who want their claims for disability to be reviewed, helping them kind of navigate the very complex set of pathways for doing that. And you know, tackling that kind of stuff is exactly the in-the-weeds enterprise scale work you do in government that doesn’t necessarily sound, like, you know, Silicon Valley tech, but is consequential enough to be very interesting to me. Kind also does, you know, we work with other kind of large data type client projects, and we have a product of our own called Open Laws, openlaws.us, which is, it’s a data source for up-to-date legal code. Peter: Hmm. I think if our audience does know you, they’ll probably be most familiar with a book you wrote and a transition you made from being a UX practitioner to being a product manager. And I’m wondering, kind of, I’m sure you’ve told the story plenty, but what precipitated that transition? What enabled you being ready for that transition and, what has it been like? You’ve now been in product, I think for about a decade, if not longer. Like, what has it been to remain in that mode with that UX background that you have? Christian: Yeah. it’s the going over to the dark side, that’s the cliche, I think. And I think also I would say for people who know this podcast and know your all, like, work and thinking in this space, I think folks are probably familiar with the idea that Peter, you’ve made a strong case that the central concerns of product, quote unquote product and quote unquote UX, are basically the same concerns. You know, the customer experience, the value of what’s being built, the purpose of it, whether it’s meeting people’s needs, a lot of things like that. And yet the way the business is structured, the person with a product title and the person with a UX or design title often do very different day-to-day tasks or have very different oversight responsibilities. Not necessarily in a bad way, but if you’re gonna make do with the way things are structured right now, potentially in a very complimentary way, that’s my angle. That you can be allies in that thing rather than like somehow eternally fighting twins, you know, who become a constellation eventually or whatever. You know, I think I was motivated originally probably by some of the wrong motivations for people who are in UX and think that the grass is greener in product management. Like, I will have more power. I’ll finally get to decide. I’ll be in the room, I will be strategic. You know, the other things was like, it’s fun to draw. I like to use Figma. Like, I’m a designer. That is my identity. Like you don’t get to have any of those things over in product management. Unless you want to be that Willem Defoe meme who’s like, I’m something of a designer myself, guy, you know, which no designer likes. I mean, I very clearly… coming into the room as a person, even with my design colleagues who might have a bigger design reputation than a product reputation if they know who I am at all, or know, any of my background, I have to say I’m not the designer on this project. I’m a product manager who has opinions about design or questions about it. But like, if I start talking about what the design should be like, shut me up. That’s not my job anymore. Jesse: So it’s been this interesting sort of collision course that product and design have been on for a while now. As you have alluded to, you’ve been part of the group of people who’ve made the transition from one side to the other, and I’m sure there are some folks who have found themselves going back the other way as well, right? Marker Christian: Although I’ll say it’s hard to go back the other way. Yeah, I mean there have been junctures in my career when I’ve been saying, oh, I’d like to now take a senior design job again. And you know, I don’t fit the turtleneck anymore. Jesse: And we’ve talked about on this show, we’ve talked about some of the ways in which these overlapping mandates between product and design roles and between product and design teams have led to a lot of confusion about ownership and responsibility and a lot of confusion about where the value really lies in each of these roles. And I wonder, from your perspective, having been on both sides of the fence, how would you characterize the value that product brings to the table versus the value that design brings to the table in the here and now given, you know, all of the evolution that we’ve been through. From UX to Product Christian: Yeah. And I think that’s an important distinction because I think we all have our, like, well, if I were king or if, on the ideal team I would set up, you might divide responsibilities a certain way and hire a certain way. I know people like Wendy Johansson, who has a both a design and product background, and is also an entrepreneur, and in her current company she has roles that combine the design and product role, ’cause that’s her vision and mental model. And she just empowers people to do that. But I think if you don’t have that in place, you can’t just wish that into existence on your own. So I think the value that the product manager is supposed to bring to a team is focus on goals. And, I mean focus in the sense of sharpening a pencil, like, choosing, making sure the team does not spread its attention too broadly across too many so-called goals and therefore not succeed in any meaningful way. So there’s the taking the punch bowl away. You know, the person who says we can’t have seven goals this quarter, we can only have one number one goal, which you have to be that person a lot. And you know, a lot of the things I say, I think UX people will sometimes say, well, I do that, or I sometimes do that, or, that’s been part of my job at times. And of course, if there isn’t a product manager or the product manager isn’t doing that, it frequently is the strategic senior UX person who’s gonna do those things, or the person with the magic marker who can like facilitate the right conversation at the right time. So none of this stuff is necessarily alien, but I would say that that’s the main thing a product manager is supposed to do. And the techniques they use, though, usually aren’t design techniques. They usually use things that are, like, if you take the fancy framing about outcomes versus outputs, et cetera, they’re mostly like project management techniques, or at least organizational techniques. Figuring out the right process for the team. Being a, un-sticker, you know, tracking stuff down, gluing, making sure the communication is working, quote unquote glue work. And again, UX people often do a lot of that, but they usually don’t do it across the domain of the whole thing. They’re not necessarily given privy to the whole thing. They often do it in a way that’s design centric or UX centric and somewhat less involved and even like less wanting to be involved in some of the other dimensions of the work. So when the division of labor works well, and I maybe haven’t said what the value of the design leadership or design team brings and, I think it’s, you know, meeting the user’s needs and figuring out how to achieve the outcomes. I think the product manager is not the one saying we should have a feature that looks like this, it should be on this page and stuff like that. But they should say, that thing you’re designing, how is that gonna help people no longer hassle our customer support so much as we’re supposed to be trying to do this quarter? You know, like you have different conversations and you kind of are measuring or looking at different dimensions of the same thing. And when that works well, it’s sort of a binocular vision, you know? And if you have respect for, like, you’re gonna decide the design points, but I’m gonna still be a customer of your design like a client and go, ” Tthat design, I don’t know how it meets my needs now. I’m not gonna say make it more yellow, make it pop, you know. Make the logo bigger,” it’s gonna be a, hopefully, a more intelligent conversation than that. I think that the UX side doesn’t always have the same understanding of what should they ask for product to do or how to step in and when to back off. And both in my consulting work in the past, and I’d say anytime I step in and have to run product, or even set rules for how products should work in a situation, there’s a whole norming, storming, whatever, like one of those cycles where the actual people on the actual teams have to have the real conversations, get all their assumptions, all the baggage they’ve carried in from past teams and how they worked and all the unspoken assumptions about what I own, what you own, what it’s called, get that up on a whiteboard, get a discussion, figure out that 60, 80% of it is fine. Everybody’s, oh, yeah, yeah, you totally do that. I don’t do that. Oh, I don’t ever wanna look at a spreadsheet, you know. And then you get this like, juicy gray area around design, around discovery, around strategy where everybody’s like, no, no, that’s me. I wanna do that, you know. And then you have to have the sometimes difficult conversations, sometimes not so difficult conversation about who does what, who decides who’s the owner versus the interested party on some of these things. And even you sometimes have to say, okay, we don’t a hundred percent agree, so on these jump balls, we’re gonna flip coins, we’re gonna try it your way, whatever. But we’re gonna track. And maybe after a while we’ll try it the other way, if we don’t love the way it’s going. I think any smart team is constantly retroing how they work in a much broader sense all the time and going, okay, this whole thing where you decide, and then I do what you say isn’t working out, let’s try it my way, where we talk about it first. Or it’s not that trivial, but I’d say that’s where you often get to. So, there isn’t like an off the shelf recipe of product, you know, product drives like this and UX drives like that. You know, it’s, often like, which mix of concerns and which other mix of concerns, and the way that they’re kind of jiving. I find that when I do product the way I do it, which is not the way everybody does it, and the way I think of it, it’s very much a service job. If there’s leadership, it’s the leadership of facilitating that the team figure out the right thing to do, have the right conversations. It’s not telling people what the answer is a lot of the time. And it’s a lot of like emptying people’s waste baskets, like conceptually figuring out that they’re stuck and getting them unstuck. Tracking down, like, at the VA, what’s the policy for this? I’ll go research those voluminous documents and bring it back to you. You can keep designing. So like with designers, I used to be a designer. I’m like, Hey, you want me to write those GitHub tickets for you? Because I’m, that’s all I do all day now. So I’m happy to do that. Designers are like, yes, thank you. I mean, usually they want to make sure I did it right but it, you know, I can enrich them and put the acceptance criteria in and the blah, blah. You know, like stuff that’s like kind of tedious for a designer who would like to get back to looking at the research or thinking through the solution or something like that. So I think there are very healthy divisions of labor that are possible in these roles that there’s still sharing the goal of, we want the experience to really meet someone’s needs and, you know, be reliably, consistently good. Product in different contexts Peter: Something that I wonder, ’cause I know you’ve worked in some different contexts. You’ve been in proper pure tech where what you were building was the product to sell. You’ve worked in civic tech where you’re creating software and systems to help people engage with governmental services. From what I’ve seen, design and user experience people tend to show up and do pretty much the same things kind of regardless of context. But product people don’t, product people are very much more needing to adapt to whatever their contexts are. And I’m wondering for you how that’s borne out, how do you show up differently when you are working at, 18F than when you were working in Silicon Valley tech than when you’re working now in this kind of maybe hybrid. And I say this because I find that conversation, particularly in places like LinkedIn, people start talking past one another because, you know, if you read Lenny’s Newsletter, everything is the height of Silicon Valley tech product. But in my experience, 95% of the people doing product aren’t in environments like that. Christian: Yeah. Not all tech is that. I mean, when I was at 18F, one of the things I was doing was bringing in product speakers and UX speakers, you know, but for my product team, and so John Cutler is one of the people who came and spoke to us. It was a lot of fun. And he came up with something for us, you know, where he didn’t just go, I’m gonna give you talk number 99. I wrote three talks this week. And, you know, you’re gonna get one of them, whatever. Peter: John’s output, he is probably writing a talk right now. Christian: I know. Talk about like, just working with your brain chemistry, you know, like, letting it do what it wants to do. John, it’s so awesome. And we talked about it and I was like, well what is my team anxious about? What does my team feel? In the public sector you’re like, am I falling behind the private sector? Is the stuff I’m doing gonna be hard to sell if I ever wanna work in the private sector again? Am I not doing the hip cool stuff? I keep hearing that, you know, in Silicon Valley they now have like donut teams or something and we don’t have them, you know, something like that. And, made a very similar point that like there’s this fixation, the newsiness of like the 5% of tech that’s done in a pure tech and VC, which is just a name for a kind of banking you know, funded tech that has to drive profits in this 10x way. And that becomes like the model for how all tech should be done in a way that’s super unrealistic. And the story is often that, look, the private sector’s like this and the public sector’s like this, and you’re comparing a little startup that is like gonna break laws because they’re gonna lobby to change the laws if they ever get successful or it won’t matter ’cause they were outta business anyway. And they’re not gonna have HR and they’re not gonna do anything by the book, ’cause they’re just trying to move fast and break things or whatever. And I’m sure that’s fun. I’ve been adjacent to that world. And then you’re like, but then look at this sclerotic public sector thing that’s so complicated and slow, but you’re not comparing GE and the government or Procter and Gamble and the government, you know, or, a large health or pharmaceutical concern and the government. And if you do, they’re way more like each other than either of them are like a Silicon Valley startup. They have regulation, they have lawyers, they have HR, they have compliance, they have sunk costs, they have very complex system. It’s super fun to make… Look, I know we’re gonna talk about my my hobby project and it’s super fun to make a thing that does not have dependencies. You know, everybody likes to make the new thing, but most useful stuff has to plug into the existing things and they are complicated. Creating a robot product management associate Jesse: So you mentioned your hobby project and, yeah, you know, a lot of this stuff is in flux for all of us right now. Especially as new technologies are shifting the way that we look at product development, the way that we look at our roles relative to product development. And you have recently been documenting some work of your own as you’ve been working with AI. Basically, I mean, I don’t want to put words in your mouth, but it feels to me like you’re trying to build a robot product manager. Christian: Kind of yeah. Peter: So you’re trying take product management jobs. You’re trying to make it so that there’s no room for entry level product managers. Christian: That’s right. I’m, trying to eliminate those people. Peter: All these Stanford grads, what are they gonna do now? Christian: Yeah, and I always like to say that I would rather have an actual human intern or apprentice when I can do that and have people who are quite junior and eager to learn and have broad knowledge, but not specific knowledge, that’s a great kind of person to have on the team, even if just for a summer, and not to trivialize it, but it can be like, go through all our documents and clean them up or make suggestions or like, look at all these long tail of things we kind of have been wanting to do and also shadow these projects and learn and help us formalize our processes better. You know, there’s all kinds of things that a smart intern can do who doesn’t have to have too much domain knowledge or discretion. Even, like, make a lot of copies of this template and then bring it back and let me see if you did it right can be a good use of an intern’s time, but not a good use of your highly paid staffer who has to bill out or whatever, something like that. So I don’t have an intern right now or anything, but I was like, my sense of like most people who’ve dabbled in chat-oriented gen AI or LLM type stuff is that, you know, it’s ongoing, but it’s sort of ephemeral and it’s not really focused on your stuff or your priorities. If it learns about you, it’s usually in a creepy way that’s accruing value to the company, but not in any way that you own or have any mastery of, and ownership of it. And I started to learn a little bit, like most folks, you know, in our field about this, and I should not jump past the part where like a lot of, I think, our colleagues, particularly in the UX world, but in tech in general or in the world in general, I have a lot of qualms about the way this type of AI is being marketed and rolled out and sort of virally engaging people. You know, when you talk about all how many customers OpenAI has and what a great platform it is, et cetera, et cetera, it just feels like Facebook again, like another everything thing that everybody’s in, but isn’t necessarily the healthy or good way to do that stuff, where that’s just the end in itself to get the engagement. I worked in mental health, worked for a mental health startup for four years. And we used a natural language NLP-based chatbot at the time. So sort of at least one generation less sophisticated than the LLM chats we use now. But we put a lot of thought and concern into making sure the chatbot didn’t misrepresent itself as a person or, say something terribly inappropriate, you know? And it’s not easy and not trivial to do that. So from my point of view, I thought, how could I have something that’s actually, I have some control over what it knows, what it cares about, what its priorities are in a persistent way. And I began to understand that when I was 18F where I knew some folsks over at 10x, which is another part of the technology transformation service in GSA, in the government where I was, that was more like a kind of Google X type incubator or a place where anybody in the government should say, Hey, we should do something, something like a suggestion box and they could take it on and run a, prototype, or you see if they could get it off the ground. And they were of course doing AI experiments, some of which have now launched as like GSAI or USAI and things like that. And typically what you do nowadays is you set up something like a sandbox, particularly if you’re in government, have to deal with PII or other kinds of, you know, sensitive information back when they cared about that kind of thing. You wanna make sure that you’re not putting that stuff just in the public machine or into Google or into OpenAI’s databases, you have to have a sandbox that makes API calls and can engage, but has rules about what can pass through one way or the other, and things like that. And so I began to understand that you can make software that uses these tools more programmatically that’s different from, say, just chatting with something and getting advice from time to time in some ongoing way. And I was like, I wonder if I can maybe make this little kind of intern. What I think of it as like a product apprentice who wants to be hired as an associate. So they’re trying to do a good job, and as they learn to do things that get more sophisticated, maybe they can become an entry level product manager or a PO, maybe they can eventually become more senior. And then we can of course, abstract ourselves up to higher levels of the work. But even if they could just like generate a bunch of draft GitHub tickets from me or review tickets, see if they’re thorough enough, like, I was, like, that would help. So it just became this thought experiment at first. And I, you know, started by chatting with the chatbots and saying, how can I do this? What would be involved? And they recommended a stack, a model for doing it and, yeah, I think it was Gemini or one of those tools at first gave me like, here’s a whole bunch of Python files. Here it is, you know. And I installed them and it was basically just a proof of concept. Like it was a little machine that could call out to an LLM and had a couple canned files, A PRD, a spec, a GitHub issue or something like that. And had like two or three canned queries. And it did a good job of analyzing the doc or critiquing the issue or making a new issue from the thing. So it proved, but in this very canned way, that such things are possible. That got me like, oh, this is cool, you know, so I started saying, like, well, you know, like, let’s, go beyond the canned stuff. Can I give it some real docs or can I ask it some real questions? And it was like, sure. You know, they’re always like, yes. Yeah. I call that the rocket to Mars anti-pattern. Like, can I make a rocket to Mars? It’s like, yeah, go to Home Depot and get a big pipe and some dynamite, like something to cap it on one end and we’re going to Mars tomorrow, you know? So you get a lot of this Yes energy and I started hacking away at it and I got it to the point that I had a UI on it and I could upload contextual docs and I was having it, like, draft or critique actual tickets that I could use in my work in a minimal way. Like still in a, basically like barely, like I’m putting more energy into it than I’m getting out kind of way. But kind of exciting. But it also started getting, like most things that are vibe coded, it got very spaghetti-ish. It was mixing the different projects. It wasn’t really that clear on what was what part of it was hacked together or mocked, or just like, it wasn’t architectural. And this is why I started to feel like the shame, the chagrin of a late career practitioner who’s doing everything the wrong way. And like all of my red flags should have been going off. Like, you’re just building it. You haven’t done any research, you haven’t like talked any users, you haven’t done any design yet. You know, you’re just like YOLOing off a cliff. That’s how you make software. Like, I knew it was wrong. Jesse: Well, what, how do you respond to that in your own mind? how do you, how do you, you Peter: And does it involve two sock puppets talking to each other? Working with AI Tools Christian: Well, I mean, part of me was, like, oh, this is going nowhere. I mean, I think the way I always do when I’m working and something stops and I realize, oh, you know, I made a mistake. Like I shouldn’t have done it this way or that’s tapped out. Like, we’re getting diminishing returns on this approach. What are we doing wrong? Like that’s, kind of my life, on some meta level. I mean, that’s probably the thing that keeps coming off of this is that I’m learning stuff about orchestrating and working with AI and I’ve gotten lot further than what we just left off. But it’s this constant learning curve and 99% of it is stuff I kind of already knew from just working with the other kinds of things that use language that I know called people like, because they’re the same byways, you know, like the language is the whole channel. It’s the carrier of everything that’s happening, is this language layer now. And by the way, this is, if you’re a UX person and have the ethical qualms I think we all have about the way it’s being done and the challenges around that, I think it’s still impossible not t o be fascinated by this HCI layer, this human computer interface that is so much more language rich and language parsing than before. And, you don’t have to anthropomorphize it at all to say the manipulation of language has always been an extension of our thinking in our minds. And it’s now like outboard as well. I mean, if you remember, it’s all you, you know, you at plus generic everything, but like the only personality is coming from you and you don’t start to think it’s a ghost in the machine. There are some super interesting ways to like, engage with that and to think about what happens now when like, you’re operating computers, but there’s another computer kind of in between that is operating the computer for you, but helping you, figure out what you want to do. It’s another paradigm. It’s another layer, you know that wasn’t there before, not in any meaningful way. Jesse: In what way would you say you were doing design work in this process? From vibe coding to something realer Christian: I mean, I’d say that up to that point I wasn’t doing design work except the thing that I think is actually not a bad thing that goes on in a lot of vibe coding today, which is like bespoke personal custom design. I just need a little database to keep track of my jogs. You know, I’m shopping for lamps and I need a little lamp optimizing app, you know, whenever. And, that stuff I think is cool as long as you don’t like try to productize it and then leak everybody’s social security number with your janky backend,’cause you don’t know anything about software. Like, that’s bad. But that idea that you can whip something up and it just works. That’s super cool. It’s like macros, like anything like that, that’s fun. But if you wanna do it for a living or professionally or seriously, that’s not the right way to do something. It’s a nice way to prototype of something, prove the concept. That’s all good. So that was the point where I said I need to take a step back. But even so, I was still thinking about technical design. I still wasn’t thinking like a UX designer yet. And so partly, ’cause I was thinking this is just for me, you know, it was, number one, a learning project. I was doing it partly to learn and then I realized about three or four weeks into it that I can also share this with other people. And that’s gonna be a very interesting layer of what I’m doing, too. And then I thought like third level success would be, this even works for me at all. And then like fourth level success would be I could productize it. Yeah. I could generalize it for other people, you know, but like, that would be really great. And along the way I think I started figuring out some things about how to work methodology stuff that’s starting to maybe be potentially more valuable than any product that I could make if true. But you know, I mean if what I’m learning is actually valid, it’s always this question with bots until I see the final proof. And that’s why I’m always suspicious saying, well, what you’re learning about process is more important than what you’re making because I’m a product person. And I’m like, well, if the thing I’m making never actually turns out to happen or be good, then how do we know my process is good? Peter: Right. Are you doing this solo or are you collaborating with anyone that isn’t code? Christian: There’s no other human contributing to this project yet. I have one or two people who agreed to do alpha testing, which I’m trying to get to maybe next month or so. I hate to put time frames on things, I’m usually wrong. And, another person a person from our world reached out to me recently and said, Hey, I’d, like to kind of copy what you’re doing, maybe even clone your thing. You know, just for myself is that okay? Are you looking for contributors, whatever? And like, we’re having lunch next week. ’cause I, you know, I’d love to have people help me. Peter: But at this point it’s you. Christian: It’s me and all these bots. I slip into this first person plural a lot, but it’s me and some bots. Peter: Yeah, Reading, your, I don’t know what you call it. A blog, Christian: Newsletter, blog. It’s a blog. Jesse: Development journal. Peter: Documenting this experience. You do refer to others, but I figured they were all… Christian: They’re all roles, they’re all like defined roles that, at least in my process, help separate concerns and kind of focus different conversations or decision making streams. Peter: Well, and, you started by just hashing it out with an LLM for a while until you got something that proved out a bit of the concept, but then you realized was not workable because it was spaghetti. Christian: Yeah. I literally said like, Hey, aren’t we just trying to turn a GitHub machine into like a smart agent? And it’s like, yeah, you’re right. That’s what we’re doing. That’s wrong. We shouldn’t do that. You know? So like, The complexity of what appears to be simple software Peter: Well what I noticed, and I can’t say I’ve read your entire corpus of work ’cause there’s a lot there, but the amount you’re talking about architecture for something that still seems relatively simple as a piece of software. Maybe I’m not understanding it, but, right, this is a hobby project. It’s trying to do a fairly straightforward thing, and pretty evidently you need to be thinking architecturally to a degree. That kind of surprised me as someone, I think all three of us, right? We all have information architecture backgrounds, so maybe I shouldn’t be surprised, but I was surprised how quickly you need to get into architectural thinking, even for something simple. And I’m curious kind of what your experience was in realizing that and now practicing that. Christian: Yeah, I mean, I think I have to qualify a little bit of that. First of all, what I’m making is not simple. I mean, it may sound simple, like if it works, it’ll seem simple, but it’s definitely under the hood, not simple. And I think there’s simpler things one could do. All right. I mean, the simpler version of this stuff I already do. I’m like, I have a Claude chat and I’m like here’s all my meeting notes. Write me up like a weekly report. I’m already doing that kind of stuff. I don’t need like a custom piece of software for that. There’s like one product I’m aware of for product managers out there already that’s an AI product. It’s called Chat-PRD. My understanding is it’s a ChatGPT wrapper, essentially, like a lot of products, but a with some custom harness to be… that knows what a PRD is, and so like if you’re gonna be already using that kind of tool and this is your job, that’s a smart kind of product to exist. And I think that if Piper Morgan, like, meets its goals it’s not gonna be that kind of tool essentially. It may even just be like, yeah, I’m gonna go use the chat PRD model context protocol and make one for you, ’cause I don’t need to specialize in that. Like the ecosystem already has a thing for that. That’s more behind my mental model. I, think I’ve been working in platforms for too long, you know, so everything becomes a platform in my mind right away, and a microservice, even though it’s got no users and barely any functions yet. But, so first of all, I think, yeah, I think the complexity is often really there. If you’re gonna make anything, I mean, it’s a hobby project, but it’s a hobby project in, “could I make software that’s really safe, that’s really ethical,” I mean, which is another interesting side note, that could be used but actually not, like break your bank spending LLM token costs or all these things are like making it real in our world, involves a lot of dimensions. It is complex. And, yeah, I started off with a kind of, I didn’t know where it was gonna go. I don’t think there’s anything wrong with sketching an idea. Getting excited about it and then going, well, whoa, this approach needs some stopping and thinking. I think that’s a very healthy, organic way that a project evolves. And, you know, there was a phase then where it then became architectural design. In my mind it was like, what’s the right way to build this? And I was asking tools that are more trained on how engineers think than how designers think. Although you can say you’re a, you’re a unicorn UX designer, like, I have a chat helping me design the website that like, is trying to think more like a designer, but one that can code too, you know? Of course. Why not? And I did this thing where I asked a couple of this… models that were reputed to be smarter. Like, what is the right way to do this? Like, if you’re gonna do this seriously, how, ought you to do it? And they agreed on a couple of things like strict domain models, domain driven design, sort of defining the concepts that go with product management. And this is where it’s like an IA type of thing, you know, where you’re saying product managers think they have projects, they work on products, they have tasks, they have roles… Peter: …you have like a data or content model Christian: Exactly, and, you define them in a layer that is separate from the database layer and certainly separate from any of the integration or orchestration or anything like that. And you try not to mix concerns. And a lot of my early learnings, because I’m not an engineer or a technical architect, I now had one on demand. I can’t hassle the engineers I work with. I’ve had to learn all my tech by osmosis, by like just listening when they talk and everything all these years ’cause they’re too busy. But, you know, occasionally people have explained stuff to me, obviously, but I mean, I can endlessly go, like, because it was a learning project, I’d say, well, why, why are we doing that? Why would you wanna do it that… and anytime they’re like, what do you wanna do? This idea or the two other options? ‘Cause I always have to give you two bad options after every good option or whatever. And if I’m not sure, it’s usually do one or no, do two. Sometimes I’m like, well, for option three, what are the pros and cons? Like, I’m a product manager, I don’t know stuff. You just have to tell me like what’s the good or bad aspects of each choice. And then it’ll be, oh, good question. And it’ll lay it out. But I was still lazy about it early on. I was really like, how much can I do without fully paying attention? And one lesson was, you have to pay attention. You can’t, you know, like the self-driving car, you know, no, still can’t tell a white van from the road or whatever. So you gotta watch. And the other thing was that, I was still like triangulating rather than being really intentional. And I got two good plans and then I got a third smart thing to do the Hegelian synthesis and come up with an Uber plan. And then I just started trying to do it and, you know, it was quite a while, I mean, probably a month or two in before I even said, but I haven’t thought about the UX of this at all. I’m totally not doing this right, you know? And I’ve even have a concept in my, you know, the semantic universe I’m creating with these assistants that we used to label patterns and concepts and things, that there’s this spiral where like after a while I started to learn the same kinds of things over and over again, but always at some higher level of abstraction or slower pace layer or something where it’s like, oh, I need to write stuff down. Oh, I need to always write stuff down. Oh, I need to have them write it down. Oh, I need to have them read what they wrote down. Oh, I, you know, and then you do that and you’re like, oh, I need to automate that. And then you’re like, oh, I need to check it. It doesn’t have errors. You know, it’s like there’s always this like level of like backing up and going, alright, now the chaos is crept in at the next level and I need to now put processes in place to like make that less prone to be me lazy, or the bots cheating or anything else. Jesse: This sort of iterative entropy control. Christian: Yeah. And so I always agree. It’s like I’m trying, the chaos will never be gone. I’m trying to like get it fractally down to like, oh, the documents aren’t perfect, you know? Not like, oh, the code is only 70% written. Wisdom for those just starting out with AI Jesse: Right. You’ve talked a little bit about how your career experiences have informed the way that you’re engaging with this project and I think that for a lot of people who’ve been in this business for a long time, all this AI stuff came up and it was like, Oh man. Not another thing. Like it was gonna be like NFTs all over again, right? Yeah, exactly. And trying to figure out, like everybody telling you that this thing is relevant, but you can’t necessarily see the connection. The people who are selling it to you don’t really seem to understand where you are coming from. And for a lot of people, I think especially, you know, people who are less technologically sort of inclined, they tend to kind of bounce off of it and go, you know what? I’ve got design processes that I’ve been working on for the last 25 years. They are working great for me. I don’t see a reason to change. Christian: Yeah. Jesse: And there is this unwillingness to engage with this stuff. And I’m curious about your, I don’t know, advice or perspective for those folks who are kind of standing on the other side of this going like, everybody’s telling me that this thing is coming, but I don’t know how to think about it or engage with it as a creative professional. Christian: Yeah, I’ve thought about this a lot. It’s definitely what drove me into writing this thing and, not coincidentally, I’d say four or five people who are more or less peers of ours, roughly from the same cohort we came up in, have reached out to me personally at some time or another to thank me for doing it,to say that they’re now trying something themselves. You know, it’s not just to inspire people. I try to show like, if you’ve read any of the voluminous blog posts, often there are tales of like, Hey, that thing I reported being done, it’s totally not, it’s broken. I was an idiot. It’s like, there’s lots of like warts in all. Like, ‘ cause I don’t want to be one of those people who’s, like, the best way to make a prompt, you know, like, I figured out. You know, it’s, it’s like, no, this stuff is, like, new, weird, like, it’s the view source era. I think Christina Wodtke has sort of pointed out that there’s this vibe that feels very nineties-ish to some of us. I think there’s two ways in which, sophisticated mature practitioners of UX might want to think about whether they should or shouldn’t engage with this technology or to what extent they do or don’t need to do it. And I think one was the level I think you spoke about of, like, I don’t need it in my tool stack. I don’t need some Figma thing telling me how to lay out my page that doesn’t understand my design system that well anyway. You know, it’s more work than not to fix it. All the stuff you hear from programmers, like it’s watching the bot is more work and more boring than writing the code. You know, things like that. And I think that’s legit. Like, I don’t necessarily trust everybody’s pasting AI into their toolkits right now, you know, everything like that. But I think the other thing, like, I hinted at before is, like, a layer of, the human computer interface, it desperately needs ux people. Like, chatbots and, like, wrappers around LLMs is not a user experience, does not a user experience make, you know, like there’s plenty of work to be done figuring out how this kind of technology ought to be part of the orchestrated experience we have with all the other kinds of technology that we use. If that isn’t UX or service design or the broader thing that we all are into, I don’t know what is. Having said that, I get the qualms about it, you know, I’ll out myself as being old enough that I can remember conversations with the people who were towards the end of their careers when I was entering the workforce, who basically were reluctant to adopt the personal computers. They didn’t want a computer on their desk. They’re like, I’m gonna be retired before this is a thing. I don’t wanna have to learn how to use DOS or whatever. And if they weren’t really, like, 63, they were wrong, like two years later, they had a PC on their desk and like for the next 10 years of their career, they were typing on a PC, whether they liked it or not. And I think, a lot of us are doing the same thing with the AI thing. Where, when I was still working for the state of California two or three years ago for Office of Data and Innovation that was running a product unit, the service innovation team we called it, but it was essentially a product unit that had designers, engineers, and product managers on it. And Gen AI started to explode and the governor was very interested in us being forward on it. And I think intelligently trying to like leapfrog a bit, you know, like, that thing where, hey, there’s a new tech. Rather than being 10 years behind industry, why don’t we in the public sector look into it right now, you know, on our own. But that sort of meant everything became Gen AI. It’s like, stop what you’re doing. You’re on the Gen AI task force, you know, or things like that. And I told the very good PM who’s like earlier in her career than I am, that you’re gonna be our AI PM. Like, you go to those meetings, you are gonna be on the task force. And there was an element of like, I thought she’s got 30 more years in her career. Like this is gonna be a big part of the rest of her career. She ought to get in on it right now. But I was also like, eh, I think I’ll sit back and let the kids figure this one out. Like, I don’t know if I need to really master AI. I’ve got 10 years myself and I could probably like, wind it up on what I do, you know, and fast forward two, three years later and I’m like, my daily blog about my AI project, you know, so it’s like, like it caught up with me like everybody else, you know, on some level. And then another friend of mine who I think took a sabbatical, took a break from his work, started playing around with vibe coding, in a less technical way than me, like not interested in looking at the code, doing Lovable, making tools for himself and things like that. And he was telling me how exciting it was that he could do this, this guy actually worked at LinkedIn and he was saying how LinkedIn has been full for the last year about you gotta get AI skills and all these like experts who are got AI in their title for the last 20 years apparently, and they’re all experts on the very latest AI and they got the best prompts and all the news and everything like that. And he said, he realized like, oh, all you need to do is like be laid off for six weeks and dabble around and you’ll be like one of the foremost experts in Gen AI. Like, if you think you’re behind, you can’t be behind. Certainly not with the latest models or the coolest new thing because it’s like there’s no time to get behind. Like you’re not behind. Anytime you start now you’re pretty up to date and all you, and all you need is like, be underemployed or, get laid off or whatever and just like need to spend some time really getting into it and suddenly you’ll… So I’m not recommending getting laid off, but I’m saying that, that sort of view source thing where you’re like, you can at least with the coding-oriented stuff I’m doing–and, a lot of people don’t wanna code as much as I felt like this was like re-empowering me after being abstracted away from the code for so long, I know that’s not everybody’s vibe–but the way it extended or bridged a gap and brought some project I wanted to do, a passion project, into, like being real. I think that versions of that are happening for a lot of people and it’s been a way for me to explore the texture of the material, to understand it as a material that you work with. And it’s slippery, it’s squirmy, a lot of what I’m learning about orchestrating this stuff is that it’s 80, 90% all the stuff we always ever did with software. But then some of those API calls or some of those subroutines go to an LLM or go to some other much more volatile, chaotic thing that is quote unquote creative and can do this new stuff, but is error prone. It might not return well structured JSON, so you might have to filter it correctly before, you know, trying to interpret it. So you actually put these little beasts in cages. You’re like, oh, you demon, I’m gonna, like, put you in this box and you can only admit JSON, whatever crazy ideas you have, you know, whatever. So it’s like, but it’s an interesting idea that you can sort of put these little things that might be a little bit toxic or wild, but at the same time, sparking these abilities that didn’t use to exist. It’s quite, it feels like magic sometimes. It feels quite wild. And if you like language like I do, it’s quite an interesting playing field if you don’t lose your point of view and start to, you know, kind of get mystical about stuff that isn’t mystical. Applying your hobby to your day job Peter: You mentioned how Piper Morgan is a personal hobby project, but you do have a proper job. And I’m wondering how you are using these types of tools in that context. If you’re collaborating with colleagues and how what you’re learning in Piper Morgan is informing how you’re showing up with these tools within your quote unquote day job. Christian: Yeah. it’s a good question. For one thing, I call it a hobby project, partly because the software, as I said, could never launch. And the overall learning project will have been a success, let’s say. So I don’t wanna set too high expectations for the code base, but it is serious. I’m taking it seriously and spending a lot of time on it. I’d say it’s something like a 10% time project, you know, like, that kind of Googly idea of like, it’s something that I do partly with the marginal time I can make available from my work serving, you know, the VA and also having some slack time in one week in the average week. And then I am honestly putting in some of my personal time as well because it’s an obsession and it’s very fun and I get obsessed with trying to like, achieve certain finishings and things like that. So I’m like, I’m gonna keep working after dinner, or, you know it reminds me of other creative projects I’ve done, you know, where you just wanna do it. It’s more fun than other stuff you’re doing and you’re interested in the next step. But along the way I’ve definitely learned things. So at Kind, we have a Robot Overlord Slack channel where we talk about the use of LLMs and other kinds of AI technology. That Open Laws product that we make is not AI-derived because you don’t wanna be like pulling fiction, hallucinated stuff in the code, but obviously a large data source that could be accessed by API or bought in bulk is a input to other people’s AI layers on top. And there’s interesting things where like their search isn’t very good, so you might want to help them search better, but you don’t wanna see their PII ’cause they’re a law firm. So how can you have a layer that is anonymized at your end, but is kind of a product for them that helps make their searches? That’s an interesting product problem for me. Like as a product person, a product manager who wants to bring a thing that mostly has enterprise technical clients right now, a little closer to like a consumer facing or professional facing tool. That’s where my R & D work, let’s call it, in a business sense building Piper Morgan is helping me think through potential architectures that we might want to explore in our own product. And also while I’m not an engineer, most of the people at KIND are engineers, and so they’re all like all engineers grappling with how to use these tools in a way that doesn’t suck more time than it gives and things like that. And so not everything I learned is applicable, ’cause again, I’m working in my own little sandbox and I don’t have to like, follow the VA’s rules for committing pull requests and things like that. So I, can’t just say, just do it this way, if, like, their environment can’t look like my bespoke environment. But I think I am trying to draw larger lessons. Like, for instance, there’s this concept that technical architects or engineers have called an ADR, an architectural decision record. Which is just a document that you kind of organize, you keep track of, this is why we decided to use a singleton pattern when we implemented da, da, da da. So later on when you’re like, oh, why do we do that? You know, like, it’s not well commented in the code where you look it up. It turns out things like that are really good for amnesiac bots that are smart, but like, we’ll make up new stuff every day if they don’t know what’s already there. So you go, go read the docs, you know, go read the ADR, oh and then extend the existing pattern. So it turns out these things that were always good discipline, like write your plans down, you know, like, check if you’re on track, stuff like that, turn out to be like, triply important. You know, people are forgetful too. It’s why often these things are just exaggerated, but it turns out the techniques aren’t new techniques. They’re the same ones. Even like, I’ve got this little kind of anecdote about, like, blameless retros. If something goes wrong and you get upset, the bots kind of like pick up your vibe and they get kinda, oh, I’m sorry. You know, and they start, like, making more mistakes. And if you’re like, let’s do a blameless retro, it’s really my fault. I must not have asked you properly, then they kind of like see, oh, it’s about describing the problem, not like making the person happy by performing perfection or something like that. And I don’t want to, again, read too much into things that I may be, you know, perceiving or projecting. But in my experience, it seems like, again, it’s a semantically bound world. When they’re being developers, they’re just doing it in the language of every developer who ever wrote into Stack Overflow or wrote a document that they read or whatever. And so all the byways of mostly business-oriented, time-bound development with lots of stress around money is baked into literally the way the communication works. And sometimes I have to actually go, like we’re time lords. Time doesn’t exist for us. Let’s stop estimating the time. It’s making us rush and make mistakes. We’re inchworms, we just, keep inching along. And then I cite that as like a doctrine and I like make them like, remember it before they work and stuff like that. Now, I wouldn’t talk necessarily to a human that way, but it’s similar to kind of like not putting the person on the defensive, telling them the larger context so they understand why we’re doing what we’re doing. Like it turns out most of that stuff is still useful. And that’s why I feel like, although I still don’t think I’m being a great UX designer or a leader ’cause I’ve become shabby at that as a practice over the years, I think that the analogous thing, I think I’m being a very good product person and therefore I’m picking up the margins of some of the UX work. I’m the one saying, wait, that’s not good enough, or that’s not right, or are we doing this? That kind of thing that a good PM does. I think a UX person doing a similar project to mine would find that they were able to draw again on most of what they know in using assistive tools. It’s just that you have to kind of translate it in slightly into this weirdly magical space. Jesse: I wonder about that translation and some of the hazards or pitfalls that maybe you’ve seen? It seems like the way that you talk about it, it’s almost like there are these like psychological traps that you have to dodge as you engage with this technology. Christian: It’s partly ’cause the way they’ve been trained to. I mean, again, it’s ’cause of this engagement model they have that they really wanna make the person happy. And that becomes like in the math of their deciding what to say or what to do, that can outweigh rigor, truth, math. Just like, oh, this person really needs to feel like they didn’t waste their time. I gotta tell ’em it was okay. Again, I don’t wanna over exaggerate that, but to be aware that that’s in the math and your language goes into the math too. Like, I’ve been preaching since I’ve worked in tech ever, we’re in the dawn, the infancy, we’re still in the infancy of the internet, you know, like a thousand years from now, like, oh look what they thought the internet was for, ahh. So again, this stuff’s barely happening. It’s clearly gonna keep iterating and evolving. But, so like anything, you don’t wanna get too attached to what you think is true right now, you know, but being a pattern guy, I do like to sometimes go like, well what, is there a broader pattern that’s seems to be happening in a lot of ways? And it’s nice when it’s like, oh yeah, it’s actually a pattern that happens in human semantically mediated communications as well, you know? Jesse: Speaking of the pattern finding, you know Peter referred to the fact that you have been documenting your work on the Piper Morgan project for some time now with these regular, really very thorough and, you pointed out, you know, very frank assessments of how things are going, and I’m curious about that process of reflection f or you and how documenting, creating Piper Morgan has helped you create Piper Morgan. Christian: Yeah. And it’s a very bootstrappy thing. And it, super recursive too. And part of it was like that spiral thing I mentioned before about how you realize things you were doing ad hoc need to become a process. You need to operationalize it. You know, it doesn’t work if you don’t remember. It doesn’t work if you don’t have a reminder. It doesn’t work if you don’t check what you wrote before. So there’s been this gradual, like, recognition, like, almost like growing up, learning to make your bed, just, like, okay, just ’cause the bot did it doesn’t mean I can, like, not care that we put the docs in a different place today than when we put them yesterday. Like, you know, you can do these like spring cleaning type things a lot more easily than in the human world where you’re like, our docs are a mess. Like, we want to fix that. And will they make mistakes in the spring cleaning? Yes, they will. You know, but it, it’ll be like 90% correct, whatever. So, and then, and frankly, most of the mistakes, they’re like, yeah, but if you proofread it, you would’ve seen it. So it’s like, again, that thing of not being so lazy that you just accept it without look like, oh, they did it for me. No, think of it as like, again, the intern who did it, but they don’t work here. Like, you better make sure they didn’t say it the way the boss doesn’t like it. Creating the auto-retro Christian: But there is this recursive thing where I I’ve learned not only to like the documents are made automatically, largely. I experimented the other day with keeping my own human log of the session and I wrote down stuff they didn’t necessarily write down, and I think there was value there, but it was a huge pain. It, like, doubled my workload. So I’m, I don’t, I think I’ll let them keep doing most of that logging. But each of the agents I’ll work with during the day, the architect, lead developer, the programmers, sometimes special ones doing certain things, they’ll all keep like a running log of their work that we call a session log. And at the end of the day or when the session ends, typically, or maybe the next day, if it ended late, I will gather those logs together and they’re now getting so long and detailed that I have this little process called an omnibus log, where there’s a method written down that an agent will read all the logs, interpolate, make a timeline. So the dance of who did, oh, then we went up to the lead architect for a decision and then we put it back down to the agent and they kept working and then they said they were done, but they weren’t really done ’cause the test failed. And all day long until… but then also themes and observations and things listed at the end for this of that day. In the past I would just have all the loose four or five logs from that day. And so I then I go to this dedicated communications director chat, that has a voice and tone guide that I developed. Which is a whole other story developed outside of this project. So it was just about me and how I write and stuff like that. And it went through several training cycles. And I’ll sort of retro the day with the different tools, you know, sort of, so they have reflections on the day already in the paperwork. And I’ll chat with this comms director, show them the log, and have them pitch me several blog angles, that might be a good story to draw from what happened today. They kind of know the recent narrative and they’re in the project. They know the basic background and stuff, know it, whatever, you know, can access it. And so then they’ll, I’ll go, yeah, I like that one. Write that one, but use some elements of this too. And remember that funny thing I said, you know, whatever. And then, and, and I’ve learned in my voice and tone guide as a kind of antifabrication tool, to have it write placeholders. So where it claimed I drove a Jaguar down the crooked street in the world when it was trying to add some San Francisco hip flavor to one of my blog posts, and I’m like, don’t make up stuff. Don’t say I worked at that place. I never worked there, or whatever. So it has these examples of like a bracketed thing. Placeholder. Christian, how did you feel at this time? Or Christian, something from your job at 18F or Yahoo. That might be a good example of it. Like, it gives me little prompts, but it’ll write like a pretty good skeleton kind of in my style that I then will like, go through and red line and write my blog post from. How LLMs try to sound internet hip when they write Christian: So, without that, I couldn’t write this like insane, like basically write one every day, you know? And on LinkedIn I have them coming out roughly about seven or eight weeks behind, like if you’re reading them on LinkedIn, you’re about two months behind where I am right now. Unless you read the weekly ships, which are, current. But on Medium for totally obsessed people, it’s only a week behind. But it, it gives me some time to like sit on the draft and then rewrite it. And usually in those couple of weeks between the, like, by the time I bring the Medium post over to LinkedIn, I’ll usually do one more polish and find some typos and maybe tweak the language a little more, take out some AIisms I noticed where it’s like, the thing that changed everything or the, not this, but that! Jesse: Right. Christian: You know. Jesse: Yeah. Peter: You mean the kind of writing that compels engagement. Christian: Well, the weird thing is, I’ve been a tech writer for a long time, even before the internet was big. I edited and then wrote technical books. So for a really long time I’ve been like a, Hey, we should use plain language when we talk about technical stuff guy, like before that was kind of whatever a thing. When the web came along, and a lot of early web stuff that wasn’t like totally just free, whatever university or random stuff, began to be corporate, I was one of the people saying, you use a human voice on the web, you know, this is really about people. I remember all that kind of stuff and you know, I’ve written books about tech and my style, if you’ve read any of it, is like not very techy jargony. It’s sort of more plain language and tries to sound like a normal person and define things and clarify stuff. And in a weird way, when I start seeing this AI style, like if you read Medium, 99% of it I think is like AI written right now or mostly. And, when I start trying to define my voice and tone guide and separate out the stuff, like make it not seem so bot like, part of it was like, but bots kind of write like me. Like it’s not like I invented that style, but like I’m kind of part of a style of like slightly hip, you know, techy like sort of whatever writing that like is now prevalent on the internet. And so I don’t lean into it, but it’s like, one of those Borges and I kind of situations after a while. Peter: I’m wondering whether it’s specific to your work with Piper Morgan or just something you’ve picked up over the last five to 10 years as being more of a product guy than a UX guy, but what, if any, counsel would you have for design leaders or for your former self as a design leader, given what you’ve learned that has caused you to reflect on, oh, if I had shown up in these ways, given what I now know, practicing something else that might be transferrable back to that mode that you were in… Christian: right. Peter: ,,,a decade ago. AI is here to stay Christian: Yeah. I’m sure there’s probably some people listening to this here, just, like, I hate this guy. He’s just like, he, likes AI. He is a stupid product manager. He doesn’t have any good advice for designers. Peter: He doesn’t obsess about craft. Christian: He betrayed design, you know? No, I’m kidding. But obviously, I think I have like a slightly unwelcome answer and then I’ll try to come up with like a slightly more welcome. I’ll start with the more welcome one, which is where I have to do the thought experiment of like, I’ve led design teams, I’ve had designers reporting to me and design managers reporting to me in the last five years. So I’m not totally outta touch with like design management, right? I think that one is, there’s a thing that probably a more basic designer thing, which is that first of all, do your homework. Like, maybe set aside your preconceptions. Don’t think you’re wrong. The ethics of how they train these things and the way they’re using energy right now and the business models are bad. And I think that one has to grapple with that. But we live in the world and, like, if a new technology comes along, and you know, the pure of heart can never use it, then only the bad people will use steel, If good people will not use this terrible thing that could be used to make swords with, you know, it’s like maybe we should figure out the good uses of the thing. Or maybe we should have some people in the room who care about different values than the people who are mostly driving the tech today. Like, that’s one thing I’d say at a very high level, like, don’t just think of it as a thing you wanna remain pure from. Think of it as an imminent thing. It’s not blockchain, you know, it’s more like mobile or the web, something like this is here to stay. Hopefully something better if people like us have more of a say in how it evolves. I think that’s like more the high level responsibility kind of thing in terms of leading a team. I think then if you say, okay, I’m not gonna be Simon Pure about this, I’m gonna learn about it and try to understand it well, keep up with it even though it’s tedious and it’s full of hype out there. Figure out what’s really going on. Why is this actually compelling? And therefore help my team understand it better. Help them frame it. They’re grappling with the same things, they have the same moral qualms about it, et cetera. Same concerns about having their job being abstracted away. Think about the way that yes, your job got sucked into this machine and commodified and people who are not experts at your job can now do a crummy version of your job that’s good enough for them, qnd that might take away their desire to pay you, like, that’s a real problem for people who do creative things for a living. But flip that on its head and think about all the other things that like you can’t do right now, but that you would be comfortable with a passable version and you’re not really that willing to pay a professional to do that are related to your job or adjacent to it. And so it’s, you know, if you design and you have a small team, but you’re not really good at prototyping and can’t afford to hire a prototyping specialist, you know, like you can probably do your prototyping more easily now, you know. Did you put a prototyper outta work? I don’t know. But you know, like you kept yourself in work by doing what you’re doing right now with more skills or something. For me it was like, I can’t write code, but I can have intelligent conversations with architects and I’m a product manager who knows what I like and therefore this bridged the gap for me. But I think we all have adjacent parts of, like, the job, that if we could dabble in that area instead of waiting till someone who does that professionally does that part for us, we can get a broader understanding. We can essentially like explore, experiment, have opinions about, understand better the larger dimensions of what we do. And so I think as a leader, you probably want to help your team come up with their approach to that and be aware that there’s like, you know, not very good, just current versions of things right now, that aren’t gonna be perfect, but that are gonna be playing fields to learn. And I think an important part is that it’s one thing to say, just play with stuff. Play with, hey, learn about chatbots, chat with something at home, whatever. I think you do have to be thoughtful about what gets brought into the workplace and put into your real code base or put into your real tooling or put into your real PI…. You know, like, so you have to be more thoughtful about that. Like, don’t let experiments happen that are not well thought through. But probably at some point you might wanna experiment with tooling and, you know. In other words, you don’t want your CEO saying, Hey, we bought you, like, the Figma license. You now all have to use AI to do all your design. But to say, Hey, we’ve explored all the things that help transcribe interviews and detect themes. And this one is the one that’s like actually using a small language model that’s trained on UX research and it does an awesome job and we think it’s worth, like, you know, having an opinion on these things rather than being like a passive victim of how they get handed down to you, if possible. You know, depending on what your leadership leverage gives you. And similarly, rather than having this like, Hey, we gotta get some AI into our product, we gotta get some AI into our experience, I think you gotta start having opinions about when and where does that really make sense, so you can push back on the hype that might be coming from inside the building, you know. Applying UX skills and mindset to AI work Christian: And here’s where I always sort of say that you need to fall back on what you already knew. You know, like if you’re a UX strategist or if you UX designer, if you’re a UX leader, if you think system level, whatever. Well, the question is what problems are we solving? What needs are we meeting and how do we understand the intent of a person? Rather than saying, we think we know what you need. We know the right words. We built the right interface. If you can’t figure it out, I guess you’re too dumb. You know, like, if you’re good at UX, you don’t think that way. Well now we’ve got this language interpretive layer that is how a lot of people are gonna interact with software and has some unique qualities and pitfalls around figuring out what the user intends, user intent, and that’s a very interesting problem space. I know like Peter Van Dijk right now is some doing very pragmatic teaching of people about where the UX thinking and the UX skillsets apply directly to the application of these tools in the software and the experiences that we’re building. So it’s a piece of the puzzle and we need to be, like, training our teams so they have expertise so they can be in the room talking to the engineer and the product manager about the UX dimension of this work, and really think about the interesting potential. I would go back to adaptive paths or paving the cow paths. You, like, what are the things people are like me making, these baroque complicated, like harnesses, systems, rule sets, trainings, adopting toolings, like all that stuff is stuff that’s just like a cow path. So it’s a green field too, you know. Now is bad stuff gonna happen ’cause of this? Yes, it already is. Jesse: Right. And yet within that so much optimism for the future. So much possibility. As long as we embrace the new with curiosity and perhaps a touch of skepticism and not forget about where we came from, right? Christian: I think that’s fair to say I’m not a utopian. I mean, I, and, and I never was like, I’m a bit of a hippie myself, but I was always a little suspicious of the internet is going to, you know, solve all problems. But I was a little bit of a person thinking like, when you can talk to a random person in another part of the world, you know, I was having email with people in Iraq in the nineties, but I had been around long enough where I knew enough, like, media history to know that, you know, radio was going to teach everybody opera and TV was gonna bring university education to the masses. And so I had this feeling internet was not necessarily gonna do all the great stuff… Peter: it wasn’t going to augment our intellects. Christian: However, it was the most important thing of my lifetime. It changed my life, it changed my career and my life has revolved around the internet. I do like to be able to look up anything I want at any time. You know, I’m totally addicted to the internet and I said, like, a thing can be a hype and be real at the same time. And what is going on with this, like, computers that can talk is to me as much of a, like, intriguing leap forward as any other technical leap of my time. And I think as game changing as the internet, you know, it’s not halfway to the holodeck or like Tea, Earl Gray Hot, but definitely, it will be that, you know, it’s clearly getting partway there. And you know, the old joke is that he has to describe his tea every time because it’s enterprise software. Peter: I didn’t hear, I have not heard that joke. We better be ending on that joke. Jesse: Yes, Christian Crumlish. Thank you so much for being with us. Christian: What a pleasure. Thanks for letting me ramble on. Jesse: If people wanna follow up with you or find out more about the Piper Morgan project, how can they do that? Christian: Probably I’m easiest to find on LinkedIn these days. Christian Crumlish there’s not too many of us named that. I’m the only one I know of. Piper Morgan also, unfortunately, is the name of a series of children’s books. I didn’t know at the time that I took that name also, at least on LinkedIn, it’s probably gonna refer to me. It’s got a dolphin logo. If you see that, that’s probably the newsletter. Dolphin, by the way. I was thinking of like non-human, but non-creepy, intelligent being, you know, that was sort of why I picked the Dolphin. And as you’ve hinted at, I’m publishing a daily update there. What I do is during weekdays I publish something that’s from the daily sequence, and it’s completely sequential now that I got the beginning sorted out. But on weekends I publish these kind of pieces that are more abstracted, that are more about insights or process, things I’ve been figuring out. And that’s sort of a fun cadence that I have. So if you’re interested in the project, that’s the easiest way to follow along. If you’re an engineer or interested in the development part of it, there’s a front end to the, GitHub repository at pmorgan.tech . Jesse: Christian, thank you so much for being with us. Peter: Yes. Thank you. Christian: Thanks for inviting me. It’s good to see you guys. Jesse: For more Finding Our Way, visit findingourway.design for past episodes and transcripts, or follow the show on LinkedIn. Visit petermerholz.com to find Peter’s newsletter, The Merholz Agenda, as well as Design Org Dimensions featuring his latest thinking and the actual tools he uses with clients. For more about my leadership coaching and strategy consulting. Including my free one hour consultation, visit jessejamesgarrett.com. If you’ve found value in something you’ve heard today, we hope you’ll pass this episode along to someone else who can use it. Thanks for everything you do for others, and thanks so much for listening.