Data User Experience KPIs - The Most Key Takeaways

Sign up for Data Mesh Understanding's free roundtable and introduction programs here: https://landing.datameshunderstanding.com/

Please Rate and Review us on your podcast app of choice!

If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here

Episode list and links to all available episode transcripts here.

Provided as a free resource by Data Mesh Understanding / Scott Hirleman. Get in touch with Scott on LinkedIn if you want to chat data mesh.

Transcript for this episode (link) provided by Starburst. See their Data Mesh Summit recordings here and their great data mesh resource center here. You can download their Data Mesh for Dummies e-book (info gated) here.

Karen Passmore (CEO at Predictive UX) led this discussion with Wannes Rosiers (Product Manager at Raito) and Alice Parker (Data Engineer at DNB). This panel was held in partnership with Data Mesh Learning - you can see a link to the video here: Panel: Data User Experience - An Introduction (Data Mesh Learning and Data Mesh Radio)

Alice's LinkedIn: https://www.linkedin.com/in/aliceparker/

Wannes' LinkedIn: https://www.linkedin.com/in/wannes-rosiers/

Blog post 'The Importance of UI/UX - and why Raito’s first hire was a designer': https://www.raito.io/post/the-importance-of-ui-ux-and-why-raitos-first-hire-was-a-designer

Raito blog: https://www.raito.io/blog

Karen's LinkedIn: https://www.linkedin.com/in/karenpassmore/

Predictive UX: https://www.predictiveux.com/

Some key takeaways from panelist Wannes Rosiers:

1. DUX handles user experience using data for all your users (data producers, data engineers, data analysts, data scientists, report users, ...), hence it is not - and certainly not restricted to - data platform user experience.
2. Data products are typically chained from producer oriented data products up until consumer oriented data products. Downstream product and UX requirements are completely different than Upstream, yet you should be considering all of them. This friction between global UX and local UX is the biggest challenge to scale Data User Experience.
3. Much more than typical digital products, you have no idea how people interact with your data products. We are moving more and more to self-serve analytics, which means that while developing a data product, you don't know what will end up on the screen of your user. You can wireframe fixed dashboards, you can't for certain other data products.
4. As always, accessibility of your data product is a huge part of your user experience. Next to this, it is important to monitor usage to continuously evolve your product. (Not surprisingly the two key elements we work on at Raito)
5. Domain thinking exceeds source thinking in your product. You should abstract away the concept of data sources, users are interested in valid and bounded insights, typically domain-bounded.
6. And last but not least. You need top-level decisions on data product granularities. To make sure you can connect certain data products, they should be interoperable and preferably on the same granular level. Which level to pick is mainly guided by company KPIs, hence you need management support to pick these.

Scott note: I am pretty new to thinking and dealing with UX in data so I took an opportunity to write down some of my own takeaways that may or may not agree with any or all the panelists. Hopefully you'll find them useful.

The most key takeaways from Scott's view and learnings:

1. If data is not usable, is it useful? How many data projects fail simply because no one really made the data usable?
2. UX KPIs are really important because it's such an easy thing to overlook.
3. If you have a bunch of fragmented UXs in your data value chain, your overall UX will suffer. Think about a dish - you might prepare all the ingredients perfectly but if they were all cooked separately, will there be the right flavor harmony? Probably not, it will be disjointed flavors that don't blend well.
4. Sometimes we think about the target business process in user experience so we don't always have to think about directly using a platform - we can think about how things integrate into a business process, a workflow. If I don't have a UI but interacting with data is part of my day-to-day role, that's a user experience.
5. Data projects and data work need three things: business requirements, technical requirements, and user requirements. Many skip the user requirements at their own detriment. The best way to improve user experience is to actually talk to the users about their wants and needs 😅
6. Empathy is crucial to data UX - make sure empathy is baked in to what you do :) stop prioritizing your technical requirements and business requirements over your user requirements.
7. It's very hard to figure out the user experience path between 'I create a data product' and 'user self-service drives an action from the data product'. It can feel a bit underpants gnome-like of step one steal underpants, step two ???, step three profit. So get really in depth talking to your team about what is necessary to actually get to an actionable insight from a data product.
8. UX technical debt, while not insurmountable, will be a bigger hindrance than many expect because the UX is typically intrinsically tied to the underlying implementation. So to improve the UX to a great degree, it can also require improving a lot of the underlying implementation. Definitely not always the case but it's something to watch out for.

A lot of additional takeaways from Scott:

1. User experience is often overlooked in data because historically, most data manipulation or analysis has been performed by experts in their own domain - e.g. a business analyst is an expert in SQL so all they need is SQL or data engineers have handled ingestion and transformation so there hasn't been a great UX.
2. It's easy to get trapped in the idea that a "data user" is just a data consumer but if we aren't designing and creating a good user experience for producers and anyone else involved in dealing with data, we are looking for trouble.
3. Each user persona has different requirements and it's important to not try to design one-size-fits-all experiences because it will probably be one-size-fits-none.
4. User experience is about lowering the bar to working with data to make it actionable, whether for a producer or consumer or anyone else. How do we remove friction from data to action?
5. Data consumer user experience spans a lot of things from finding then accessing then understanding data and then often transforming into their own purposes.
6. Data user experience can be improved at any point in a data project's lifecycle but it's always best to start at the beginning if possible.
7. How can we make it easier/quicker/more reliable/etc. to get to an initial answer on is data valuable?
8. Part of ensuring a good data user experience is enabling the data consumers to easily communicate their expectations/requirements to data producers. It's not just the UX at point of interaction but also the UX to make the data better.
9. It's important to understand focusing on UX isn't about right or wrong, it's about improving the user experience over time.
10. The data producers and consumers are often an afterthought when designing data architecture. The UX is something at best smeared on top to try to make it a little easier to deal with when it should be a crucial aspect of the architecture itself.
11. When many people think about data UX, they think about someone consuming from a dashboard, not all the steps that data had to go along to get into that dashboard.
12. Metadata is crucial to helping people understand the data they are accessing - especially when you consider being able to trust data as part of the user experience. If you can't trust it, you won't use it!
13. It's important to think about UX as a lifecycle because there is a data lifecycle - information is created and then acted upon creating more information of a sort. So we can't think of it as "data is consumed" and that is the end of the line for the business when it comes to broad user experience of data.
14. It's crucial to think about user experience at the data product level but even more so at the informational/question level. How can data be combined from multiple sources? Does UX extend to data interoperability standards even? Yes, but that shouldn't be managed by the UX team necessarily 😅
15. A very big part of UX is just encouraging conversations because then UX ideas will emerge. 'Oh, you're trying to do X, that makes so much sense, let me make this change to make that less painful.' But too often users take what is given and don't pipe up - give them room and encouragement to share feedback.
16. Similarly, open communication seems really integral to getting to a good UX. You need people interviewing the users but users also need to have space to discuss too. UX knowledge transfer shouldn't only happen because someone asked.
17. Information transfer can be really hard, especially if we do it all as pure documentation. Look for ways to enable better sharing of information - lower the friction to creating simple information sharing mechanisms, whether that's sharing the data or the metadata.
18. A huge benefit of data mesh is reliable and scalable data sharing yes but it's also about the art of the possible - 'oh, we have this data, what about XYZ use case, can we do that?' How can we embed curiosity and an ability to explore more in our data systems while still protecting privacy/PII/etc.?
19. It's easy to get myopically focused on the platform UX and not the total data UX. Some of that is cultural too. You want to try to design for it but the platform team can't own developing the culture. But making it loud and clear what would drive more value might help you design a better overall data UX.
20. It's very hard to know just how a user will interact and experience a data product. So have some conversation with potential users and maybe give them a private tutorial so they don't go off the rails. It's hard to embed 'understanding guardrails' into data we share.
21. It's easy to lose sight of cost when considering user experience. How do we make it so we can enable great experiences in a cost-effective way?
22. It would be great to have data usability design patterns and methodologies emerge for creating usable data products but we are VERY early days there.
23. Usability testing is pretty crucial because while they aren't your enemy, the saying "no plan survives contact with the enemy" is still apt with data work in general. Don't design in a vacuum, work on constant communication and information flow.
24. It's crucial to think about UX at the micro and the macro level. If you only think about it at each data product, you are missing the information of the organization and how it all interoperates to tell stories. But it's also pretty easy to focus on the grand platform and then dealing with the data products is a bit of a nightmare. So again, it's not easy :)

Data Mesh Radio is hosted by Scott Hirleman. If you want to connect with Scott, reach out to him on LinkedIn: https://www.linkedin.com/in/scotthirleman/

If you want to learn more and/or join the Data Mesh Learning Community, see here: https://datameshlearning.com/community/

If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here

All music used this episode was found on PixaBay and was created by (including slight edits by Scott Hirleman): Lesfm, MondayHopes, SergeQuadrado, ItsWatR, Lexin_Music, and/or nevesf