The Value of Explainability in the AI Ethics Space

I think that people completely mischaracterize the value of explainability and where we should be using explainability in our tool set. Explainability is a little bit of like the odd one out in that trio because monitoring and observability are really something that you do once your model is deployed to production. This has a lot more to it than just understanding which features a model uses, right? Like a credit score, but do we know whether credit scores are fair? You need to be able to segment according to a protected class in order to know whether your model is discriminating against that protected class.