The Importance of Explainability in Surgery

Some AI technologies are designed to be completely assistive, right? A parallel would be an overlay, perhaps a guidance overlay in surgery. How important is it to not have explainability or to try to extract some explainability by interrogating the sort of black box nature of it, or does it really matter? If it helps us as surgeons, we should do it. But when an example, let's say we were to train an algorithm to predict cancer or no cancer in a patient where the therapeutic implications are very severe, chemotherapy with high morbidity or radical surgical procedures,. On the one hand, there's, I think, data that a lot of patients would be uncomfortable with that decision being