Adaptation and correction
Research
A tool that works changes the person using it. Skills shift, dependencies form, and some alternatives stop being practical — usually without anyone deciding that they should. Three questions follow from that: why some problems persist even when people keep solving them, how today’s solutions change tomorrow’s possibilities, and what helps people retain the capacity to change course.
I study that change where it is easiest to see: institutions that keep relying on people to repair known failures, even when those people cannot safely stop. The question is what that dependence does to the people who provide it — whether repairing everyone else’s failures costs them the capacity to refuse, challenge, or leave, and what evidence would show that cost while the institution’s performance looks good.
Status: an inquiry. The evidence here is a 2012 pilot with six clinicians and three field cases; the study that would test it does not exist yet.
Each case is read against four questions
- Who absorbs the variance?
- Could the affected people refuse?
- Does the institution see what the repair work costs?
- Can evidence of that cost alter the cause?
Questions the research is organized around
| Question | Where it shows up |
|---|---|
| Why does a healthcare technology fail after it solves a real problem? | After the software is installed |
| How would I know I am solving the wrong problem? | Reading the contract before the physician signed it |
| Why does more software not give professionals more authority? | The Competence Physicians Already Have |
| How does an institution hide a known failure behind the people covering for it? | People Are Not the Error-Correcting Layer |
| Why does an innovation that succeeds leave the arrangement unchanged? | Toothless Ethics |
| Who decides what counts as better, and can that decision be changed? | The framework |
| How do I design a system that stays open to correction? | The framework |
Field observations
Each case shows a different kind of correction failure: the institution could not recognize the problem, could not correct it, or would not.
| Case | Mechanism | Who absorbs the cost | Which diagnostic questions | Kind of correction failure |
|---|---|---|---|---|
| Emergency department loudspeaker | Patients are called for triage by name over a loudspeaker. A Deaf patient cannot hear the call, so they cannot tell when their turn has come. | The patient; triage delays compound while the patient has no way to detect or contest the queue. |
| Cannot recognize |
| Epic: a misrouted clinical alert | A misrouted alert could bury a critical lab result. | Whichever clinician trusted the queue; nobody in the routing system answered for it. |
| Cannot correct |
| Andwise: monetization | The monetization paths most likely to fund growth would have made employers or financial institutions the customer. | The physician, the party the product answered to — preserved by shutting the company down. |
| Will not correct, and refused |
Field observation: how recommendations shift clinician judgment
In a 2012 pilot of clinical decision support for type 2 diabetes at Georgia Tech and Emory, clinicians recorded medication recommendations, then saw an algorithm’s recommendation and could revise their own.
- Internists n=2 62% 92% change +30 pts
- Familiar with the rules n=4 64% 86% change +22 pts
- Endocrinologists n=4 68% 76% change +8 pts
- Unfamiliar with the rules n=2 71% 71% change +0 pts
The comparisons cannot separate specialty from familiarity: both internists knew the rules, and both reviewers unfamiliar with them were endocrinologists.
These figures measure agreement, not correctness; they cannot show whether clinicians deferred to a wrong answer. The framework requires accountable human review before automated outputs become institutional policy.
The pilot measured agreement, not why it moved. “How to Design for Cognitive Scarcity” takes up the conditions under which review happens at all.
Jain, K., Patel, K., Rowland, J., Yong, C. DiaMonD (DIAbetes MONitoring and Dosing) System. Wallace H. Coulter Department of Biomedical Engineering, Georgia Tech / Emory University. Advised by Lawrence Phillips, MD.
Where this work is published and built
- Essays: The Crumple Zone, named for Madeleine Clare Elish’s “moral crumple zone” (2019), documents the work people do to compensate for institutional failures and offers ways to identify those demands.
- Proposed standards: Ethotechnics specifies how systems should record evidence, limit authority, and support challenges and corrections. The standards include schemas and audit requirements.
- Software: NextConsensus (commercially offered), Ambit (experimental), and Refract (live, open source) implement parts of these mechanisms. Current work describes what each project does and its stage of development.
The aim is not to make institutions better at accommodating people. It is to make them more accountable for accommodating people — and to give the people doing the accommodating a say in how that happens.
Revised