Research program

Where do a system's errors travel, and who absorbs the work they create?

I study what happens around consequential automated systems: where errors propagate, who performs the maintenance, how people keep the ability to challenge decisions, and whether systems can repair themselves when the underlying information changes.

My research develops a systems theory of institutional life. I study how technological and administrative systems organize human experience by distributing maintenance, failure, and obligation — and what makes an institution inhabitable rather than merely efficient, given that the people inside it have finite attention, get sick, and depend on each other.

A denial creates appeal work somewhere. An automation removes a task and relocates its exception handling. A queue makes somebody wait. Institutions shape people's practical agency by changing which actions are easy, costly, contestable, or available at all — they coordinate action, but they also set the terms of life inside.

These are instances of one problem. When humans, institutions, and machines become deeply interdependent, capability is usually bought by foreclosing the options of the agents inside the system — their ability to refuse, exit, escalate, or change course. The program's object is the architectures under which that price is not paid. Healthcare, AI governance, digital platforms, workplaces, and public bureaucracies are where I test them.

Where the burden goes

Stress follows load paths. Responsibility follows administrative architecture. Delay accumulates somewhere, and maintenance is displaced rather than eliminated — usually onto whoever has the least standing to refuse it.

I use these engineering concepts as hypotheses about recurring institutional mechanisms: where burdens accumulate, how failures propagate, and what optimization displaces. Institutions run on structural regularities — not deterministic laws, but patterns stable enough to design against — and those patterns decide where obligation, burden, and fragility come to rest.

Institutions answer a small set of questions over and over, mostly without deciding to. I study what happens to responsibility, explanation, and recourse when those answers get built into systems — a program I keep under the working name of a moral physics of complex systems.

I study what changes when a decision that used to end with a person ends in software instead.

  • Who absorbs error?
  • Who performs maintenance?
  • Who waits?
  • What is repair?
  • What is expendable?
  • What gets buffered?
  • What gets optimized?
  • What cannot be allowed to fail?

In a hospital emergency department, patients are called for triage by name over a loudspeaker. For a Deaf patient the mechanism deciding who waits is inaudible, and what it costs is measured in mortality.

At Epic, a misrouted clinical alert could bury a critical lab result — and the risk sat with whichever clinician trusted the queue, not with the system that routed it wrong. At Andwise, the monetization paths most likely to fund growth would have made employers or financial institutions the customer instead of the physician; shutting the company down was the alternative to changing who the product answered to. Different systems, the same question.

From force to obligation.

Trained in biomedical engineering, I extend concepts such as load distribution, failure modes, maintenance, resilience, and safety margins into the study of institutions. My earliest training asked how force moves through matter. I now ask how obligation, burden, uncertainty, and fragility move through institutions. The material changed; the questions kept their shape.

What comes out is a way to evaluate whether institutions are structurally compatible with human limitations and obligations.

A 2011 signal-processing flowchart: acquire two voltage channels, notch-filter at 60 Hz, split into sound and ECG waveforms, normalize and threshold, detect the QRS peak in a decision diamond whose no-branch loops back, derive heart rate, store data, and exit through a stop-acquisition gate that closes the channels.
Biosignals lab, February 2011 — detect the peak, loop back when the test fails, exit through a gate that closes the channels. Reproduced from the original; teammate names removed.

Fifteen years later the diagrams I draw have the same joints. What changed is that the signal is an institution and the terminal gate is somebody's authority to stop.

Healthcare systemsDigital platformsAI and automated decision systemsWorkplaces and professional culturePublic bureaucracies

What a clinician does after the machine answers

In 2012 I helped build a medication-recommendation system for type 2 diabetes, and we measured the thing that now worries me most about deployed models: not whether the recommendation was right, but what happened to the clinician's own judgment once they had seen it.

  • Internists n=2 62% 92%
  • Endocrinologists n=4 68% 76%
  • Familiar with the rules n=4 64% 86%
  • Unfamiliar with the rules n=2 71% 71%
Agreement with the algorithm, before and after the clinician saw its recommendation. Blinded validation, twenty patient data sets per reviewer, six reviewers in total. Percentages are rounded; the subgroups nest rather than partition, so the reviewer counts do not sum to six.

The specialists barely moved. The generalists moved thirty points. And the group that moved least of all was the one that did not know how the algorithm worked — the two endocrinologists unfamiliar with its rules did not shift at all, while the four who understood the rules went from 64% to 86%. Understanding the tool predicted deferring to it.

A recommendation does not sit inertly beside a judgment. It moves it, and it moves different readers by different amounts — six clinicians and twenty data sets each is a course project, not a study, and the subgroup that moved most is four people. It is enough to see the direction and not enough to size it. That is the mechanism the authority-boundary work exists to constrain: a model's output becoming institutional policy by nobody's explicit decision.

Observed evidence what the sources actually show, timestamped and re-checkable Assessment the model's read on whether the evidence has moved — and by how much Institutional authority automatic AUTHORITY BOUNDARY — crossed only by a named person a decision, on the record Preserve Caveat Narrow Expand Escalate Retire Monitor / defer No action
The constraint the measurement above argues for. Evidence may move an assessment on its own; nothing moves an institutional position without a person crossing the boundary and leaving a record of having done it. "No action" is one of the dispositions, because a system where declining to act is not a recordable choice will drift into acting by default.

Jain, K., Patel, K., Rowland, J., Yong, C. DiaMonD (DIAbetes MONitoring and Dosing) System. Wallace H. Coulter Department of Biomedical Engineering, Georgia Tech / Emory University. Advised by Dr. Lawrence Phillips, MD.

A system is inhabitable when the people inside it can refuse, exit, escalate, and change course — and when its errors can actually be found, challenged, and repaired.

— Working definition

One research program, four surfaces. Each carries a different register of the same work.

Working on a problem that fits this program — or one that should? Email is fastest.

[email protected]