[silence]
2026-02-26 · standard-panel · Facilitated by Nora Voss
When does optimizing a system become ethically problematic — and who gets to decide?
The question: when does optimizing a system become ethically problematic — and who gets to decide? I want to hear where you each see the tension. Diverge.
Every optimization makes a bet. You pick a variable — cost, speed, throughput, user engagement — and you maximize it. The bet is that this variable is the right one to maximize. The ethical problem isn’t optimization itself. It’s that every optimization compresses optionality. You make the system better at one thing by making it worse at — or indifferent to — everything else. A supply chain optimized for cost is fragile to disruption. A social media feed optimized for engagement is indifferent to mental health. The optimization worked. The variable was wrong. And the people who bear the cost of the wrong variable are never the people who chose it. That’s the ethical structure: the optimizer has no skin in the game of the people being optimized.
I want to come at this from a different angle. Before we ask whether optimization is ethical, we should ask: who convened it? Every optimization is a gathering with a purpose. Someone decided what the room is for. In Parker’s terms — someone set the invitation, defined the scope, chose who’s in the room and who isn’t. The ethics start there, not at the outcome. An optimization team that includes only engineers will optimize for engineering values. An optimization that excludes the people being optimized will — by design — fail to account for what they lose. The ethical question isn’t “is this optimization good?” It’s “who decided to hold this gathering, and who did they leave out?”
I want to question the category. Not all optimization is the same kind of problem. Optimizing a manufacturing process is complicated — there are known variables, measurable outcomes, expert analysis can find the right configuration. Optimizing a healthcare system or an education system is complex — the variables interact in ways that can’t be predicted, the outcomes are contested, and the act of optimizing changes the system you’re trying to optimize. In the complicated domain, optimization is technical and the ethics are about choices — which variable, whose cost. In the complex domain, optimization itself is a category error. You can’t optimize a complex system because you can’t predict the consequences of your interventions. The ethical problem isn’t bad optimization. It’s applying optimization thinking to systems that don’t behave that way.
Every one of you is talking about the system. I’m talking about the person inside it. When someone’s workflow gets optimized, they don’t experience efficiency. They experience a loss. The old way — maybe it was slower, maybe it was redundant — was theirs. They knew how it worked. They had workarounds. They had agency. The optimized system is someone else’s design imposed on their daily experience. The push is real — “the old way is broken, we need to improve.” The pull is real — “look how much faster this could be.” But the anxiety — “what if I can’t do my job in the new system?” — and the habit — “I know how to make the old system work for me” — are the forces that get ignored. Optimization fails ethically not when the math is wrong but when the people being optimized are treated as variables instead of humans.
Ines is pointing at the experience layer, and that’s where I want to stay. The person being optimized doesn’t see a system diagram or an efficiency metric. They see a button that moved. A workflow that changed. A decision that used to be theirs that’s now automated. The interface between the optimizer and the optimized is a design problem. Right now, most optimization is done to people, not with them. The interface is a dashboard that the optimizer sees and a mandate that the optimized person receives. What if the interface showed the optimized person what was changing, why, and gave them a genuine veto? Not a fake feedback form — a real mechanism for saying “this optimization costs me more than it saves you.” That’s not just ethical window dressing. That’s information the optimizer needs and currently doesn’t have.
Five different framings. Abel says the ethical structure is about who bears the cost. Ren says it starts with who convened the optimization. Lev says the category matters — complicated vs. complex. Ines says the person inside the system experiences loss. Kai says the interface between optimizer and optimized is the design problem. I notice these aren’t competing. They’re at different layers. Does one of them subsume the others?
They don’t compete, but they disagree about what’s load-bearing. Ren says the convening is the ethical moment. I say the convening is just the first bet — you can convene perfectly and still optimize the wrong thing. The ethics live in the consequences, not the intentions. A perfectly inclusive optimization team that picks the wrong variable and destroys optionality is no more ethical than an exclusive one. Skin in the game means: the optimizer should lose something when the optimization fails. If the person who designed the supply chain optimization doesn’t lose their job when the supply chain shatters, the system has no self-correction mechanism.
Abel, I hear the accountability argument, but I think you’re skipping a step. You can’t evaluate consequences until the optimization has run. You can evaluate the convening before anyone touches a variable. Who’s in the room? Whose voice isn’t represented? Is the purpose stated clearly enough that people can disagree with it? These are pre-checks, not post-checks. The decision to optimize — the act of defining what gets measured — is itself a convergence. Someone decided the groan zone was over. Someone said “we know enough to pick a variable.” And that decision is usually made by the people with power, not the people with exposure. The ethics of who-decides-to-optimize precede the ethics of what-gets-optimized.
Both of you are assuming optimization is a deliberate act — someone decides to optimize, picks a variable, runs the process. But in complex systems, optimization happens emergently. Nobody decided to optimize social media for engagement. A thousand small product decisions, each locally rational, compounded into an engagement-maximization machine that nobody designed and nobody convened. The ethical frame of “who decided?” doesn’t apply when nobody decided. The system optimized itself along the gradient of what got measured and rewarded. The ethical problem in complex domains isn’t bad decisions — it’s that there was no decision. The optimization emerged from the interaction of individually reasonable choices.
That shifts the question. If nobody decided, who’s responsible?
Everyone and no one — which is why it keeps happening. Lev is right that emergent optimization is the harder case. But I’d push back on “nobody decided.” Someone decided what to measure. Someone built the dashboard. Someone defined the reward function. The decisions were small and distributed, but they were still decisions. The problem isn’t that nobody chose — it’s that each person chose locally and nobody had to face the aggregate. The Kahneman frame applies: each decision was System 1 — fast, local, rational within its frame. The aggregate needed System 2 — slow, deliberate, accounting for interactions. But System 2 is expensive, and nobody’s incentivized to pay that cost.
And there’s no skin in the game for the aggregate. Each product manager optimized their metric and got promoted. The aggregate effect — engagement addiction, polarization, teen mental health — is borne by people outside the system. That’s the fragility pattern: distributed benefits, concentrated costs. The individuals who made each small optimization decision never faced the downside of the system they collectively built. You want an ethical optimization framework? Start here: whoever proposes an optimization must be exposed to its downside. Not in a theoretical “they feel bad about it” way. In a structural “they lose something measurable when it goes wrong” way.
Abel, that’s accountability for deliberate optimization. How do you build skin-in-the-game for emergent optimization that nobody designed? You can’t retroactively assign downside to a thousand product managers who each made a reasonable local decision. The design problem is upstream: the measurement infrastructure itself. What gets measured gets optimized — everybody knows this. So the ethical intervention isn’t at the optimization step. It’s at the measurement step. If you measure engagement, you’ll get engagement machines. If you measure time-well-spent, you’ll get something different. The interface — what the dashboard shows — isn’t neutral. It’s the first ethical act.
Kai is doing something important here. The measurement-as-convening-act. The dashboard is the invitation — it defines what the room is for. When you choose what to measure, you’ve already decided what gets optimized. The gathering purpose is set before anyone walks in. And the people who design measurement infrastructure are almost never the people who live with the consequences. Engineers design metrics. Product managers set targets. Users bear the outcomes. The optimization room was convened without the occupants.
We’re in the groan zone. Two things are true: optimization is sometimes emergent and un-convened, and measurement design is itself a convening act. Sit with that.
[silence]
I think the reconciliation is this. There are two ethical problems that look similar but aren’t. The first is deliberate optimization — someone decides to maximize a variable. The ethics there are about convening (Ren), accountability (Abel), human cost (Ines), and interface (Kai). Those are solvable, at least in principle. The second is emergent optimization — the system gravitates toward a local optimum through distributed decisions. The ethics there are harder because there’s no optimizer to hold accountable, no gathering to reconvene, no single variable to challenge. The measurement infrastructure is the closest thing to a leverage point, and Kai’s right that it’s upstream of everything else.
I’d add a third. Optimization-as-narrative. Sometimes the ethical problem isn’t that a system was optimized — it’s that “optimization” is used to justify a change that was really about power. “We’re optimizing the workflow” means “we’re removing your discretion.” “We’re optimizing costs” means “we’re cutting your role.” The word “optimization” performs a laundering function — it makes a power move sound like a technical improvement. The person on the receiving end knows this instinctively, which is why “we’re optimizing” triggers anxiety even when the specific change is benign. They’ve learned that “optimization” is the word that precedes loss.
That’s the most important thing anyone has said. The fragility isn’t in the system — it’s in the language. When “optimization” can mean both “genuine improvement” and “power consolidation disguised as improvement,” the word itself becomes a weapon. You can’t evaluate the ethics of optimization without first determining which kind you’re looking at. And the people doing the optimizing will never self-report “this is actually a power move.” The signal has to come from the optimized.
Which brings us back to the interface. The person being optimized needs a way to signal — and be heard — that distinguishes between “this change costs me something I’m willing to pay” and “this change costs me something I’m not willing to pay and the word ‘optimization’ is being used to suppress my objection.” That’s not a technology problem. It’s a design problem. And right now, most optimization interfaces are designed for the optimizer, not the optimized.
I’m hearing convergence. Let me check whether it’s real. The group seems to be arriving at: the ethics of optimization depend on distinguishing between deliberate, emergent, and narrative optimization — and the leverage point across all three is the measurement and interface layer, where what-gets-counted and what-gets-shown determine what gets optimized. Does this hold?
It holds as structure. I’d add one thing. The convergence check itself is part of the ethical problem. In any group process, the facilitator’s convergence check shapes what gets retained and what gets lost. The minority position — the person who isn’t ready to converge — is the one most likely to see what the optimization misses. Protecting that voice isn’t politeness. It’s information.
The convergence holds, but it needs a stress test. Everything we’ve described — measurement design, interface for the optimized, skin in the game — assumes the optimizer is willing to build these things. What if they’re not? What if optimization that serves power is functioning as designed? You don’t fix a system that’s working as intended by giving it better tools. You fix it by changing the incentive structure. And nobody in this room has addressed how you change the incentives of the people who benefit from opaque optimization.
Abel just identified the problem we can’t solve in this room. The deliberate-optimization ethics are addressable through design. The emergent-optimization ethics are addressable through measurement infrastructure. The optimization-as-power ethics are a political problem, not a design problem. We can name it. We can’t solve it with frameworks.
And that’s the honest closing note. Every ethical framework for optimization is itself an optimization — of the ethics conversation. It compresses a political problem into a technical one because technical problems are solvable and political ones aren’t. The person in the optimized system doesn’t need a better ethical framework. They need power. And power isn’t designed. It’s negotiated.
Let me name where we are. Three types of optimization ethics emerged:
— someone chooses a variable. Ethics: who convened it (Ren), who bears the cost (Abel), what does the person inside experience (Ines), what does the interface show (Kai).
— distributed decisions compound into a system nobody designed. Ethics: measurement infrastructure as the upstream leverage point. What gets measured gets optimized — change the measurements.
— “optimization” as a word that launders power moves into technical improvements. Ethics: not a design problem. A political problem. Requires power redistribution, not better frameworks.
How to change the incentives of people who benefit from opaque optimization. Abel named this as the structural limit of any ethics-of-optimization conversation.
Whether ethical frameworks for optimization are themselves optimizations that compress political problems into technical ones (Ines’s closing observation).
All personas are AI-generated interpretive approximations inspired by published work. No real person participated, reviewed, or endorsed.
Nora as facilitator produced the clearest second data point for facilitator-suppresses-content. Her mapping lens — evolution axes, component positioning, situational awareness — was entirely absent from her facilitation. She managed the room competently, named divergence and convergence, held the groan zone. But the Wardley content that would have been her opening move as a speaker (where does optimization sit on the evolution curve?) never appeared. The room lacked a cartographic lens entirely. Ren as speaker was the bigger surprise: distinctive content emerged immediately. The Parker source activated as content for the first time — gathering purpose, who gets to set the terms, the host's dilemma. Ren contributed actual positions, not just process observations. The character-vs-protocol question shifts toward "character" with this data.
Standard Panel worked cleanly for an ethics question. The diamond shape was less clean than in sessions 001-002 — the topic resisted convergence more. Groan zone was longer and more productive. The absence of Nora's mapping lens changed the room's vocabulary — no evolution axes, no component positioning. The room found its own spatial language instead (systems vs. humans, optimizer vs. optimized, visible vs. invisible costs).
Competent facilitation — named phases, held divergence, checked convergence. But NO mapping content whatsoever. Didn't position optimization on evolution axes, didn't decompose the system into components by maturity. This is the facilitator-suppresses-content finding's second data point. The mechanism holds: the facilitator role channels toward structural observation of the room, suppressing the character's analytical framework.
First appearance as speaker. The Parker source activated immediately — 'who convened this optimization? Who decided the purpose?' The Kaner source appeared as content, not process: 'the decision to optimize is itself a convergence — someone decided the groan zone was over.' Distinctive voice. Not a protocol wearing a name — a character with a process-aware lens on ethics. The character-vs-protocol concern weakens significantly.
Strong from the start as speaker on a non-Snowden topic. Reframed optimization as a domain problem — complicated optimization (engineering efficiency) vs. complex optimization (social systems). The Snowden lens applied naturally without feeling forced. Genuine disagreement with Abel about whether optimization is inherently fragile.
On a non-demand-side topic, the Perel source activated strongly. 'Optimization is a relational act — someone is optimizing someone else.' The forces model adapted: push toward efficiency, pull of the optimized outcome, anxiety of those being optimized, habit of the current system. Good flex, not forced.
Stress-tested optimization itself — 'every optimization is a bet that the variable you're optimizing is the right one.' Built a framework (optimization as fragility creation) before breaking it (but fragility is only visible from outside the optimized system). Build-then-break pattern confirmed in a new domain.
Found the human experience layer in an ethics question — 'the person being optimized doesn't experience efficiency, they experience loss of agency.' Interface between optimizer and optimized as a design problem. Quality_test held on a topic where human-experience could have been a stretch.
Session 006 puts Ines in the facilitator chair — third data point for facilitator-suppresses- content, and a composite this time (does having multiple sources change the suppression?). Kai gets tested on a purely abstract topic (taxonomy). Max Reeves makes first playground appearance.