Archive / Ethics

Ethics · 9 min read

Begin with the case the critics of Silicon Valley find hardest to answer, because it is genuinely strong, and it deserves to be built honestly before it is tested.

You are building a product. You have limited engineers, limited capital, limited time, and the choice of what to build is not optional: the resources will be spent on something no matter what you decide. Two roads lie ahead. On the first, you build a feature that measurably improves life for a hundred million people: faster access to information, a translation tool, a map that helps the illiterate navigate, a payment system that lifts sellers in slums out of cash economies that rob them. On the second, you devote the same resources to designing meticulously for the people at the margins: users with rare disabilities, users on decade-old hardware, users in jurisdictions with hostile regulators, users whose needs are real but whose numbers are small.

The utilitarian case for the first road is a sentence: the greatest good for the greatest number. And the sentence has an august pedigree. Jeremy Bentham founded the tradition by asking a deceptively simple question: what is the point of morality, if not the well-being of the people it governs? Systems that judged actions by conformity to divine command, natural law, or abstract duty had produced centuries of confident cruelty, burnings and persecutions defended as service to principles no one could verify. Bentham proposed a standard that was, at least in principle, checkable: pleasure and pain, experienced by sentient creatures, measurable in principle, aggregable across people. An act is right to the extent it adds to the total sum of happiness. Wrong to the extent it subtracts. Every person counts for one, he added, and nobody for more than one. That last clause was radical in 1789: a viscount's pleasure weighs the same as a farmhand's. The ledger does not care about rank.

Mill, refining the doctrine a generation later, took the edge off the arithmetic by insisting that pleasures differ in quality, not just quantity, and that a dissatisfied Socrates counts for more than a satisfied fool, on the fool's own testimony if pressed. But the core survived every refinement: ethics is about outcomes, outcomes are about welfare, welfare is about people, and people can be counted. No ghosts, no categorical imperatives, no mysteries. Just the suffering and flourishing of the living, weighed impartially.

Now bring the doctrine to product ethics, and watch it become the industry's native moral grammar, the one its leaders reach for when they need to justify a decision at a podium. Free communication tools for billions: the largest expansion of human connection in history, full stop. Machine translation collapsing the language barrier. Drones mapping rural clinics. Disaster coordination. Market access for the smallest sellers on earth. Against these, the objections of critics look like aesthetic complaints from people who already have their connectivity, their infrastructure, their seat at the table of the connected century. And every product decision since has inherited the logic. If a change helps two hundred million users slightly and harms two hundred thousand users severely, the numbers rule for the change. Ship it. The utilitarian will concede the two hundred thousand, express regret in a footnote of a slide deck, and hold the line, because the alternative, agonizing over every edge case while the majority waits, is a decision procedure that helps no one, ever, at any speed.

That is the steelman. Take a moment to notice its strength. It is not greed wearing a philosophy costume. It is an inheritance from a genuine moral revolution, one that dethroned priests and kings as the arbiters of right and handed the job to experience, to actual lived outcomes for actual people. When the industry intones "we built this for everyone," it is repeating, in a corrupted accent, one of the noblest sentences ever formulated about human equality.

Now the objection. And it comes from a man who spent his career asking what justice would look like if we could design it from behind a curtain.

Part II: Behind the veil

John Rawls published A Theory of Justice in 1971, and the book's famous device is a thought experiment. Imagine you are about to be born into a society, but with no knowledge of which station you will occupy. You do not know your class, your race, your sex, your talents, your health, your generation, or your luck. Rawls called this position the original position, and the ignorance it imposes the veil of ignorance. From behind the veil, you must choose the principles that will govern the society you are about to enter. What rules would you pick, not knowing who you will be when the veil lifts?

His answer, and the reasoning behind it, are the objection the utilitarian ledger cannot survive. Rational choosers behind the veil, Rawls argued, would never agree to a system that sacrificed a minority's basic interests for the majority's gain, because behind the veil, you might be in the minority. Not the majority's trivial amusements, note. The minority's basic interests: their liberties, their standing, their access to the preconditions of a life. A chooser who does not know whether they will be the one in the basement of Omelas, the one in the misclassified cohort, the one whose account the classifier wrongly suspends, does not maximize the average. They protect the worst position, because that might be their position. Rawls formalized this as the difference principle: inequalities are justified only insofar as they work to the benefit of the least advantaged. And he made explicit what this does to utilitarianism: the aggregation of welfare across persons, treating gains and losses as commensurable units in one sum, fails to take seriously the separateness of persons.

That last phrase is the hammer. The utilitarian ledger is elegant precisely because it melts a hundred million separate lives into one sum and permits smooth subtraction across them: your loss, my gain, net positive, ship it. Rawls's point is that there is no sum. There are only persons, each with exactly one life, and the loss borne by the misclassified two hundred thousand is not canceled by gains scattered across the comfortable majority. It is simply borne, wholly, by each of them, for the entirety of their one existence. There is no cosmic account book where their debits meet the majority's credits. There is only the actual world, where each person experiences only their own balance, forever.

Apply the veil to product ethics and the entire decision procedure inverts. The question "does this help the most users?" becomes "would I assent to this rule if I did not know which user I would be?" And the products then have to be designed for the worst positions in them: the user wrongly banned with no appeal, the user whose data leaks, the user whose feed radicalizes them, the user in the small-language market the roadmap deprioritizes, the moderator laboring at the takedown queue, the bystander harmed by content amplified for engagement. Notice that the veil does not ignore outcomes. It is obsessed with outcomes. It simply refuses to let the arithmetic average stand in for the fate of every actual person underneath it, and it forces the designer to sit, imaginatively, in the worst seat in the house they are building.

Part III: The ledger versus the lottery

The objection can be sharpened into a dilemma, and then the strongest utilitarian rejoinder deserves its turn.

The dilemma: either the aggregation is honest or it is not. If honest, the utilitarian must bite the bullet the doctrine's founder would have bitten: yes, the suspension of two hundred thousand innocent accounts is correct if the net welfare is positive, yes, the misclassification of the rare condition is acceptable if the aggregate diagnosis improves, yes, the poisoned feed for the vulnerable five percent is the price of the enriched feed for the rest. This is consistent, and it is monstrous, and everyone involved knows it, which is why no one at the podium ever actually says it. What they say instead is that the harms were unintended, edge cases, being worked on, which is the sound of a moral theory refusing to state its own conclusions out loud. If dishonest, and the aggregation quietly smuggles in protections for minorities whenever the cameras are on, then the theory is not guiding the decisions at all. Something else is: a latent sense that some losses must not be netted away, whatever the sum says. The utilitarian in practice is a Rawlsian in costume, or worse, a Rawlsian only under subpoena.

The rejoinder, and it is real: the veil has costs, and the utilitarian can itemize them. Perfect protection for the worst-off is unaffordable. A society that tolerates no risk to its most vulnerable position forfeits the dynamism that funds safety nets; a product that designs exclusively for the edge case ships nothing and helps no one, including the edge case, who needed the tool that never launched. Rawls himself was addressing the basic structure of society, constitutions and legal systems, not software roadmaps, and his principles allow inequalities that lift the bottom, which is precisely what the tech industry claims to do: rising tides, global access, the poverty reduction that the connectivity critics' own charts show. The utilitarian can also point at the veil's own arbitrariness: probability-weighted expected welfare behind the veil collapses back into maximizing average welfare, unless the choosers are extremely risk-averse, in which case the resulting society cowers, minimax-ing its way to stagnation while real people with real names remain poor and unfed. The ledger does feed people. That is not nothing. That was never nothing.

And both rejoinders carry a final sting aimed at Rawls's heirs among the critics: the veil is a thought experiment too, and running it does not tell you who the least advantaged actually are, or how far protection extends, or what the worst seat in the house requires in a specific sprint. The utilitarian at least has numbers, feedback, and the humility of measurement. The critic has a curtain and a feeling.

Part IV: What ships

So the two theories stand, facing each other across the same corridor of decisions, and the industry has been living inside their unresolved tension for two decades while calling it roadmap prioritization.

What would settling it honestly require? Here is the uncomfortable part, and the reason this column steelmans before it argues. The honest utilitarian and the honest Rawlsian do not merely disagree about marginal cases. They disagree about whether the costs of the world are permitted to fall hardest on identifiable, nameable individuals so long as the scatter-pattern favors everyone in aggregate. The utilitarian, pressed, says yes, with regret. The Rawlsian says the regret is the tell: a principle you can only follow by practicing ritual discomfort at its outcomes is not a principle you believe. It is a principle you are excusing.

And the veil question, when it finally reaches the individual, is the one this essay leaves unresolved, in keeping with the house method. You are a user of these systems, and a builder of some of them, and a bystander of most. The next decision will be made by someone, somewhere in the chain, using some ledger or some curtain or neither. If you could not know which position in the system you would occupy, the obsequious majority or the discarded margin, the fed or the fed upon, which decision procedure would you want that someone using? You are about to find out what you would choose behind the veil, because you did choose it, last year, in a config file, in a purchase, in a policy you scrolled past and accepted. The positions were assigned while everyone was arguing about the arithmetic.

The greatest good for the greatest number. It sounds like arithmetic, and it is. That was always the seduction. The objection was never that the math is wrong. It is that people are not the kind of thing that adds.