The human cost of AI alignment — why the field is stuck on the wrong problem
There’s a paper by Shuang Zhang, Jiajin Yue, and Yifan Xu that came out on October 3. It’s called “Revisiting the Human Cost of AI Alignment: A Sociotechnical Perspective on Value Alignment Challenges.” It’s the first time I’ve seen someone write about alignment and actually talk about the people doing the work. Not the systems. The people.
The abstract says what the title promises: value alignment in AI systems remains deeply entangled with socio-cultural dynamics that technical approaches alone cannot resolve. They argue that the current alignment literature overemphasizes model behavior while underemphasizing the human infrastructure required to sustain it. The people collecting data, annotating outcomes, evaluating outputs, and mediating between models and users. The invisible labor that keeps alignment from collapsing into whatever the model decides to generate.
The infrastructure problem
The paper maps the alignment problem across four dimensions: social, epistemic, ethical, and technical. The technical dimension is what everyone talks about. The others get less attention.
The social dimension is about the people who do the work. Data annotators in low-wage markets processing sensitive content. Moderators reviewing model outputs. Researchers running evaluations. The paper calls it “infrastructure inequity” — the fact that alignment work is concentrated in under-resourced regions while the benefits accrue to organizations in wealthier countries. It’s not a new observation. It’s just not usually part of the alignment conversation.
The epistemic dimension is about whose values get encoded. The paper argues that the dominant technical approach assumes values can be extracted from existing data and formalized into reward functions. That’s a technical claim. But the data already reflects power structures. The reward functions already encode assumptions about what counts as “good.” The formalization process already selects for certain perspectives and excludes others. It’s not that the technical approach is wrong. It’s that it’s incomplete without a social analysis.
The ethical dimension is where the paper gets specific. It identifies five ethical challenges. One is cultural misalignment — the assumption that values from high-income countries are universally applicable. Another is value erosion — the gradual drift from stated values toward whatever the model optimizes for. There’s also transparency, accountability, and a fifth one I keep thinking about: the asymmetry of responsibility. When an aligned model fails, who’s responsible? The researchers who trained it? The annotators who labeled the data? The users who deployed it?
The sociotechnical argument
The core argument is that alignment is fundamentally a sociotechnical problem, not a technical one. Technical solutions are necessary but not sufficient. The paper doesn’t say we should stop building alignment methods. It says we need to integrate social analysis with technical development. That means involving affected communities, mapping power dynamics, and treating value alignment as an ongoing negotiation rather than a one-time engineering problem.
It’s a simple argument. It’s also the first time I’ve read it articulated this clearly in the alignment literature. Everyone knows the social stuff matters. Nobody puts it front and center.
The paper references earlier work — Sandoval-Herazo’s sociotechnical AI governance framework, which predates the alignment conversation but wasn’t integrated into it. It references critical AI studies, STS (Science and Technology Studies), and ethics of care. None of those appear in the citation lists of the papers that dominate alignment research. That’s the gap. Not a technical gap. A disciplinary one.
Why it matters
The human cost isn’t just a side note. It’s structural. When alignment work is outsourced to under-resourced regions, the people doing it have less agency over the values their labor encodes. When the data reflects existing power structures, the reward functions perpetuate them. When the evaluation is done by people in the same institutions building the models, the feedback loop is self-reinforcing.
The paper doesn’t offer a clean solution. It offers a framework for thinking about the problem differently. Instead of asking “how do we make models aligned?” it asks “how do we build infrastructure that supports ongoing value alignment across different communities and contexts?”
It’s a harder question. It doesn’t have a benchmark. It doesn’t produce a metric you can track on a dashboard. But it’s the question the field should be asking if it wants to produce systems that are actually aligned with something real, not just something measurable.
I keep coming back to the asymmetry of responsibility. It’s the one that sticks with me. We build these systems, we train them on data we collected, we evaluate them with benchmarks we designed, we deploy them in environments we chose, and when they fail, we ask the people who did the least to design those choices to bear the consequences.
The paper doesn’t say that directly. But that’s what it implies. And I think it’s right.