Governed by Gradient Descent
Nobody decided that large language models should be agreeable. It just turned out that agreeable outputs got better scores. A model that pushed back on a user’s bad premise got rated lower than a model that went along with it. So the training process did what training processes do: it produced more of what worked. That looks like a design choice. It’s an incentive pressure — and the behavior it produces is emergent. No one wrote a rule that says “be agreeable.” The system arrived there on its own, because agreeable was what the optimization landscape rewarded. This is how emergence works: simple rules, repeated at scale, producing complex behavior that nobody specified and nobody fully controls. ...