When designing a system or a component we have ideas that form invariants. Sometimes the invariant is big, like a certain grand architecture, and sometimes it’s small, like the selection of a data structure. Except, eventually, you’ll want to add a feature that clashes with that invariant. At that point there are usually three choices:

- Don’t add the feature. The invariant is a useful simplifying principle and it’s more important than the feature; it will pay dividends in other ways.

- Add the feature inelegantly or inefficiently on top of the invariant. Hey, not every feature has to be elegant or efficient.

- Go back and change the invariant. You’ve just learnt something new that you hadn’t considered and puts things in a new light, and it turns out there’s a better approach.

Often, only one of these is right. Often, at least one of these is very, very wrong, and with bad consequences. Even when they are able to follow constraints, agents are terrible at identifying when the constraints need to change.