Why we measure first.

In classical ML you cannot ship without measurement. Every model that went live at Agoda or Quantcast had an offline eval set, an online experiment, and drift monitoring that fired when the world moved. None of it was optional.

LLM features ship without any of it. The demo works, the team ships, and then nobody can say whether last week’s prompt change helped or hurt. The arguments are about taste, because there is no number to point at.

That gap is why Baseweight exists. We build the measurement first, fix what it surfaces, and leave the harness in your repo so the next change is a number rather than an argument.

The rest follows from it. Reliability, inference cost, a model of your own: each one starts from something measured.