Every mature SaaS company has a customer health score. Almost none of them were built with the statistical rigor to know whether it actually predicts what it's supposed to predict.
The standard health score is built by a customer success leader who, based on intuition and experience, assigns weights to a set of signals: product usage, NPS, support tickets, engagement, renewal date proximity. The resulting weighted sum becomes the health score. It's better than nothing. It's not a validated predictor.
The myth of health scores: that a reasonable-seeming model is a reliable predictor. In practice, most health scores are correlation noise — they correlate with health scores at churn time because churned customers look unhealthy, but they don't reliably predict churn before the decision is made.
Building a health score that actually predicts:
Start with outcome data, not input data. Collect 12-24 months of renewal and churn outcomes for all accounts. Then model backward: what were the behavioral, engagement, and relationship signals 90-180 days before each outcome? This is supervised learning — train your model on actual outcomes.
Use a holdout set to validate. If your health model was built on the same data you're validating against, it will look predictive when it isn't. Validate on accounts that weren't in your training set.
Measure prediction accuracy, not score distribution. A good health model should correctly predict 70%+ of churns before they happen. If your health scores aren't flagging the accounts that eventually churn while you still have time to intervene, the model is wrong.
Rebuild annually. Customer behavior, competitive dynamics, and product usage patterns change. A health model built in 2023 may not be a valid predictor in 2026.
Build it to predict. Validate that it does.