Every model in meryn — regardless of engine — reports its metrics through the same set of columns, split by train, validation, and test where holdout validation was turned on (see each engine's holdout setting: Robyn, Meridian, PyMC). The Compare page, reached from a Benchmark node, lines these up across every connected model.
Reading the metrics
| Metric | What it tells you |
|---|---|
| R² | How much of the variation in your outcome the model explains. Closer to 1 is better, but a very high R² on the training split alone doesn't guarantee the model generalizes — check it alongside a validation/test split if you turned holdout on. |
| NRMSE (Normalized RMSE) | The model's typical error size, scaled so it's comparable across models with different outcome scales. Lower is better. |
| MAPE / wMAPE (Mean Absolute Percentage Error) | The average size of the model's error as a percentage of the actual value. Lower is better. If you've added calibration data to a Robyn model, its MAPE reflects how well the model matches your lift tests specifically, not general day-to-day fit — don't read it side-by-side with an uncalibrated model's MAPE as if they measured the same thing. |
| ROI | Total return per dollar of paid media spend, across the whole model — contribution from baseline, organic, and non-media factors is excluded from both sides of that ratio, so it's specifically "what paid media returned," not a blended number. |
| RSSD (Robyn only) | How far each channel's modeled share of the outcome is from its share of spend. Robyn's optimizer treats a lower RSSD as a sign the result is more plausible — a solution where one channel gets 5% of spend but is credited with 60% of the outcome is exactly what a high RSSD flags. |
| Convergence (Meridian and PyMC only) | Whether the underlying MCMC sampling actually settled on a stable answer. Robyn doesn't use MCMC, so this doesn't apply to it. Treat a model that didn't converge as not yet trustworthy — see each engine's page for what to adjust before trusting its other numbers. |
Comparing across engines
A few things make an apples-to-apples read harder than the numbers alone suggest:
- Holdout methodology differs. Robyn and PyMC hold out the final stretch of the date range; Meridian holds out scattered geo × time cells across the whole period. Two models with a similar validation score aren't necessarily being tested the same way.
- Organic vs. contextual isn't split the same way. Meridian distinguishes organic media from other non-media covariates; Robyn and PyMC treat both as plain covariates. A channel-by-channel contribution comparison between Meridian and the other two won't line up perfectly for anything mapped as Organic.
- A higher R² isn't automatically the better model. Prefer whichever model has the more honest validation setup (holdout turned on, converged if applicable) over whichever one simply reports the highest training-set fit.
Choosing a winner
From the Benchmark node, pick the model (and its allocator, if it has one) you consider the right answer. This is purely internal curation — it doesn't show anything to anyone outside your team by itself. See Sharing & publishing for the separate step that actually makes a winner visible outside the team.