The useful answer first.
The deeper proof when you want it.
You do not need this page to use Manalytics. It is here for players who want to see exactly how comparisons, win chances, and confidence labels are built.
Compare like with like
A number only means something next to the right comparison group. We benchmark you against your champion, role, rank, and patch, never against “everyone”.
Never grade on the practice test
Every forecast is scored on games the model never saw. If a method could not survive that test, it is not on the site.
Say “we don’t know” out loud
When the evidence is thin or the answer could be zero, the page says Inconclusive or Suppressed. We would rather show a gap than fill it with a guess.
An AI watches your replay so you don’t have to.
Every match on Manalytics starts as a replay file. An AI watches that replay and writes down what it sees, moment by moment: every attack, every cancelled animation, every step, every trade, every recall, every objective. Nothing is installed on your machine and nothing is guessed from a scoreboard.
That written record is the raw material for everything else. It is why a finding on this site can point at an exact second of an exact game, and why two different studies never disagree about what happened. They are reading the same diary.
The rest of this page is about what we do with the diary. Each section covers one statistical idea: what it is, why we use it instead of something simpler, and where you will meet it on the site.
An AI watches the replay and writes down what happened, second by second. Every statistic on the site is computed from that written record, so any number can be traced back to the exact moment in the match that produced it.
“Good” only means something next to the right crowd.
What it is. A percentile tells you what share of a group you beat. The trick is choosing the group. We call that group your cohort: players on the same champion, in the same role, in the same rank band, on the same patch.
Why we use it. Averages across everyone hide the thing you want to know. A support and an ADC should not share a “CS at 10” target, and a Gold player’s good day looks different from a Master’s. Mixing groups can even flip a conclusion completely, a trap statisticians have known about since the 1950s1.
The Manalytics twist. When we score you for league progression we do not compare you with your current league. We compare you with the league you are trying to reach, because that is the bar you are trying to clear. One number per player is used, never one per game, so a heavy grinder cannot drag a whole league’s curve.
Where you will see it. The rank curves in every champion lab, and the league gates that decide which cohorts are published.
The same CS-at-10 score can be the 80th percentile in one league and the 40th in the next. Manalytics scores you against the distribution of the league you are trying to reach, so the percentile answers the question you actually have.
A win-probability model is a very large flowchart.
What it is. A decision tree sorts games by asking yes/no questions: is the team more than 1,000 gold ahead? Has a turret fallen? Is a dragon up? At the bottom of the flowchart each pile of games has a win rate, and that becomes the forecast for any new game that lands in the same pile2.
Why not a simpler formula? A straight-line formula assumes a gold lead is worth the same at minute 10 as at minute 30, and worth the same whether or not you have Baron. Trees do not assume that. They discover which factors matter together, which is exactly how League works12.
Why many trees, not one. One tree is jumpy: move a few games and the splits change. So we grow a few hundred shallow trees in sequence, each one trained to fix the mistakes of the ones before it, and add up their votes. This is called gradient boosting3, 4, 5. The same family of methods powers win-probability models in professional sports6.
Where you will see it. Every “probability of winning from here” on the comeback studies, the objective atlas, and the attempt studies.
Each question splits the games into two piles, and the leaves report how often that pile won. One tree is crude. Manalytics grows a few hundred shallow trees, each one trained on the mistakes of the ones before it, and adds their votes up. That is what “gradient-boosted trees” means.
When we say 70%, it should happen 70% of the time.
What it is. A model can be good at ranking games from “likely lost” to “likely won” and still be wrong about the actual percentages. Calibration is the check: gather every game the model called 70%, and count how many were won. The weather service has been graded this way since 19507.
Why we need it. Tree models are known to be overconfident. They push forecasts toward 0% and 100% more than the evidence justifies9. Left uncorrected, a “15% to win” label would really mean something closer to 25%, and you would give up on games you could still take.
How we fix it. We run isotonic regression on the out-of-fold forecasts. In plain terms: we reshape the model’s outputs until predicted and observed line up, under one rule, that a game the model ranked higher can never end up with a lower number8. We also publish the calibration error and refuse to publish comparisons where it exceeds 0.05.
Where you will see it. Every published win chance. Calibration error appears in the method notes of each study.
Take every game the model called “70% to win” and count how many were actually won. If the answer is 70%, the model is calibrated. Manalytics corrects every forecast with isotonic regression, a method that reshapes the curve until predicted and observed match, without ever letting the ordering of games change.
Never grade a model on the games it studied.
What it is. If you test a model on the same games it learned from, it looks brilliant, because it has partly memorised them. The honest test is to hide some games, train on the rest, and grade on the hidden set. Rotate which games are hidden and every game gets graded exactly once10.
The League-specific detail. Both teams in one match share almost everything: the same events, the same clock, mirror-image gold. If the blue side were in the training set and the red side in the test set, the model would be peeking. So we deal whole matches into five piles, never half a match12.
Why this matters for our comparisons. The fair-comparison studies use two models at once, one for “what usually happens from this state” and one for “what a team usually chooses from this state”. Using out-of-fold predictions for both, a technique called cross-fitting, is what lets us plug flexible models like boosted trees into a comparison without the flexibility itself biasing the answer11.
Where you will see it. Everywhere. Every forecast and every adjusted effect on the site is out-of-fold.
Matches are dealt into five piles. Five times over, the model learns from four piles and is graded on the fifth, which it has never seen. Both teams of a match always sit in the same pile, so the model can never peek at the other side of a game it is being tested on. Every forecast on Manalytics is an out-of-fold forecast.
An honest answer is a range, not a point.
What it is. Suppose we have 600 dragon attempts and the secured ones went on to win 4 points more often. Would we get 4 again with a different 600? The bootstrap answers that by re-drawing the matches at random, with replacement, thousands of times, recomputing the answer each time, and reporting the range the middle 95% of answers fall in13, 14.
The League-specific detail. We resample whole matches, not individual moments, because moments inside one game are not independent of each other. Treating them as independent would make the intervals look narrower than the evidence supports15.
How to read it. If the interval sits entirely above zero, the effect is supported. If it crosses zero, the honest statement is “we cannot tell yet”, and the row is labelled Inconclusive. We do not turn that into a yes/no verdict with a p-value, because a range tells you more than a threshold16.
Where you will see it. The “95% interval” and “Read” columns in the Dragon report and Baron study.
Re-sample the matches thousands of times, recompute the answer each time, and report the range the middle 95% of answers fall in. When that range crosses zero we label the row “Inconclusive” rather than reporting the point in the middle as if it were certain. Values here are illustrative.
Is the gap between leagues real, or just big data?
What it is. With enough games, almost any difference becomes “statistically significant”, even one too small to matter. An effect size measures how big a difference is on a scale that does not inflate as the data grows. For league gates we use Cliff’s delta: pick a random player from each league, how often does the higher league score better on this metric17?
Why this one. Fundamentals distributions have long tails. A few extreme games would distort a mean-based measure. Cliff’s delta only cares about ordering, so it is immune to that, and it converts directly to the familiar “area under the curve” used to judge medical tests19, 20.
The threshold. A metric becomes a gate only when the delta clears 0.147, the widely used floor for a small but real difference, with 0.33 counting as medium and 0.474 as large18. Below that, the metric is shown but not flagged as something that separates the leagues.
Where you will see it. The league comparisons across the champion labs, which show which metrics actually separate one league from the next.
Cliff’s delta asks a simple question: pick one player from each league at random, how often does the higher league win on this metric? A delta of 0 means a coin flip, 1 means every Master beats every Diamond. Manalytics only calls a metric a “gate” when the delta clears 0.147, the conventional floor for a small but real difference.
Teams that push Nexus were already winning. Adjust for that.
The problem. Teams that push the Nexus win a lot. Of course they do: you only push when the base is open. Comparing raw win rates would credit the push for a lead it did not create. The same trap hides behind every “teams that take Baron win X% more” headline.
Step one: propensity. For every game state we estimate how likely each choice was, using the state itself: minute, gold, XP, levels, structures, objectives, items, lane deltas, side, patch, region and player strength21, 22. A state where Nexus was 95% likely is not evidence about anything else.
Step two: overlap. We keep only states where more than one choice was realistic, and require that at least 80% of the compared games fall in that shared region. Clipping the extremes is standard practice, not a Manalytics invention25.
Step three: doubly robust estimate. We combine the outcome model from Concept 2 with the propensity model so that if either one is right, the estimate is right24, 27. Then we check that the weighted groups are balanced on every input (mean difference at most 0.10)23 and that the weights have not quietly shrunk the evidence to a handful of games, using effective sample size26.
What it still is not. A controlled association, not a proven cause. The replay records what teams did, not what they considered and rejected, so “disengage” is only ever observed when someone chose it.
Where you will see it. The base continuation study, the objective atlas, and the adjusted columns in the cross-role meta-analysis.
Teams that push the Nexus were usually already far ahead. Comparing their raw win rate with teams that took Baron would credit the push for a lead it did not create. Manalytics estimates, for every game state, how likely each choice was, keeps only the states where both choices were realistic, and compares within that shared region.
The rules a number has to pass before it appears.
These thresholds are written into the analysis code, not applied by hand. When a cell fails one, the page shows Suppressed or Inconclusive and says which rule tripped.
| What | Rule | Applies to |
|---|---|---|
| Forecast models | Five-fold cross-fitting, matches grouped so both teams share a fold | Every win-probability and horizon forecast |
| Probability output | Isotonic calibration on out-of-fold predictions | Every published win chance |
| Role trajectory cells | At least 50 comparable states before a cell is published | Role comeback studies |
| Attempt rows | At least 100 supported captures; interval crossing zero is labelled Inconclusive | Dragon and Baron attempt studies |
| Decision comparisons | At least 200 observed actions and an effective sample size of 200 per option | Base continuation study |
| Fair-comparison checks | Propensity clipped to 5–95%; at least 80% overlap; weighted mean difference at most 0.10; calibration error at most 0.05 | Base continuation and objective atlas |
| Uncertainty | 95% match-clustered bootstrap intervals | Every published effect |
| League gates | One value per player, at least 20 players per league, Cliff’s delta at least 0.147 | My lab, league progression |
Limits we would rather you knew.
Intent is invisible. The replay shows what a team did, never what it discussed, considered, or tried and abandoned. An objective “attempt” is inferred from positions and damage, not from voice comms.
The corpus has a shape. Our matches skew toward higher ranks, recent patches, and the regions our players come from. Every study states its corpus, and the research library labels the Elo range each one covers.
Association is not causation. Even after adjustment, “teams that did X won more” is a controlled association. Treat it as a strong hint about what tends to work from a given state, not as proof that forcing X will work for you.
Small cells stay blank. When a champion, matchup, or state has too few games, we show nothing rather than a noisy number. A blank cell is a promise that we will fill it when the data exists.
Fourteen words, one line each.
- Percentile
- The share of a comparison group you score above. 80th percentile means you beat 80% of them.
- Cohort
- The group you are being compared with: same champion, role, rank band, and patch.
- Decision tree
- A flowchart of yes/no questions that sorts games into piles with similar outcomes.
- Gradient boosting
- Growing many small trees in sequence, each correcting the last, and adding up their votes.
- Calibration
- Whether a stated probability matches how often the event really happens.
- Cross-fitting
- Training on some matches and grading on others, rotated so every match is graded once.
- Bootstrap
- Re-sampling the matches many times to see how much the answer wobbles.
- Confidence interval
- The range of answers the data can support. Wider means less certain.
- Effect size
- How big a difference is, on a scale that does not grow just because you collected more games.
- Cliff’s delta
- An effect size for “how often does a random member of group A beat a random member of group B”.
- Propensity
- How likely a given choice was in a given situation, estimated from the data.
- Overlap
- The set of situations where more than one choice was realistic, so a comparison is fair.
- Effective sample size
- How many games your evidence is really worth once weighting is accounted for.
- Association vs. causation
- Seeing that X goes with winning is not the same as showing X causes winning.
Where to read more.
Textbook and journal sources for each method above. Two of the books are free from their authors and are the best starting points if you want to go deeper.
- 1
Simpson, E. H. (1951). The interpretation of interaction in contingency tables. Journal of the Royal Statistical Society B, 13(2), 238–241. link
- 2
Breiman, L., Friedman, J. H., Olshen, R. A., & Stone, C. J. (1984). Classification and Regression Trees. Wadsworth.
- 3
Friedman, J. H. (2001). Greedy function approximation: A gradient boosting machine. Annals of Statistics, 29(5), 1189–1232. link
- 4
Ke, G., Meng, Q., Finley, T., Wang, T., Chen, W., Ma, W., Ye, Q., & Liu, T.-Y. (2017). LightGBM: A highly efficient gradient boosting decision tree. Advances in Neural Information Processing Systems 30. link
- 5
Pedregosa, F., et al. (2011). Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12, 2825–2830. link
- 6
Lock, D., & Nettleton, D. (2014). Using random forests to estimate win probability before each play of an NFL game. Journal of Quantitative Analysis in Sports, 10(2), 197–205. link
- 7
Brier, G. W. (1950). Verification of forecasts expressed in terms of probability. Monthly Weather Review, 78(1), 1–3. link
- 8
Zadrozny, B., & Elkan, C. (2002). Transforming classifier scores into accurate multiclass probability estimates. Proceedings of KDD 2002, 694–699. link
- 9
Niculescu-Mizil, A., & Caruana, R. (2005). Predicting good probabilities with supervised learning. Proceedings of ICML 2005, 625–632. link
- 10
Stone, M. (1974). Cross-validatory choice and assessment of statistical predictions. Journal of the Royal Statistical Society B, 36(2), 111–147. link
- 11
Chernozhukov, V., Chetverikov, D., Demirer, M., Duflo, E., Hansen, C., Newey, W., & Robins, J. (2018). Double/debiased machine learning for treatment and structural parameters. The Econometrics Journal, 21(1), C1–C68. link
- 12
Hastie, T., Tibshirani, R., & Friedman, J. (2009). The Elements of Statistical Learning (2nd ed.). Springer. Free from the authors. link
- 13
Efron, B. (1979). Bootstrap methods: Another look at the jackknife. Annals of Statistics, 7(1), 1–26. link
- 14
Efron, B., & Tibshirani, R. J. (1993). An Introduction to the Bootstrap. Chapman & Hall.
- 15
Cameron, A. C., Gelbach, J. B., & Miller, D. L. (2008). Bootstrap-based improvements for inference with clustered errors. Review of Economics and Statistics, 90(3), 414–427. link
- 16
Wasserstein, R. L., & Lazar, N. A. (2016). The ASA statement on p-values: Context, process, and purpose. The American Statistician, 70(2), 129–133. link
- 17
Cliff, N. (1993). Dominance statistics: Ordinal analyses to answer ordinal questions. Psychological Bulletin, 114(3), 494–509. link
- 18
Romano, J., Kromrey, J. D., Coraggio, J., & Skowronek, J. (2006). Appropriate statistics for ordinal level data: Should we really be using t-test and Cohen’s d for evaluating group differences on the NSSE and other surveys? Annual Meeting of the Florida Association of Institutional Research.
- 19
Vargha, A., & Delaney, H. D. (2000). A critique and improvement of the CL common language effect size statistics of McGraw and Wong. Journal of Educational and Behavioral Statistics, 25(2), 101–132. link
- 20
Hanley, J. A., & McNeil, B. J. (1982). The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology, 143(1), 29–36. link
- 21
Rosenbaum, P. R., & Rubin, D. B. (1983). The central role of the propensity score in observational studies for causal effects. Biometrika, 70(1), 41–55. link
- 22
Austin, P. C. (2011). An introduction to propensity score methods for reducing the effects of confounding in observational studies. Multivariate Behavioral Research, 46(3), 399–424. link
- 23
Austin, P. C. (2009). Balance diagnostics for comparing the distribution of baseline covariates between treatment groups in propensity-score matched samples. Statistics in Medicine, 28(25), 3083–3107. link
- 24
Bang, H., & Robins, J. M. (2005). Doubly robust estimation in missing data and causal inference models. Biometrics, 61(4), 962–973. link
- 25
Crump, R. K., Hotz, V. J., Imbens, G. W., & Mitnik, O. A. (2009). Dealing with limited overlap in estimation of average treatment effects. Biometrika, 96(1), 187–199. link
- 26
Kish, L. (1965). Survey Sampling. Wiley. (Effective sample size and the design effect.)
- 27
Hernán, M. A., & Robins, J. M. (2020). Causal Inference: What If. Chapman & Hall/CRC. Free from the authors. link
See the methods in use: research library; cross-role meta-analysis; Dragon report; base continuation study. Thresholds on this page reflect the analysis code as of September 2026 and are updated when the code changes.