It’s been written about before. The great Ken Pomeroy pointed out the instability of plus-minus data years ago, and I cover the same ground in Thinking Basketball the book. But I’m still inundated with questions like “how many minutes does a lineup need to play before I trust the results?” Sometimes it’s difficult to pin down answers to these questions in plain English, but there are a few simple tricks for grasping when to trust lineup data.
Say a lineup plays 50 minutes together. You’ll see this early in the year, where “top lineups, minimum 50 minutes” are cited as legitimate indicators for the success (or failure) of a unit. What’s even sneakier is when they use possessions to double the number (there are around two possessions per minutes). The first question to ask is what would happen to the net rating if this lineup had a bad stretch?
A cold two minutes
Note: For the simulations in this post, I’ll assume that net ratings — the efficiency edge a lineup gains based on net points per possession — are generated by playing around league average pace and is anchored to league average efficiencies.
Let’s start with a lineup that has played 100 possessions together with a +15 net rating. Sounds impressive, right? (That’s about 50 minutes of court time.) Well, what happens to that rating when they give up a 15-0 run? These are common enough: one team goes cold while the other team catches fire.
Stop and think about your guess for a second.
Are they now a +13 lineup?
10 points better?
Have your answer?
The team’s new net rating is 0. That’s right, all it takes is a quick 15-point swing to completely wipe out any edge a lineup had generated in around 50 minutes of play at today’s efficiencies and pace. And if you want to see the effect of a 15-point run on every 2020 team after 30 percent of the season, Ryan Davis just wrote about this topic and published a 15-0 run simulation for every top lineup in the league. Here’s how the 76ers top lineup would change right now per those simulations:
Note what this means for single-game efficiencies: Saying “Team A had a +40 net rating with Player X in the game” is often just the equivalent of saying “Team A outscored Team B by 8 points with Player X in the game.” In other words, it’s noise.
So what happens to a 300-minute lineup with a +15 net rating that yields one of those 15-0 run? They’ve played over 600 possessions together, so they only drop to a +13 net rating. You might be tempted to think “ah, so 300 minutes is where lineup ratings start to stabilize!” But not so fast. We don’t know how many games it took to accumulate those 300 minutes.
What if our lineup in question only played four minutes per game? Would you play the best lineup in NBA history four minutes per game? Of course not. Lineups like this are selectively deployed by coaches to take advantage of matchups. If a lineup plays 25 minutes a game, they aren’t being selectively deployed — they’ll face starters, closing units, whatever.
The top five-man unit in the league last year played 16 minutes per game together. So we also need to consider the number of games (or minutes per game) when thinking about the stability of that net rating, even before we numerically adjust for “opponent strength.”
An off night
Instead of a hypothetical 15-0 run, a clearer way to demonstrate the stability of a lineup’s net rating is by having that lineup play a full game where they are blown out. This happens to the best of them — remember when the 1985 Lakers lost 148-114 in Game 1, then won four of five games to win the title?
What happens to that +15 lineup that has played 100 possessions when they lose a game by, say, 25 points? They lose 22 points off their net rating and become a -7 lineup! Now that is an unstable sample. What happens if they played 300 possessions and suffered such a loss? They’d lose 11 points off their net rating and drop to a +4 lineup. Here’s a full reference chart for +15 lineups before and after a blowout like this, ordered by minutes played before the blowout occurred:
| MP | Est Pos | Net Rating | After 25 pt loss | Change |
| 25 | 53 | 15.2 | -11.1 | 26.4 |
| 50 | 105 | 15.2 | -4.4 | 19.6 |
| 100 | 210 | 15.2 | 2.3 | 13.0 |
| 150 | 315 | 15.2 | 5.5 | 9.7 |
| 200 | 420 | 15.0 | 7.3 | 7.7 |
| 250 | 525 | 15.0 | 8.6 | 6.4 |
| 300 | 630 | 15.1 | 9.6 | 5.5 |
| 350 | 735 | 15.0 | 10.2 | 4.8 |
| 400 | 840 | 15.0 | 10.7 | 4.3 |
| 450 | 945 | 15.0 | 11.2 | 3.8 |
| 500 | 1050 | 15.0 | 11.5 | 3.5 |
| 600 | 1260 | 15.0 | 12.1 | 2.9 |
| 700 | 1470 | 15.0 | 12.4 | 2.5 |
| 800 | 1680 | 15.0 | 12.8 | 2.2 |
| 900 | 1890 | 15.0 | 13.0 | 2.0 |
| 1000 | 2100 | 15.0 | 13.2 | 1.8 |
You’ll notice that even after 300 minutes, a blowout loss drops this lineup under +10, and after 600 minutes, it still shaves 3 points off its net rating. And here’s where a lineup’s minutes per game become such a great shorthand for eyeballing its quality — if you see a 300-minute lineup 20 games into the season, they aren’t selectively picking on vulnerable opposing units for two or three minute stretches every night.
But if they’ve only played 20 games together, they might not have played many good opponents! To “trust” raw lineup data, we almost always want samples that are more than a few hundred minutes and we want them to have played diverse opponents. Oklahoma City had one of the best five-man units in the league in 2019 (+8.2 net rating in 919 minutes), but against the top-five playoff teams in the West, the unit was almost dead even in 258 minutes.1
So the simple tricks for assessing raw lineup data are this:
- Check how many minutes per game the lineup is playing. Around 10 minutes per game is a healthy indicator. Four or five minutes per game should raise some flags.
- Check how many total minutes they’ve played together. Even two hundred minutes (or 400-something possessions) is generally an unstable sample.
- If you can, check who they’ve played. This holds true for lineups that play 300 or 900 minutes, because a team like OKC can beat up on weaker opponents without succeeding against quality opponents.
