The Effect of Online Popularity on Matchmaking in Warzone

Let’s do something dangerous. Let’s shut down our critical thinking apparatus and ignore the ensuing (and hopefully loud) ring of our cognitive alarm bells. If we maintain this state long enough while looking at the following figure, we will be rewarded with outrage.

[Click here for the Reddit link of this post (878 upvotes, 291 comments)]

This figure shows the median lobby KD distribution of the 5 most popular Warzone streamers (going by watch time on Twitch in the last 30 days: Nickmercs, Timthetatman, Swagg, Symfuhny and Aydan) compared to the median lobby KD distribution of 5 high KD Warzone streamers with fewer than 300 Twitch followers^[1] as well as of 5 high KD players that, to my knowledge, are not streaming^[2]. The players for the latter two categories were obtained via the US leaderboards of codtracker.

For each category, the last 500 matches of each player are considered. After excluding non-BR game modes, we are left with a total of 2,391 matches for the top streamer category and with, respectively, 2,312 and 2,101 matches for the other two (for an explanation of how I obtained this data, see my previous post on SBMM/EOMM in Warzone).

While the orange and green lines seem almost identical, the blue line representing the lobby KD distribution of the top 5 Warzone streamers is markedly skewed to the left and lacks the distinctive peak that marks the most common lobby KD of around 1.15 seen for the other two distributions. In addition, the average median lobby KD of the top streamers is also statistically significantly lower (significant at the 5% level, if you are interested) than the average median lobby KD of the other two categories (1.03 vs 1.10/1.09).

So is this statistical proof supporting the commonly mentioned theory that top streamers are favored by the matchmaking system in order to advertise the game by posting exceptional gameplay obtained in easier lobbies? Is this enough to grab our pitchforks and torches and march against this unfair treatment of the player base? After rebooting our critical thinking systems, the empirical base for these claims disappears rather quickly.

The top streamers only played 505 matches during the SBMM prime time from 8pm to 2am (see Figure 2 of my previous post) compared to 1,012 and 924 matches for players of the other categories (since I don’t know the location of all players, I assume that everybody lives in the Eastern Time Zone. This results in only minor classification errors since the lobby strength before 8pm is much less different from that of the SBMM prime time than the lobby strength after 2am). In order to make sensible comparisons, one has to therefore first restrict the analysis to a common playing time. The SBMM prime time seems like the obvious choice here.

However, the top streamers are more likely to play in international tournaments in which they load into BR Quads games and compete in 2vs2 kill races. This will bias their lobby strength during the SBMM prime time downwards as opposing European duos could already be inside the weaker time periods of SBMM. Every game hosted by the European opposition will then, on average, reduce the lobby KDs of our top streamers.

So to remove this systematic difference between the player categories influencing lobby strength, I remove all BR Quads observations from the next analysis. Moreover, I only include matches in which our players are part of a squad with a minimum KD of 3.0 to avoid most “Random Duos/Trios” games.

What remains then?

We see that the difference in average median lobby KD between the categories shrinks massively (1.11 vs. 1.12 vs. 1.11). In addition, the difference between the top streamers and the other players is not statistically significant anymore (graphically represented by the inclusion of the average median lobby KD of the top streamers in the range established by the whiskers of the other two categories).

So to everybody claiming that the most popular streamers play in easier lobbies than the average high level player: you are right. But to everybody interpreting this as top streamers being favored by SBMM: you are wrong (actually, to be more precise, there remains no statistical evidence to support your interpretation after accounting for the reduction in average lobby strength associated with streamers playing relatively more matches during off-peak hours).

[1] The players in this category are Ryda#11960, Mawnstrosity#1125, viteettv#1352, MrJuice#11327 and mcap#11939; Average KD of their own squad: 3.31.

[2] The players in this category are AdultNoJob#8109122, MFCoast, Shifty#12932, shane_m15 and oVisionzs#1313; Average KD of their own squad: 4.08.