The Great Winrate Fallacy

There seems to be an informal consensus among the online poker community that when you combine a tracking software-derived winrate with some magic arbitrarily-defined minimum number of hands that produced the winrate, then poof, you can “know” whether or not, and to what extent, you can expect to be a winner or loser at some specific form/structure of poker. The corollary is that if your “sample size is too small” then your results are just noise, and nothing can be concluded from your winrate.

This is all message board mythology. It’s complete BS.

A winrate is worthless without context (and maybe even with context), regardless of the size of the sample attached to it. The whole idea that a winrate can indicate confidence in future results, or even state anything meaningful whatsoever about past results, is mortally flawed.

There are numerous game factors in poker that a winrate fails to reflect in any way. A winrate is just a final tally. It doesn’t tell the whole story, or really any relevant part of the story at all, regardless of the size of the sample it’s coupled with.

Assumptions

The winrate as a meaningful entity depends on two blanket assumptions that, in most cases, can’t be made:

First, that the characteristics of Hero’s play were, are, and will remain consistent and uniform. In other words, Hero’s playing ability never improves, changes for the worse, or tilts (or that in cases where Hero “factors this in” that he will do so in a way that correctly adjust his effective winrate, which is math that we certainly can’t trust Hero to do).

Secondly, and far more importantly, that Hero’s collective opponents, and the characteristics of THEIR play, are and will continue to be uniform as well.

This—the opponent factor—is where the “winrate + large enough sample size = confidence” myth completely breaks down.

In poker, the rate at which one wins or loses depends far more on the collective play of one’s opponents than on his own. But the modern online horde of poker players tend to be marvelously self-centered about their own stats—such as their “see flop” percentages, aggression factor, winrate, etcetera—as if stats have some fixed and absolute meaning or “optimum” outside of the context of opponents (which of course they don’t).

Adjusted for rake, my winrate is and always will be far higher in my local brick-and-mortar Omaha 8 games than it will be in any online game, and higher still in my home game. That has nothing to do with me, and everything to do with my competition.

The tendency of many players to fixate on themselves is partly due to the nature of online poker, where one’s opponents are faceless, and, under most circumstances, in a state of constant flux. Rarely will Hero play against the exact same lineup day after day or even hour to hour, particularly at lower stakes. The irony is that this constantly churning mass of opposition makes Hero’s winrate even less meaningful than it would be in a live setting, where the player population and other game factors tend to be more consistent.

The fact that Hero wins one day at one table and loses the next day at a different table (or wins at two open tables while losing in two others) is at least as much a function of the difference between the dynamics of one lineup and the next as it is of luck or variance.

Variables

Using a winrate as a basis of confidence in predicting how Hero will fare is a little like blind-betting on a basketball game when you only know one of the two teams that are playing, or on a presidential candidate when you don’t know who is running against him.

To place any sort of confidence in a winrate is to assume that the ebb and flow of luck is the only factor that causes fluctuations in Hero’s day to day results. As long as the game environment remains dynamic in terms of players and even seat positions (as poker does by nature) this simply couldn’t be more wrong.

If the composition of Hero’s opponent pool changes from hand to hand, day to day, year to year, with old opponents leaving, new opponents coming, and existing opponents changing and evolving (or de-evolving) in their play, and Hero does the same, Hero’s winrate will never “stabilize.” No sample size will ever be large enough to state anything confident about Hero’s potential results, because the variables are not fixed.

Consider the presence of a single “live one” at one table, on one day, giving away bet after bet, or buy-in after buy-in. If you’re a slightly losing player over some large number of hands, and you happen to run across this fellow, and you keep ending up in favorable situations against him, and his idiocy knocks you into the green, does that mean you’re now a “winning player?” No. It means you ran into an idiot who handed you a bunch of money.

So, do you remove that session from your database so that the goofball doesn’t “throw off your winrate?” Maybe so.

But then… if you do that, what about that other guy who was tilting that one time and paid you off in all those hands?

Or what about the night that you were tilting?

Or the time when the red pro sat down at the table and beat you all silly and you were just happy for the opportunity to get in and play hands against him?

Or the hands where you misclicked?

Do you see? As you go deeper down the rabbit hole of the “unique” circumstances that came along to artificially affect your winrate, you’ll begin to notice that these “unique” circumstances are constantly popping up, day after day, session after session.

Unless the exact same players take their exact same seats and play the exact same form/structure of poker every single time with an unwavering level of skill, and the players never tilt, or focus, or bluff, or anything else, in any different measure than they ever have, then the data set is hardly a data set at all. It is, instead, an endless succession of unique circumstances and situations, with only the rules remaining fixed.

Yes, the ups and downs of the cards, and even the situations, will theoretically cancel out over time, but the game dynamics will not. They change drastically in the short term; and in a more subtle manner over the long term (edging, most likely, toward increasing difficulty). Variance is a pipsqueak of an issue when compared to the much larger and more complex issue of an ever-evolving game environment.

Variables Within Variables

There’s a whole lot more to consider when it comes to the winrate/sample size question. Even assuming a game environment where Hero and his opponents are static entities, the sample size that Hero would need in order for a reliable winrate to stabilize would vary widely based on:

1. The form of poker (holdem? 5 card draw? Omaha 8?)
2. The betting structure (no-limit? spread limit? fixed?)
3. The number of players (heads-up? 6-max? 10-handed ring game?)
4. Hero’s playing style (aggressive? tight? loose?), and any highly particular and persistent tendencies (either favorable or unfavorable) that Hero exhibits
5. Skill/playing style of any one given opponent
6. Characteristics of opponents as an inter-relating collective, and any possible overriding tendencies (which in some cases could be extremely exploitable)

Some specific examples: A high/low split game will not require as large a sample as a straight-high game, since split games entail less variance. Seven card stud would need a larger sample because it has generally more variance than razz. Limit games are steadier since they revolve around the aggregate results of many subtle decisions made often, with the bet sizes having a close relationship to the blinds, whereas big bet games revolve around the results of critical decisions made less often, and with bet sizes that frequently dwarf the blinds. A player with a loose-aggressive style will need a larger sample than a player with a tight-aggressive style (since the former style is inherently higher variance). And so on.

Point 6—collective tendencies—merits special attention. I used a similar example in a previous post, that time in relation to rake, but let’s assume that we’re playing some fixed-limit poker variant, or even a rotation of several, with 9 opponents, all of whom have three bizarre proclivities: One, they play only the top 25% of hands. Two, they play them to the river without exception. And three, if they have the nuts after the last card they will bet/raise. If they do not, then they will check/fold to any bet.

Now, obviously, in the real world one will never find players who play this badly, and Hero will crush the game if he plays the rather simple style needed to beat it, but here is the point: Hero’s winrate will stabilize in an extremely brief span of time. You wouldn’t need 50,000 hands, or 10,000 hands, 1000 hands, or even 500. Hero would know quickly approximately how much this game was worth. And again, the form of poker doesn’t matter in this case. Almost all that matters in creating Hero’s winrate are the particular, massively exploitable quirks which Hero’s opponents exhibit.

The above list could go on well past 6 items, but I think the point is made. One might argue that confidence ratings which include the standard deviation in their calculations will automatically “bake into the cake” many of those sorts of phenomena, but many other factors simply can’t be reduced to math. And, in any case, there are still plenty of armchair sample-sizers who don’t take standard deviation into account, and I expect to see them continue to fly in out of the blue on poker boards with the bold assertion that “you need 50,000 [or whatever] hands at a game before you have any idea whether or not you’re a winning player” (though they’d still be wrong even if they did take standard deviation into account).

A player with a 1 million hand database, a standard deviation of 15, and a winrate of 2BB/100, with all hands coming from one stake and several sites over 7 years of online play has a very different data set than a player with a 2BB/100, 15-standard-deviation winrate over 1 million hands at one stake that he just spent the last few months grinding out on PokerStars. Identical results, but both samples are HIGHLY problematic (and for totally different reasons!) when it comes to any sort of statement or prediction.

It’s All Noise

In terms of winrate, poker can’t be treated the same as a card counter would treat blackjack, a game where the rules, the basic strategy, the opposition, the standard deviation, and all other factors remain perfectly static, and a reliable winrate inevitably emerges after a quantifiable number of hands are played. Many “winning” poker players, though, seem to take for granted that their playing field isn’t constantly moving and shifing around them.

There is simply no way to produce a meaningful winrate without certain strict and specific game factors having been nailed into place over the course of the sample.

People should not even bother asking “how many hands in the sample?” before asking “how consistent were the game circumstances?”, the second being relatively far more important. Unfortunately, it isn’t easy to apply a figure to “game consistency.”

Given a sample size of 1000 hands against the same 9 opponents versus a sample size of 50,000 hands against hundreds of opponents, and asked which sample was more telling about Hero’s prospects, I’d take the former in a heartbeat. (But then, of course, I’d have to know what question we were trying to answer.)

The fundamental problem with focusing on winrate in and of itself is that it puts Hero and Hero’s play at the center of the universe, when it’s “the universe”—the collective play of Hero’s opponents relative to his—that matters so much more. Nothing else even comes close, certainly including the comings and goings of the cards/situations. Even if Hero’s play is exactly the same every single time he sits down, the game never is.

Winrates are the new starting hand charts—a crutch that anxious poker players gravitate to in order to try to simplify an issue (in this case, “am I a winner, or a loser?”) that is enormously and irreducibly complex.

Advertisements
Explore posts in the same categories: Uncategorized

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


%d bloggers like this: