Deciphering Balance

[TerranUp16]

I am going to start this rational discussion post with a link to a rather short, but highly interesting, article that I'd like everyone to read as a preface-
"Over-nerfed" by Artosis

(and because the site seems to be down now, here's a link to a thread on TeamLiquid that has the first part of the article and the core of its premise- link)

The article is quite genius in its analysis of the SC2 Beta phenomenon that many Westerners have often attempted to use as leverage to back-up various claims about balance this or balance that. Granted, the article is also Game Balancing 101, but it is a caveat rarely articulated so well to the general gaming populace. And it is also a principle of balancing and design that designers themselves often forget.

How do we evaluate the data and feedback we get from our strategy games? More specifically, what are the caveats we should observe when considering the balance of our game?

The first, and most important consideration is what skill level we want to balance the game for. For a strategy game, balance should always be targeted at the highest skill level. If the game wants to limit the skill level and be more casual, then facets of the game's design other than balance should promote this.

With that said, there can be some discretion. The easiest case of discretion to consider is that some very easy strategy that relatively low or mid-skilled players can execute consistently and easily should be counterable with a reasonable amount of skill. If only the very best player can execute the counter while nearly all other players struggle to reliably execute such a counter, then the counter likely requires some innate talent and cannot be "learned"- forcing players to have some innate talent to counter a strategy which does not require such to execute is generally unfair and counter-productive- even for games whose sole aspiration is eSports, as the point of the game should be to test the skills of both players, but clearly in such an instance the skill of only one player is being tested.

Extending from this, we can immediately see that skill matters. Comparisons of the effectiveness of strategies, builds, units, etc... all must be taken in the context of skill. Ideally, two players of the same skill level will play each other to a stand-still. In reality, this is never truly feasible because skill as most people think of it is actually a composite quantifier. Skill itself is composed of many skill subsets, whereby a player can be better with one build than another, better against one faction than another, etc... Nearly always will these factors toss a match-up in one direction or another. But more importantly from a designer and balance perspective, strategies are going to generally take different amounts of skill and to counter. As long as these counters exist and are obtainable, this fact is generally ok, as generally players can learn the counters and practice them and improve their game, up their skill level in that area, etc... Player growth basically.

Fomenting and requiring player growth is a good thing.

At the same time, designers must be wary of strategies that push such slight skill imbalances too far. To consider this, we will delve into some theoretical math. This math is quite too perfect for actual usage, but it conveys the theory and provides rough guidelines for employment in real-world conditions.

We are considering a strategy that may be imbalanced. The first question we need to answer for this is whether the strategy can be countered or not. If we know it can be countered, then we can proceed with this evaluation. If we do not know for sure if it can be countered, we need to resort to other measures.

If the strategy is counterable, we must briefly consider the case of discretion mentioned above- is this strategy relatively easy to execute and does the counter require the player to have some innate (and extremely uncommon) talent to pull off against inferior opponents? If so, we can leverage our discretion to immediately acknowledge that this is an imbalance.

Instead, if we find the counter to be feasible then once again we proceed. To go farther, we will need to consider a way of quantifying and ranking player skill. The development of such evaluation for the real world is a Rational Discussion all its own and then some- TrueSkill and ELO are just two of many advanced systems developed to determine this. We won't bother with either since learning the math and mechanics of those is not our present intent. Instead we'll use a simple system that ranks players from skill 0 to skill 100. Higher is better.

Thus, consider that this strategy requires skill level 25 to execute. To counter this strategy requires skill level 50. Thus, we can immediately see that at low levels of play this will be perceived as imbalanced. However, we deem skill level 50 to be quite attainable by all players with the will to learn and some effort. In an ideal situation, these values are flat- the effectiveness of the strategy does not scale with the skill of the player leveraging it, and thus we can consistently say that as long as a player can reach skill level 50, then he/she can counter it.

Most situations are not ideal. However, of these the easiest situation to consider is that both the strategy's skill and the counter's skill scale linearly. Specifically, if skill 25 is the lowest that a player can have to successfully execute the strategy and skill 50 is the lowest that a player can have to successfully counter the strategy, then a player of skill 26 executing the strategy will require a player of skill 51 to counter it. We can quickly see that this is a problem. How will anyone counter a player of skill 76 implementing this strategy? The countering player would need to be of skill 101, which is by our own definition impossible. Therefore, this is clearly an imbalance. And it is helpful to know that this is the case where both the strategy and counter scale at the same rate, so if the strategy scales at a superior rate then we have the same issue. Therefore, that case is also a clear imbalance.

The more nebulous case is when the counter scales better with skill than the strategy. We then must consider where the two overlap. That is, when the skill of the player executing the strategy can be countered by a player of equal skill. Consider our skill 25 strategy versus the skill 50 counter example again. If we the counter scales normally but the skill only scales at .25, then we find that a skill 59 player will be able to successfully counter a player of skill 59 executing the strategy. Mathematically:

59 - 25 = 34 (that is, the strategy requires skill 25 but our player is 34 skill above that)

50 + 34 * .25 = 50 + 8.5 = 58.5 (if skill 50 counters skill 25 and the strategy scales at a rate of .25 while the counter scales normally, we have 34 extra skill to put towards the strategy which will earn us only 8.5 effective extra skill points; this means the counter will only require a skill of 58.5)

59 > 58.5

Since our rates are currently unchanging, we can reliably state that past skill level 59, the counter will always prevail between opponents of the same skill level.

Of course, we must be concerned with cases where this changeover does not happen until after skill level 100 is hit, and those cases will cause us to conclude imbalance.

However, before we do that we must consider ceilings and changing rates. These are more complex and tougher to convey in simple math, so I will only bother with ceilings which can be simplified rather nicely. Taking the above example again, it's great that at skill level 59 the counter should prevail, but what if the counter actually does not benefit from any skill beyond 55? Meanwhile, if the strategy scales to skill 65... we have an issue. This issue affects not only skill 55 (where we find that the counter will not prevail), but all skill levels above it. If we have two skill 100 players, because of the skill ceilings it doesn't matter- we may as well have a skill 65 player using the strategy and a skill 55 player using the counter.

Now, of course, all of the above comparison of skill and direct imbalance checks are described in a vacuum. In reality, the situation is much more complex and there are potentially hundreds of these scenarios just in a single match. But, again, considering those guidelines can provide for a clear and comprehensive interpretation of our qualitative data.

But again, this process proves and reaffirms that player skill matters when we consider and evaluate balance. And the article at the beginning of this post I feel provides an excellent example of this. Korean SC2 players are of a generally higher skill level than Korean Terran and Protoss players. The causes of this are that more ex-pro Brood War players are active for the race than other races, thereby bringing overall skill up a little bit, but more importantly providing lower-level players with more examples of good, high-level play than any other race. On top of this, Artosis also accurately notes that in any good strategy game, it takes time for players to fully familiarize themselves with the race and to test the millions of combinations and strategies it offers. In the case of the Zerg, they have the fewest combinations and therefore a greater percentage of their combinations/strategies/etc can be unlocked in the same amount of time as can Terran or Protoss combinations/strategies/etc. This all contributes to higher Zerg win rates in Korea, but these do not necessarily indicate an imbalance as for the most part the skill levels of those Zerg players are currently higher than the skill levels of Terrans and Protoss they face.

However, Blizzard has reacted by yanking out the nerf hammer in an attempt to homogenize win rates across races. As we can see from our enlightened vantage, this may be somewhat premature.

I cannot stress enough the importance of understanding the current state of a game's metagame when considering balance. Just because the forums are lit-up with people whining left and right that there is this strategy they do not know how to counter and that no one knows how to counter it does not mean that there is not a feasible counter. Likewise, just because win rates temporarily skew towards one race or another does not necessarily mean that an imbalance exists- it often simply means that the current metagame favors one race over another and the race that is not favored needs to adapt its metagame. Designers all too often respond to these metagame shifts by exerting control over the game to fix what is not broken, often creating new, actual problems. Understanding the contributing factors behind your data mining is huge.

It is also notable that for games in very early stages, such as beta, great caution needs to be exercised with balance changes. Designers need to exercise patience as much as possible, even when the game seems blatantly broken or is not following the original vision. Often, a player will develop a counter to whatever strategy is ailing the game and often this spreads via various means to much of the playerbase and shifts the metagame. The designer's job then is to evaluate that counter, with the tools provided above, to see if it actually does effectively solve the ailment. If not, action can be considered or further waiting may be warranted to see if additional refinements or strategies come into play to affect the situation.

But I think the big take-away from all of this for designers really is that designers need to put more faith in their own intuition and ability to reason through strategies and counters and etc. If you have a continual nagging that the win percentage data and etc you're looking at and that the complaints are just ignoring some piece of design that you or your team have implemented, you're usually right. Trust that, and wait out the storm for the community to come around and for the metagame to evolve. If necessary or warranted, find some way to accelerate that evolution.

I think it's worthwhile to look back to the very beginning of the StarCraft 2 Beta where Protoss players complained they had zero cost-effective counters to Mutalisks at all- we're not talking even real-world countering where GtA units need to worry about terrain versus the mobility of air units and such. Rather, we're talking about straight-up attack-move macro counters even. Eventually, someone noticed that in Blizzard's Help GUI, Sentries were listed as a counter to Mutalisks. Lo and behold, people labbed it and found it to be quite true. Shortly thereafter Sentries became *the* way for Protoss to counter Mutalisks. Blizzard had so much as given its playerbase the answer and yet it still took nearly a week or more of "WTFSUPERRAGE" for the metagame to evolve.

Tangent to all of this is that I am working on a rebalancing mod for StarCraft 2 with some fellow designers as a side project. We aren't pretending to have the data Blizzard has nor do we particularly want it. We are ignoring the metagame as much as possible while still watching replays, streams, etc... and getting plenty of hands-on time with the "actual" game. But our focus is primarily on fixing core issues that we feel impact the game negatively on a design level. Things like the "Evil Triangle" of the Marauder/Roach/Immortal are massive concerns for us, whether they are balanced within the context of the current metagame or not. We are looking to make specific changes to open specific gameplay opportunities for players to make use of- with these tight goals, we can effectively measure our success largely without needing to focus on or care about the metagame. We can evaluate whether our units are performing their roles properly or not and if they are, that backs-up our core, overarching designs which, if done properly should be at least "ballpark balanced". With such assurances, we can then afford to sit back for quite some time to evaluate the actual usage of our changes and can consider informed changes that we have come to after watching multiple changes in the metagame rather than knee-jerking to just a few changes.

To be quite honest, I feel that the utter disregard for balance that went into the original StarCraft and Brood War are what made it great. Balance wasn't excessively micromanaged every day upon ever shifting winds and the game was designed around units that filled roles and filled them well. That led to some absolutely ridiculous balance situations and decisions (nearly every spellcaster in the game is overpowered by traditional standards and in modern days would have elicited the greatest of "IMBA" shrills; and that's before we even get to Vulture Spider Mines, Siege Tanks, Reavers, Lurkers, Carriers, etc...), but because the roles and the designs were balanced, the metagame adapted to support that balance.

That all is certainly a massive wall of text, but for those who've made it this far I'm really interested to hear what you have to say on the matter of this most delicate subject, balance.