Dev Diary #6: Balancing the Sides

One of the three sides in my game lost to another in 43 matches out of 48. I checked five reasons that seemed obvious, and not one of them moved the result by more than a single win. This part is about how I balance the sides with the help of bots, and how this story ended. A reservation right away: the whole design of the sides is a draft for now, the names, the units and the numbers will still change, and everything I describe is an intermediate state.

The concept shows the three sides: AEGIS on the left, VARDEN in the middle and PACT on the right.

How the Sides Differ

I didn't want the sides to differ only in the price and strength of units, because then they play the same. So each one has its own "verb". AEGIS sees and hits precisely: it has few units, and they are expensive, slow to build and tough. PACT spreads out and replaces: it has many cheap units, builds fast and wins by numbers. VARDEN holds and presses: its strength is slow but late. I also spread the sides over the length of a match: PACT is strong at the start, AEGIS in the middle and VARDEN at the end. This formula is also a draft for now, and it is exactly what I'm testing with balance.

How Balance Is Tested

I have no live players to test balance, so I play with bots. These are the same bots I described in the last part, only in the simplest mode, where they play by rules. A match runs without graphics, and fifteen minutes of play are calculated in a few seconds. For every pair of sides I play 48 matches: four seeds, six maps and both starting positions. During a match the game records income, army value and losses every five seconds, and from these records I can later work out where exactly a side loses. The matches are deterministic, so a repeated run gives the same numbers, and as a control I every time run the pair of sides I didn't change. A difference of fewer than five matches I count as noise.

The screenshot shows the VARDEN base in the game, the bot is defending. The buildings and units are placeholders for now, and I watch such matches as a replay with the thought process panel.

What Went Wrong

After I moved the unit numbers from the table into the game data, VARDEN stopped beating AEGIS: 0 wins out of 48 and 43 losses, whereas before the move it lost 17 matches out of 48. Against PACT it was no better, 28 losses out of 48. My hypothesis was this: VARDEN is slow and late by design, and it gets killed before its strength has a chance to show.

The first thing I checked in the records was whether VARDEN is late with construction. It isn't: the power plant, the supply center, the barracks, the factory and the first units appear within five to ten seconds of AEGIS. The problem turned out to be in the window between the third and fifth minute, when the first fights happen.

The chart shows the army value by minute in the lost matches. By the third minute AEGIS already has a noticeably larger army, and between the fourth and fifth minute VARDEN loses almost half of its own. After that the gap only grows: by the sixth minute the income of AEGIS is almost twice as high, because it hauls resources with four helicopters that VARDEN units can't reach, while VARDEN has two or three trucks that are easy to kill. The most expensive VARDEN units don't show up before it dies at all. In all 43 lost matches the bot never built the Rubezh and the Press, and built the Dome once: it simply doesn't live to the moment they are needed.

Five Hypotheses That Didn't Work

I checked five obvious explanations by changing the data one at a time. I made the power plant cheaper and faster, returned the old price and speed to the truck, increased the health of VARDEN buildings by fifty percent, put a heavier machine in the bot's army list and, the other way round, weakened AEGIS. The result was the same: not a single change moved the number of VARDEN wins over AEGIS by more than one out of 48. Some of them helped a lot against PACT, but could not handle the main problem.

Then I took a different path and started returning the numbers to the original ones one by one. If I return the old numbers of AEGIS, nothing changes. If I return the old numbers of VARDEN, it wins 5 matches against AEGIS and 31 against PACT. So the skew was created by the VARDEN numbers, and the cause isn't one figure but a sum. If I reduce build time and price and increase damage and health at once, VARDEN wins 17 matches against AEGIS, and if I remove only damage from that set, just 2. Damage turned out to be the main lever.

A Moderate Fix

The strongest option would have been to simply make VARDEN win, but then the idea of "slow late power" would be lost, and the side would become just one more set of strong units. So I chose a moderate fix: I raised the damage of the VARDEN combat units and lowered their price, and didn't touch the health and build time. That is nine units. In the first pass the damage grew 1.2 times and the price dropped to 0.9, in the second 1.3 times and 0.85.

The chart shows the result. Against AEGIS the VARDEN wins grew from zero to one and then to six out of 48, against PACT from seven to fifteen and then to sixteen. The median time of defeat shifted from 8.3 to 9.0 minutes. The VARDEN army by the fifth minute against AEGIS grew from seven to twelve units. The first pass against AEGIS gave almost nothing, zero wins turned into one, and that is noise. The second pass is already noticeable, but it's far from equal chances: AEGIS still wins 32 matches out of 48.

What I Don't Know Yet

First, these are bots on rules, not live players and not neural networks, and with them the picture may differ. Second, 48 matches per pair is not a lot, and a difference of five matches or fewer is noise. Third, the bot doesn't build the Rubezh in any match, so I changed the number for it, but it doesn't affect the result. And fourth, the main question is unresolved: I don't know whether the goal should be parity with AEGIS in the first eight to ten minutes. To get it I would need a package of changes that includes health and build time, and that already changes the idea of the side. For now I've stopped at the moderate option and postponed the decision until I see the side in live matches.

What's Next

The changes are in a separate branch for now and are not merged into the main version of the game. In the next part I'll talk about what I've shown in pictures many times but haven't told yet: how units go from concept to a model in the game.