Numerical Tuning Lessons

At some point, if you’re doing system design, or anything that butts up against system design, you’re going to end up looking at formulas in an excel spreadsheet. I’ve spent plenty of time over the past 10 years looking at these and tweaking games and figured some lessons I’ve learned would help other people.

Lesson 1: Get Exposed

This is a fairly simple one, but important - if you’re working on a big team, and you aren’t an engineer, make sure the engineers expose as many tuning variables and formulas to you as possible. It is your responsibility, and should be your goal, to understand fully how the game’s systems work. Additionally, the ability to A/B test different values is massively helpful when trying to grok how numerical changes effect the feel of gameplay. If those values or formulas are hidden in code that you don’t have access to, you can’t really fully understand how things really work.

This has worked different ways on different games I’ve worked on - on Dungeon Siege III, for instance, I owned the .cpp files that defined the RPG systems and the numerical rules for them. On other games, these have been exposed in data tables or through script. And, in the worst cases, they’ve been hidden in code that I didn’t have access to and had to look over an engineers shoulder as they made changes for me.

The cases where these worked best were where I could fully see and understand all the details of how the system worked and could freely tweak and iterate on my own. You can’t tune a system you can’t edit, and you can’t truly understand a system that you can’t see. Get those values and formulas exposed! 

Lesson 2: Minimize Floating Variables (or: Only Tune What You Want)

Games, especially RPGs, typically have lots of numbers that interact in complex ways. There are the obvious numbers - HP, damage, statistics, and then there are “out-of-game” values, like expected time to kill a monster, power growth per level, monster kills required per level, time per level, etc, that players never directly see but that influence the relationships between those values.

You should understand the mathematical relationships between the in-game values and the tuned values, and ideally set up formulas in something like an excel sheet, so that you can isolate exactly the variable you want to tune and not affect other things - or else you risk ending up with a “man in the shower” problem, where you tune one number, which causes an overshoot in a second system, when forces you to retune that system, which causes an effect on the first number, etc.

The biggest mistake I have found is that people often want to tune the values that players see, but these are often the least important values in the game! Oftentimes, it’s the “feel” values, things like “how many hits does it take to kill a monster at equal level? At one level above?” are the real questions, and people try to tune values like HP to realize those goals. Instead, I would recommend never tuning values like HP directly and instead setting your equations to allow you to directly change the “feel” values - to make those the knobs that you have to turn - and then letting your formulas spit out the in-game values for things like health and damage. Note that these references can get a little bit circular, and so you may need to arbitrarily set some value - for instance, health is often based on a player’s ability to deal damage * expected kill time, but a player’s ability to deal damage at a given level could just as well be definied from a monster’s expected HP. You may end up just deciding “A level 1 player will do 100 damage per second” or “A level 1 standard enemy will have 100 HP” and that will scale all of the numbers in your system. But from that point forward, you should only tune the individual “feel” values you want to hit directly, rather than tuning the player-visible values in a way that will have knock-on effects in other systems.

Lesson 3: Make Big Changes

This is a pretty common lesson shared by lots of system designers, but figured I’d share it here too - especially when you are doing early tuning, double or halve numbers. Don’t nudge values by 5-10%, because it’s too hard to see the changes and it takes you much longer to understand the effect your change is having. Additionally, it can save a ton of time. If you double a number and it doesn’t seem crazy, you just saved yourself a lot of time slowly nudging up to the doubled number. If it does seem crazy, you can try the halfway point between the doubled and the original value.

Lesson 4: Start with Math, End with Feel

Math, spreadsheets, etc. all make a great starting point for tuning, but never trust that math will produce a game that feels good. For one, your math may be wrong. There are probably variables you aren’t accounting for, especially in a game with any action-ey elements. Player skill is a thing. Math is super useful for establishing a baseline, but never, ever trust that your math is correct when playing the game is telling you something different. Additionally, math isn’t going to make your boss feel good, and late in the day, don’t be afraid to change things outside of your formulas to get the feel changes you want - but DO save those changes for late in the process, when you’re sure that your core feel variables (like your time to kill) aren’t likely to change. Otherwise, those big changes can completely overwrite or blow away your fine tuning changes.