because seeing what kinds of problems are caused by the changes at that skill level is very different than seeing the problems at lower skill levels.

You almost can't look at low skill levels when determining balance, because at low skill levels essentially everything is overpowered.

Overpowered is only in relation to two equal skills levels. Something could be considered overpowered to two low skill leveled players but not to two high level skilled players.

But that's not really a problem? If thw solution to something seemingly overpowered is to learn a counter or alternate strategy, then the developer needs to do nothing.

Are you saying that if a weapon is OP when two newbs are playing with it that it's not a problem? OP is OP regardless of skill level and should be addressed.

I guess all basketball hoops should be 4 feet high then.

It's quite basic, if you let people of all skill levels play in the same arena, then you have to balance for all skill levels. If you want to make a MLG subset of the game, then go for it.

Generally I think you want to balance for MLG players. As time passes, players' skill should increase, bringing all players more in line with your design plan, just as their gear, spec, and choice of weapons will refine down to your designed for ideals as well.

It's pretty much a given that a full-auto weapon will have a lower skill threshold than a burst weapon, which is lower than a semi auto weapon. At low skill levels, then, you expect to see a lot of ARs and few pulse rifles or hand cannons. At high skill you want to balance such that the opposite is true. In a perfect world the game is narrow, such that a HC user is only marginally more effective than an AR user, and will still lose fights when the AR user gets the drop on them or is an exceptionally better player in other ways (maneuvering, tactics, use of cover) etc..

The goal, generally, I think, is for players to use the low skill guns until they get a moment of inspiration of "I think I could make a high skill gun work" and then those players should find, after much practice, that they are marginally more successful with the high skill weapons.

I'm of the opinion that Destiny (pre-patch) suffered from having too wide a gap between pulse rifles of the dragon archetype and other weapons, shotguns notwithstanding. I haven't played post-patch, but I dislike the general limitation of usable weaponry from nerfing special ammo availability. I think it changes the game to be even more about map control and camping and less about aggressive assaults and blazes of glory.

Wins is a good metric. You also want to be wary of statistical artifacts. Back when I played WoT, the player balance in game suppressed the russian tank stats, so they constantly got buffs or had their most OP tanks left alone. The US tanks, with their 5% of the overall population, had major over representation in the stats and were always getting nerfs to their already tough play styles.

Skill gaps can complicate statistical analysis quite a bit.

A good example in Destiny would be a weapon like No Land Beyond. In the hands of a poor-to-average player, it is almost useless. But put it in the hands of a very skilled player, and it can dominate the match. So while it might not see widespread use the way Mida or TLW do, that doesn't mean it is underpowered or needs a buff.

Agreed. Quickscoping is an interesting thing. When CoD: MW came out, it wasn't on the designer's radar at all. They didn't think it was a possible thing so they didn't design around it. None of the testers did it either. And then some guy on the internet posted a video of him quickscoping the shit out of everyone and - holy cow - it became a thing. Now, of course, designers design around it, making weapons specifically for quickscopers (and balancing them accordingly as very high skill = high reward weapons).

Weapon balance in most FPS games is such that you balance for the professional tier of players and let the trickle-down low skill weapons fall where they may. This is why I really liked vanilla destiny's pvp balance. At low skill levels the AR was king because it was the easiest gun to use. As you got better and better it became more and more about the slower RoF weapons, ultimately landing on shotguns and snipers as the be-all-end-all weapons in the game. Somewhere along the line some of the lower skill guns got buffed and became low-skill = high reward, while the higher skill guns were nerfed down to high skill = above average reward. That was then overcorrected such that high skill guns became the only viable guns at all for a while, then overcorrected again to the pulse rifle meta we've had for a few months, and now we're going to (I predict) hand cannons and scout rifles meta simply due to starving everyone for special ammo. We'll see. My god roll Spare Change .25 is going to get dusted off soon, and maybe it'll be her time to shine.

Titanfall is an interesting outlier here in the realm of weapon balance, as they generally give you the best guns in the game right off the bat, but then encourage you to use worse options in order to level up more quickly. It's really smart. Since everyone has access to top tier guns, there's no need to balance the rest of the weapons - just make sure they're worse than the best guns and you're good to go. People will still use them if they want to optimize leveling up over getting kills. Very, very clever.

There's a third, sadder option, and that's that bungie is using a metrics-bases approach to game balance. This seems to be the case, since they LOVE to trot out the metrics and show us charts about which classes have the higher K/Ds etc.. The problem with a fully metrics-based approach is that if your metrics are off at all, you screw up the balance. For example, Bungie is showing us that Defenders have the lowest K/D and says "so they need a buff." No. No they don't. Your metrics aren't looking at how having a defender on the team improves the K/D of the surrounding players. K/D is not everything. If you pretend it is, then your balance will suck. This is, imo, the most likely scenario. Everyone means well, but the methodology they put in place to determine balance is fundamentally flawed and no one is going to bother to fix it at this stage in the game's life. Des2ny *might* go differently, but I have zero hope that D1 will. It's just too much work to fix with too little return.

We at least have an idea that Bungie looks at more than just the stats. Your (hypothetical?) example of the Defender, for instance, is exactly 100% opposite of what actually did happen. Bungie acknowledged that the Defender was getting a lower k/d in their subclass chart but then noted that teams with Defenders tended to fare better in terms of wins. That's why the only buffs the Defender got was an instant restock of grenades and melee when using Ward of Dawn instead of something more significant.

Another example was a refusal to do much to change the MIDA Multi-tool. They said that yes, it was pretty good and (at least at one point in time) was being widely used, but even though the chart showed it to be a gun that was getting used a lot they also recognized that it was actually a pretty well balanced gun that didn't need a nerf.

Totally agree though that they should update more frequently. This whole thing with the Clever Dragon being superior at any range for months really got on my nerves. But I don't think your sad third option is the way things actually work at Bungie. At all.

It's totally fair to think that, and I see your points. I have a differing guess as to what's going on, but they're all guesses. I'm an expert at testing methodology, but am also not a mind-reader or soothsayer and I have no idea what is truly going on behind closed doors. I strongly doubt it's the sort of machiavellian conspiracy that reddit sometimes postulates. I also doubt that Bungie is following what I would consider to be best practices when it comes to design, testing, and implementation of an FPS PvP test plan. For what it's worth, I'm certain that my idea of best practices is not the same as anyone else's idea; this is all just opinion and conjecture with no real evidence to back up any of it.

That being said, I think your MIDA example is an interesting test case. You're using it to illustrate that a metrics-based approach showed MIDA is fine. I accept that. I do, however, believe that the fact that MIDA is fine is being used to ignore the fact that, in all cases, MIDA is a superior scout rifle to every other scout rifle in the same damage archetype. That, I believe, is not at all fine. Why have any legendary low damage scouts if they are just completely outclassed by this one gun? Why balance the entire archetype around an exotic, locking out players who wish to use a low damage scout from using an exotic special or heavy weapon if they want to be as powerful the metrics indicate they should be in pvp?

Of course, I'm assuming Bungie would want legendary scouts in the low-damage archetype to be viable because I would want legendary scouts in the low-damage archetype to be viable. This appears to not be the case.

Occam's razor tells me that either Bungie's design goals for pvp are for the number of viable weapons to be a tiny fraction of the number of available weapons (in which case they're pretty much nailing it), or that Bungie's play balancing practices leave much to be desired.

Since the communication out of Bungie has really told us nothing about their pvp sandbox philosophy, we don't know if they want pvp to be "bring your favorite guns" or "bring the guns we've decided should be viable" or even "there should only be one viable spec per class, gun per slot, and style of play, period." We don't know. I think the first option is the most fun and the most inclusive, thus encouraging the most players to play the pvp game, and so that's the one I am constantly rooting for, but - as you say - I may very well be in error where the truth is involved.

No but seriously, thanks for the insight. Do you think the noobs at Bingle might have had too much say at the beginning, and when they realized how quickly the playerbase got good, they stopped listening to them, and that's why the game is getting harder and harder with worse weapons?

Yes, you probably could. I'm serious when I say that finding someone who is great at having "new player eyes" at a game is very, very hard to do. The few guys with that talent usually rotate between projects so as to keep them even fresher, and because there aren't enough of them to go around (when I was at activision we had about 300 testers and maybe 3 guys who were good at new player experience testing).

I do not think that the noobs at Bungie had too much say. If anything, I think they had not enough. The new player experience of this game is pretty confusing and rocky. Imagine a world where you're brand new and know nothing of the grimoire. The game doesn't make a lick of sense. The player control is also incredibly well refined, which requires a degree of skill in the game to even recognize, let alone properly test. Bungie has shown that they pay attention the game control down to the frame (as they should), and that means there are most certainly expert players giving feedback every day.

The difficulties of having an all-expert team, however, are that - especially in a game this old - everyone gets a favorite class and spec and gun and playstyle and they will say and do anything to keep their build on top. You start having to force people to test, say, high RoF shotguns or medium damage fusions rifles, and they resent you for it expressly because they aren't as good with those guns (or those guns aren't as good and they know it), and then they give misleading feedback (not intentionally, but as a result).

Hopefully you have a team that is constantly changing their builds, gear, specs, control schemes, etc.. to keep things new and fresh, but I can't say if they are or not. It does really feel like the balance changes are done in a fairly tone deaf way that is less interested in actual gameplay outcomes and more interested in metrics-based adjustments. Pure speculation on my part, however.

Keep in mind that the designers may very well feel like the best pvp game is one without fusion rifles, and so they intentionally nerf those all the time. I can't say. I happen to like a game where every weapon can be top tier, but I also know that's a pipe dream in any game. Still, I hope every FPS designer thinks to himself "Every weapon in my game should justify its own existence there, and not simply to illustrate how much better some other gun is."

There's a third, sadder option, and that's that bungie is using a metrics-bases approach to game balance. This seems to be the case, since they LOVE to trot out the metrics and show us charts about which classes have the higher K/Ds etc.. The problem with a fully metrics-based approach is that if your metrics are off at all, you screw up the balance. For example, Bungie is showing us that Defenders have the lowest K/D and says "so they need a buff." No. No they don't. Your metrics aren't looking at how having a defender on the team improves the K/D of the surrounding players. K/D is not everything. If you pretend it is, then your balance will suck.

Why not look at WINS then? Especially in certain gametypes, K/D is less meaningful. They should see which subclasses can't win games. Ultimately, that's what matters in the end when it comes to balance.

Wins is a pretty good metric, but the fact of the matter is that no single metric is useful enough on its own to justify a sandbox change, unless the margin is massive. Like, for example, if Defenders had a 0 K/D across the board, they'd obviously need changes. I don't really want to say "a buff" because maybe their teammates never die or something, but it's a clear sign that something is broken (again, unless the design intent is for them to never be able to kill anything). In team gametypes, it's also tricky because unless wins increase with the number of, say, gunslingers on a team, all the way to 6 gunslingers having the highest win percentage, there's usually some break-even point where adding more of a single class and/or spec actually hurts the team rather than helps it. Positive reinforcement for same-class-ness is good, but you also want to counter it with an opportunity cost for combo play between classes and specs. In short, you want your players to feel like they can bring pretty much whatever class and spec they want into a game and be successful. At the highest tier of play, you want teams to be able to plan class and spec composition for ideal synergy, but you don't want that to be more important than basic game skill.

For balance testing, you want to eliminate variables, so a test plan for, say, class balance (and I'm just pulling this out of my ass, but this is basically how I'd go about setting it up), would be to start with one class and make my guys all play that one class, with identical specs, for several games of rumble. This gives us a baseline for "player X is better than player Y in this scenario." Then we take a player who landed in the middle of the pack and have him change his spec to our designed "ideal" spec, and then run the same number of matches again. Did that change affect his standing in a meaningful way? Now let's have everyone go to an ideal spec. Are we back to the original numbers or did we find a magnifying or minimizing result in the variations? This is where the internet is hugely useful, because you can quickly determine what specs people view as the most powerful, regardless of the intent of the design (and you can compare the two if they don't match). We'd want to do that with an intentionally stupid spec as well, and then with some random specs and some "try this and see if it works" specs, too. After running, say, gunslingers, we'd then run this whole test with all bladedancers, and then all nightstalkers (identical weapons for all at every step of this process). Then we'd do the same for titans and warlocks. Only after all of this would we take our testers into class vs class/spec vs. spec games. Again, the internet is helpful for metrics here, but you have to normalize for popularity of classes, whether or not the guns have ideal rolls, etc.. So now you can tweak the classes around to get them all behaving about how you want them to relative to one another.

Next you have to tweak guns. Start with designed perfect rolls on everything and balance the entire weapon class assuming such a roll (because players *want* ideal rolls, and don't want crap rolls, it's safer to start from a place of perfection than to start from a place of mediocrity or crap - in a long enough timeline, everyone will end up with a perfect roll eventually, so that's where balance should be done). For weapons testing you would want to make sure everyone has identical classes, so you can, again, eliminate variables.

Then you need to balance the weapons with the class abilities... This takes a very long time.

Throughout this whole process, I'd probably be reassigning the very best testers (not necessarily the best players) to separate playtest teams, letting them get creative in their builds to see if they could find class and weapons combos that would break balance. And of course we need to stay in constant communication with the pvp sandbox designer to ensure that our sense of balance is meeting the design goals and vice versa.

So there's at least a month of testing for initial balance. BUT... once you attain that, you can make small tweaks on a weekly basis to reduce anomalies. Maybe ARs are getting way more kills than you expected. Adjust the range 1 yard closer. Or very slightly increase the recoil spread. If it's not enough, you can make further tweaks next week, and so forth, until the gun is performing where you want it to.

Generally speaking, I find Bungie's MO to be giant massive huge sandbox changes every few months, which results in essentially a reset of the entire testing process and the meta shifting largely to favor (or not) certain weapons. This would not be my desired approach. Testing for PvP is never good enough to match the real world, which is why I don't like large changes to the meta in any patch nearly as much as small, incremental changes.

That so-called "pro-team" did exactly what I was afraid they'd do when I first heard of them. They put way too much emphasis on ranged weaponry, short TTK, and fast movement, forcing changes to all non-ranged weaponry. They butchered the sandbox, and the game just feels like a poor man's Black Ops 3. And I'm a higher-tier player, so I was able to survive well enough in the game, but it sure as heck wasn't Halo anymore thanks to the "Pro Team".

CODY: You can argue that's exactly what they wanted. I'm not saying it was right or wrong, since I don't own or play Halo 5, but tastes and trends do evolve over time. Especially given that MCC ostensibly preserves the classic experience, it would be kind of foolish to keep everything the same. New game, new feel. Halo 2 sure didn't feel like Halo in multiplayer. It is an okay to evolve the game feel, just as it is also okay for players to prefer something else. The mere fact that there is change is not nessesarily a point to criticize. Not being a player I have no idea if the changes work within the framework of the game or not, but the response has been mostly positive anecdotally.

I'm with Cody on this. They wanted to move the franchise mechanics forward. The speed of the game was part of that. I think they did very good job of doing that without breaking it (compared to Halo 4 IMO). It still has one of the lowest TTK in any major FPS. I think the sandbox is one of the most solid in terms of balance (They don't have to have a Twitch stream to announce all their balancing changes lol). I'm not sure how much you played, but after playing pretty consistently, I think each weapon is unique and useful in certain circumstances now (Halo 4 did not do this).

(And then 343 started adding in variants of every weapon, so half the time, if you find a weapon on the battlefield, you have no idea what to expect if you pick it up. Will this SAW fire normally, or will it shoot Christmas-themed confetti? Who knows! Sandbox balance!)

Again, I'm not sure how much you played but these are actually very rare in competitive. Obviously they're predominate in Warzone and Fiesta, but their might be one on each arena map; so probably not half the time ;) Each one has a visual difference from the default weapon, so if you know them well enough you'll know which version it is. And they can actually balance the map much easier without changing how a weapon works every three months.

It took me coming from Destiny PvP to really appreciate how good Halo 5 PvP is. I like both in their own right, but Halo's is just more solid for team based, balanced MP.

Wins is a good metric. You also want to be wary of statistical artifacts. Back when I played WoT, the player balance in game suppressed the russian tank stats, so they constantly got buffs or had their most OP tanks left alone. The US tanks, with their 5% of the overall population, had major over representation in the stats and were always getting nerfs to their already tough play styles.

Skill gaps can complicate statistical analysis quite a bit.

A good example in Destiny would be a weapon like No Land Beyond. In the hands of a poor-to-average player, it is almost useless. But put it in the hands of a very skilled player, and it can dominate the match. So while it might not see widespread use the way Mida or TLW do, that doesn't mean it is underpowered or needs a buff.

Sniper rifles in some Destiny players hands are OP, but I can't hit the broad side of a barn. What do you do, should you buff them, nerf them?

You can make a game for streamers and top 1% of players, but that may not be a game the rest of us want to play.

There's a third, sadder option, and that's that bungie is using a metrics-bases approach to game balance. This seems to be the case, since they LOVE to trot out the metrics and show us charts about which classes have the higher K/Ds etc.. The problem with a fully metrics-based approach is that if your metrics are off at all, you screw up the balance. For example, Bungie is showing us that Defenders have the lowest K/D and says "so they need a buff." No. No they don't. Your metrics aren't looking at how having a defender on the team improves the K/D of the surrounding players. K/D is not everything. If you pretend it is, then your balance will suck. This is, imo, the most likely scenario. Everyone means well, but the methodology they put in place to determine balance is fundamentally flawed and no one is going to bother to fix it at this stage in the game's life. Des2ny *might* go differently, but I have zero hope that D1 will. It's just too much work to fix with too little return.

We at least have an idea that Bungie looks at more than just the stats. Your (hypothetical?) example of the Defender, for instance, is exactly 100% opposite of what actually did happen. Bungie acknowledged that the Defender was getting a lower k/d in their subclass chart but then noted that teams with Defenders tended to fare better in terms of wins. That's why the only buffs the Defender got was an instant restock of grenades and melee when using Ward of Dawn instead of something more significant.

Another example was a refusal to do much to change the MIDA Multi-tool. They said that yes, it was pretty good and (at least at one point in time) was being widely used, but even though the chart showed it to be a gun that was getting used a lot they also recognized that it was actually a pretty well balanced gun that didn't need a nerf.

Totally agree though that they should update more frequently. This whole thing with the Clever Dragon being superior at any range for months really got on my nerves. But I don't think your sad third option is the way things actually work at Bungie. At all.

There's a third, sadder option, and that's that bungie is using a metrics-bases approach to game balance. This seems to be the case, since they LOVE to trot out the metrics and show us charts about which classes have the higher K/Ds etc.. The problem with a fully metrics-based approach is that if your metrics are off at all, you screw up the balance. For example, Bungie is showing us that Defenders have the lowest K/D and says "so they need a buff." No. No they don't. Your metrics aren't looking at how having a defender on the team improves the K/D of the surrounding players. K/D is not everything. If you pretend it is, then your balance will suck.

Why not look at WINS then? Especially in certain gametypes, K/D is less meaningful. They should see which subclasses can't win games. Ultimately, that's what matters in the end when it comes to balance.

"How can they fix it if they're so bad?"

The devs rarely, if ever, play their games in a competitive way. They usually want to see if the new thing they did works, or see the new thing someone else put into the game. There are usually a handful of exceptionally high skill devs, and those guys are typically of the following varieties:

- Top tier god level player who doesn't understand why other people can't beat his "simple" challenges which require pixel-perfect accuracy and millisecond-specific timing.

- Really really good players who insist on rebalancing the meta to fit their exact play style, letting them feel like god tier players when they really aren't. These guys often reinforce the god tier dev's opinions: "See? When shotguns were the only meta, Jim could finish my challenge, but now that you broke weapon balance he can't." But the actual problem is that the challenge is too hard. I hope that explanation makes sense.

- Terrible players who don't know why they're even working on an FPS. They just want to dance! These guys often have great insight, since they're always coming at the game from the perspective of a noob, but they also frequently discount their insight as "I'm just not very good" instead of truly identifying what's wrong. I include these guys in the "high skill" category here because they are incredibly highly skilled at being noobs, which is actually very, very difficult (possibly the most difficult thing) to do in terms of playtesting.

"Do they even test this?"

Yes, I'm sure they do. We playtested the crap out our games at every company I worked for. Balance is always being tested. So what gives? Honestly, I don't know. Hazarding a guess, I'm going to assume that bungie does not give its playtesters any respect when it comes to design elements. It certainly looks that way to me. Bungie also seems to ignore the meta when making changes, and seems prone to blanket nerfs once every few months rather than small adjustments on a more frequent basis (which is my preference in all cases). It's also possible that most people respect the hell out of the playtesters, but one head honcho type guy absolutely refuses to listen to reason and dictates "my way or the highway" to everyone beneath him. This would mesh with the shakeups we saw before the game launched, as well as changes since launch on a grander scale. It might not even be a Bungie employee; this could be edicts from Activision on High. I would also believe that (though, having worked at Activision and knowing Vohnderhaar personally, I would be very - VERY - surprised if it came out of his office. Either he's not involved in Destiny or it's someone over his head making the calls).

There's a third, sadder option, and that's that bungie is using a metrics-bases approach to game balance. This seems to be the case, since they LOVE to trot out the metrics and show us charts about which classes have the higher K/Ds etc.. The problem with a fully metrics-based approach is that if your metrics are off at all, you screw up the balance. For example, Bungie is showing us that Defenders have the lowest K/D and says "so they need a buff." No. No they don't. Your metrics aren't looking at how having a defender on the team improves the K/D of the surrounding players. K/D is not everything. If you pretend it is, then your balance will suck. This is, imo, the most likely scenario. Everyone means well, but the methodology they put in place to determine balance is fundamentally flawed and no one is going to bother to fix it at this stage in the game's life. Des2ny *might* go differently, but I have zero hope that D1 will. It's just too much work to fix with too little return.

Anyway, that's all just guesswork. The point is that I firmly believe there are testers who know that the play balance is fucked. There are coders who know it, too. I feel like the corporate culture around the game is the real hindrance, preventing positive changes, probably despite the best interests of everyone involved.

We can also infer from the way that most changes are blanket changes that the code of Destiny simply doesn't allow for weapon specific tweaks outside of exotics. That means whole classes of weapon have to change, which becomes a damned if you do, damned if you don't scenario. Poor design at a fundamental level, most likely due to the PvP being an addition to the PvE game and not the basic building block upon which the rest of the game was constructed.

Anyway, if anyone has balance questions or playtesting questions (including methodology), feel free to ask me. I spent a lot of time making games and have very strong opinions on them as a result of my experience, as I'm sure you all know.

You can argue that's exactly what they wanted. I'm not saying it was right or wrong, since I don't own or play Halo 5, but tastes and trends do evolve over time. Especially given that MCC ostensibly preserves the classic experience, it would be kind of foolish to keep everything the same. New game, new feel. Halo 2 sure didn't feel like Halo in multiplayer. It is an okay to evolve the game feel, just as it is also okay for players to prefer something else. The mere fact that there is change is not nessesarily a point to criticize. Not being a player I have no idea if the changes work within the framework of the game or not, but the response has been mostly positive anecdotally.

IMO, Halo 5 has different strengths and weaknesses than previous Halo titles, but it is a rock-solid PvP game in its own right. It plays very well.

because seeing what kinds of problems are caused by the changes at that skill level is very different than seeing the problems at lower skill levels.

You almost can't look at low skill levels when determining balance, because at low skill levels essentially everything is overpowered.

Overpowered is only in relation to two equal skills levels. Something could be considered overpowered to two low skill leveled players but not to two high level skilled players.

But that's not really a problem? If thw solution to something seemingly overpowered is to learn a counter or alternate strategy, then the developer needs to do nothing.

Are you saying that if a weapon is OP when two newbs are playing with it that it's not a problem? OP is OP regardless of skill level and should be addressed.

I guess all basketball hoops should be 4 feet high then.

It's quite basic, if you let people of all skill levels play in the same arena, then you have to balance for all skill levels. If you want to make a MLG subset of the game, then go for it.

- Reduced increased time to aim down sight by 20/25% with rangefinder

because seeing what kinds of problems are caused by the changes at that skill level is very different than seeing the problems at lower skill levels.

You almost can't look at low skill levels when determining balance, because at low skill levels essentially everything is overpowered.

Overpowered is only in relation to two equal skills levels. Something could be considered overpowered to two low skill leveled players but not to two high level skilled players.

But that's not really a problem? If thw solution to something seemingly overpowered is to learn a counter or alternate strategy, then the developer needs to do nothing.

Are you saying that if a weapon is OP when two newbs are playing with it that it's not a problem? OP is OP regardless of skill level and should be addressed.

I guess all basketball hoops should be 4 feet high then.