Question: Problem areas & Metric tracking

#1

Hey guys!

We’re getting a lot of great feedback on this forum about version 1.0.1, and rest assured I’m diving into the bugs as we speak. One thing has stuck out - it seems that there are some random difficulty spikes and other little problem areas that people are running into, that are kind of hard for us to predict and foresee. Furthermore, it’s always tricky for us to know whether to “spot-balance” issues brought up in the forums because we’re not sure how widespread the problem is. Sometimes it’s obvious and we just do it, but other times we’re like - “was this just a problem for this one person’s strategy and party build?”

Here’s where I’m going with this - you might have noticed in the game options that we track some anonymous metrics (you can turn this off by selecting “Don’t track me, bro!”). Right now we’re using Playtomic, and it’s not really given us any super useful information because I didn’t know what to look for when we started. I’m thinking of designing some “actually useful” metrics to get some broad data about how people play the game, so we can use that as additional information when balancing.

The main thing I’m looking for is battles that are too hard for their sequence - ie, extreme challenges that are way too tough to beat at level X, but the next two extreme challenges on the map are just fine at that level. I need a way to measure for this kind of thing that’s easily trackable, especially over large numbers aggregated together or charted on a graph.

Here’s an idea I have:

Track what level Azra is at when a certain level challenge is beaten

This would produce a graph kind of like this, I imagine (totally fake mockup):

This would tell me right away that a lot of players had to skip the extreme challenge on battle 5, then come back to it later. This would be tracked so that we could chart NG+ separately from normal game, as well as just chart all the battles of the same challenge level on a single chart. We don’t need the line to be perfectly smooth - even small spikes are okay by us, we just wouldn’t want random cliffs popping up that we can’t see because we haven’t measured things.

Stuff like that.

#2

That seems good. You’d tell when people had to grind. You could also track total respecs and defeats and scrap spent, so you could see when people had to redo their characters a lot or lost a lot. I’d imagine there would be some false positives from the level up people as a lot of people would chose to go back and grind at similar points.

Edit. How about some direct in battle tracking? Note the player level and which unit makes them lose health. That should make fixing any problems a lot easier. If you find that players level 24-27 lose an average of 700 health to round 6 units in mission 25 you can fix the problem. If that’s too complex you could just note which round they lose health in. For complete tracking, track average psi per round, deaths per round, and spells per round. That should give you a very clear indication (with just a little data) of any problem areas and allow same day fixes that don’t majorly change the game.

#3

That’d be useful after it’s beaten yeah.

Number of resets(after a time out, say 10 second to catch the people who reset because of misboosting prebattle) per challenge would be good and would let you catch it more easily if more people are like me and reset or quit the level if ythey’re going to take even one hit.

Scrap earned/on hand each time you get to/stop at a shop.