Debugging Edge Cases

Right now I am trying to close the most visible of bugs and unwanted issues to move on to more interesting aspects of the game, namely environmental hazards, but certain things are harder to catch and solve than others.

Procedural generation of contents introduces a specific kind of problem that is not easy to understand and solve, what may be called edge cases. Edge cases are an engineering concept for something that happens at the “boundary”, but not quite over, the set of possibilities of a system or, in this case, algorithms.

Empty Level

An empty level, a type of edge case

The screenshot depicts an empty level, which is a quite egregious display of edge case.

As you can guess, the problem with these is that they are not easily reproducible. It is not even a bug in the strict sense: the algorithm has done its work and it just so happened that with certain starting conditions the only acceptable solution was an empty level.

Another edge case that Daniele and I found is that certain levels do not generate enemies in them. I suspect that this one may be a concrete bug, although it seems strange nonetheless.

As a system or algorithm grows in complexity, designing against edge cases can be very difficult. A good mixture of  experience and strong design can help in minimizing the risks but sometimes it just pushes the “edge” further until catching them early on becomes nigh impossible.

For now what I can do for Wizards’ Duel is try to be catch the most evident of these edge cases and place safety nets to ensure that certain conditions never happen. Later on, when the engine is more consolidated, I will try refactoring and simplifying the most complex part of the algorithms, since simplicity usually gets along well with correctness. Finally a good round of beta testing will hopefully iron out the last of these issues.

It is my first time with a project of this size so I am still struggling to find a good way to catch these in advance or design around them as much as possible. If you want to share how you manage edge cases or some funny/interesting one feel free to leave a comment or share them with us on twitter.

Thanks for reading.

3 thoughts on “Debugging Edge Cases

  1. Is your definition of an “edge case” something that happens with a very small probability? When you say “concrete bug”, what is the opposite of that? I use the term “edge case” pretty often, but it seems like it may be in a different sense that you are using it. I use it to mean a scenario that a user is not likely to encounter because it involves a uncommon set of steps.

    I don’t know the details of your game too well, but I would say the only way you can really knock them out is to make automated testing, so you can quickly run thousands of cases through the system and verify if there are any bugs. But that may be very difficult to implement, depending on your game’s complexity and the engine you are utilizing. However, without such a system, even if you spend weeks on catching every possible bug, changing a few lines of code can introduce a hundred more bugs (in theory), and you have to start the bug fixing process again. So for large-scale projects I think automated testing is the only way to have a very high quality product.

    • I think our definition matches. Edge cases for me are something that happen very rarely and are technically correct but otherwise unexpected. Another possible example for which I don’t have a screenshot is a dungeon configured by a single long corridor: given how the layout is build from blocks it can happen that only corridors are picked during generation. If is something that a human must consider and guard against by creating specific rules checked during creation, for example as you suggest using automated testing against a set of requirements. I think my biggest challenge is bias. Empty level or corridors levels is something that I can expect and code against, but maybe there is something else that eludes my sight since I have this cognitive bias coming from my understanding of the algorithm. I hope it makes sense. Do you know of any metrics that may be applied in this case?
      Also thank you very much for your insight!

      • I don’t know the details about your algorithm, so I can’t really make a concrete suggestion. However, if you can’t completely define what “good” levels are in some form of code, then your only option is to generate a bunch of them, check them by eye, and then tweak the algorithm based on what you find.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s