Note: This is from a series of articles that outlines the things I've learned while making Chef Wars for 2+ years.

TL;DR

• We used a risk adjusted estimation system that produces near accurate estimates we can confidently rely on.

I usually dreaded being asked how long a programming task will take. I always seem to have the knack to overshoot no matter how hard I try. This is an all too common scenario that programmers face and is something that is considered to be a difficult, if not impossible, problem to solve.

This all changed when I was introduced to a helpful system that helps in producing estimates that are "accurate enough". I don't think it has a name yet but my colleagues who introduced me to it says that they got it from a Gamasutra article by Eric Preisz of Garage Games. Since then I've used this system in all my game development related projects and has helped us immensely in the development of our recent game, Chef Wars.

I'm sharing this in the hopes that it would help others too.

Risk Adjusted Estimates

The basic idea of this system is that for each task, an estimated time would be given along with a "confidence level". The lower the confidence the more padding is added automatically to the estimate for that task.

It's a very simple system and is illustrated clearly in the image below:

A legend (shown below) is used to help programmers determine what confidence level to specify based on their current situation.

The formula for calculating the risk-adjusted time is also very straightforward:

(estimated time * (10 - confidence level) / 9) + estimated time

From hereon you can easily compute for the total time and make a comparison between the estimated time and the risk adjusted time.

To see how all of this works you can check out our Technical Estimate Template Sheet at our Google Drive. Or if you are into Emacs, I also have a template for that as well using OrgMode.

How effective is it?

The following is taken from the technical estimate I made for a recent module in Chef Wars. I've logged the actual time it took me to finish the task so the results can be compared.

Note: Each estimate is in hours.

As you can see that the actual times are very close to the risk adjusted times. I overshot quite a bit during the UI related tasks but the time lost was offseted by the other tasks as seen in the totals.

Take note that this is just a small sample and results will vary. There are times where totals still overshoot but mostly it is just in terms of a few hours, or at worst a full day. Regardless, our overall experience has been great as it has proven to be accurate enough that we could confidently commit to certain dates and schedules.

Tips on using this system

• This system works great if you know the tasks beforehand. You really need to sit down and think what the steps are and try not to miss anything and to avoid adjustments.
• The more granular the tasks, the easier it is to assign a time and confidence level to them.
• Being honest with the confidence levels helps produce more accurate estimates. It also brings to light any low-confidence tasks that are in need of reconsideration.
• Make the scope smaller and easier to digest by dividing your project into smaller parts (It can be by milestone or by module) and then make an estimate for each.
• Do this long enough and you'll get better with judging estimates and confidence levels.
• It helps to review and compare how long a task took against your estimates. It will give you insight why you overshot (In the example above, I learned that I am still bad at properly estimating UI related tasks. I should adjust my confidence levels for next time).

Conclusion

This simple system has helped us a lot and we plan to use it on all our future projects. It's not 100% accurate but it is accurate enough that we can schedule confidently with it. I would be the first to admit that it may not work for everyone but if you are always overshooting your estimates then this system might be worth a try.

[Check out our game which was released relatively on time over at Android and iOS]

Note: This is from a series of articles that outlines the things I've learned while making Chef Wars for 2+ years.

TLDR

• There is more to the saying that "premature optimization is the root of all evil".
• Instead of asking WHEN to optimize, it is more important to ask WHAT and HOW to optimize.

It is a well known adage among programmers that premature optimization is the root of all evil. If this is true then I must have been very close to the devil himself.

During the early months of development on Chef Wars I did my best to optimize my code as much as possible. We were making a big game and I wanted to have a stable foundation to build our game on. I obsessed over lots of things from the interconnection of the various systems to folder structures. I was happy with all that I've built, but sadly progress was slow.

I realized at this point that I was optimizing too prematurely. If I wanted to reach my milestones on time then I needed to change my approach. This means leaving the optimizations for later. When is later though? I figured that it makes sense to do it at the end when all the systems are in place.

All went smoothly until we reached Open Beta. The game was reported to be sluggish and almost unplayable which signaled the need to start optimizing. While I was able to optimize some parts, there were some that I could not optimize properly without undergoing a major change to the code. Sadly, rewrites were not an option as we were running out of time.

The profiler has been really helpful in catching performance problems.

Looking back it is easy to pinpoint what went wrong. I was optimizing too early, later changed my approach only to find out that I was already too late to optimize certain critical parts. I, of course, want to prevent this from happening again. So the million dollar question is: How does one determine when to optimize? How does one know when is too early and too late?

The complete version

I later learned that the famous adage actually has a longer version:

Programmers waste enormous amounts of time thinking about, or worrying about, the speed of noncritical parts of their programs, and these attempts at efficiency actually have a strong negative impact when debugging and maintenance are considered. We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil. Yet we should not pass up our opportunities in that critical 3%.

From Donald Knuth's The Art of Computer Programming

Turns out there was more to the saying that completely changes the lesson to be learned. Breaking it down we can infer that the author is telling us that:

• Too much obsession over non-critical parts wastes time.
• Only focus on efficiencies that matter.
• Optimize whenever possible, but not at the expense of the previously mentioned points.

So instead of asking WHEN to optimize, it is more important to ask WHAT and HOW to optimize. In other words, anytime there is a chance to evaluate if an optimization is needed, one needs to consider whether there really is something worthwhile to optimise, and if so, how to proceed in optimizing.

Answering the WHAT and HOW

Knowing how to answer the WHAT and HOW is not easy and requires both experience and careful planning to get right. The internet is a bit divided about this as nobody really knows the best answer. In spite of this, I was able to gather some helpful nuggets of wisdom during my research that are worth considering:

• Be critical of what optimizations to use at each stage of the project. Determine how critical it is and if it can be done later.
• If setting aside optimizations for later, make sure to prepare the code so that it would be easy to do so when the time comes.
• Proper planning during the design stage can determine what to build and how to optimize in advance.
• Measuring/profiling optimizations would reveal which are the most effective to use in the future.

Conclusion

There is a certain sense of pride in producing optimized and stable code. Sadly, this kind of perfection comes at a cost of time. The solution is to always consider at all times when, what, and how to optimize.

This may all seem overkill to worry about but after going through 2 years worth of development on Chef Wars, I know all of this is worth taking the extra effort to do right. I hope that what I've learned may also be of use to you.

[Our game is running better now and you could play it by downloading it on on Android and iOS. Also check out my previous postmortem where I talk about something that went right.]

Note: This is from a series of articles that outlines the things I've learned while making Chef Wars for 2+ years.

TLDR

• All data in our game is contained in one excel file we call the "Universe".
• Prototypes can be done on the Universe excel file itself
• Iteration is easier as we only need to change one file.
• We made a system that downloads changes from our server so players don't need to update their builds.

Before we started development on Chef Wars, Cliff, my co-founder and game designer for the team, already had pages of spreadsheets containing important values in the game. It's kinda like a game design document but in the form of tables, columns, and rows. This "Universe" file contained everything from stats, dialogue, competitions, locations, chefs, and enemies just to name a few.

*This file definitely gives a hint on what type of guy Cliff is.*

Having a list of all the data that will be used in the game has helped us visualize the scope and the systems to be built, especially in making prototypes. One time Cliff made a simulation of the battle system using his Excel mastery. The universe data is fed into this simulation (i.e. competition level, recipe power) and the expected values are displayed (i.e. judging result, rewards amount). This mockup allowed us to see how the battles play out and made the whole thing easier for me to understand and implement in the engine.

All the content of the universe file is then converted to the JSON format which is used directly by the game. Iterating on the game is easy because the file would just need to be converted again for the new changes to show up. The conversion process is done manually though using a CSV to JSON tool. I would have automated the process but didn't have the time to work on it.

*It's like the Matrix*

Initially, when we wanted to update some values, we would need to push a new build version that players need to download. We figured that this is too cumbersome especially if we really have some critical changes we want to get out as soon as possible. As a solution to this, we made a system where a master copy of the JSONs are saved on our servers. We can change the data from here and the game would automatically download the necessary files that we changed. This is a really great feature that has helped us push important changes without having the need for a new build. But it does require a lot of bandwidth especially if a lot of players request for the new data so we do it only when needed like on crash producing bugs.

As you can see, we've spent a lot of time making sure that our game is data-centric as possible and it benefitted us immensely. This approach has been so useful that we plan to use it on our future projects. And hopefully, after reading this, we've convinced you to try it out too.

[Check out how the universe has been transformed into a game by playing Chef Wars on Android and iOS. Also, be sure to check out Cliff's postmortem where he talks about the things we learned during our global launch!]