Skip to main content

What you can and can't learn from personal software projects.

Ok, controversial click bait title aside, personal software projects do an enormous amount of good in teaching you how to learn new languages, write better code, architect code and may even make you a buck or two on the side (if you are lucky). However, there are some things that you are not likely to learn when writing personal software projects and I wanted to discuss a few of them here.

What they can't teach you

How to communicate your ideas

By definition a personal project is personal. You are probably the only one working on it (though you can ask a friend or two to help, which is highly advisable if you can). When you want to add a new software pattern or framework, you just do it. While that's incredibly liberating, one of the benefits of working in a team is having to explain your decisions will make you more reflective and will give you a better sense of if you are doing things the right way or not. If you meet with resistance to the idea, it means you have to think through all the pros and cons. Also, working is probably about 80% communication and 20% coding in any case (or even less coding and more communication depending on your role).


It would be great if your personal project grew to the size of Uber or Google and had to handle millions of hits per second or billions of rows of data. Heck, even most non personal projects (e.g. professional ones) don't have to deal with scale on that level.

These problems of scale will force you to deal with even simple issues in a complex manner. You may need to tweak your SQL code. You might need to denormalise your data. You might need to add a caching layer or use eventual consistency. You might need to add load balancers and use a cloud service to deliver your static assets.

All of these are probably not worth spending the time implementing on your personal project.

I even read somewhere that Facebook allows you to log in with the caps lock version of your password (e.g. if your password is "PassWord123" you can also log in with "PASSWORD123"). The reason being is that Facebook gets so many hits that redirecting users back to the error page if their caps lock is off costs them more money than letting them in.

The other issue with scale is dealing with edge/corner cases. If you have a corner case affecting 1% of your users, it's not worth dealing with it if your project gets 50 hits a day. However if your website gets hit a million times a day that's 10 thousand people who will run into your 1% edge case.


When you are writing a personal project, you are the "business" as well as the programmer. As the programmer, you probably will suit the business needs based on what is easiest to code. In non-personal projects you will get asked to do some pretty whacky things that probably don't match your model of the system. Sometimes you can push back, but it's really a process of negotiation and you will probably end up putting something together that may be good, but not perfect (what is perfect anyways?). 

Not only that, but you know that other bit of funky code you wrote a few months back? The one you just optimised? Yeah, well, we don't need that anymore. You can remove it at your leisure. Ah, you had to change the base models to get it to work and now that affects the new code you have since written? Well, good luck with that!

Product managers aren't trying to mess you around. Most of the time they just don't know what is needed until something has been running for a while.

Other causes of complexity can be added by the other topics below.

Other people's code

Let's face it. Not everybody sees the world in the same way. Also, not everybody is going to approach (and solve) a problem in the same way either. Working on a personal project does not lend itself to seeing new ways of problem solving in the same manner (though you might come across some old code and say to yourself "What was I thinking?").

If you are working with a group of programmers, well, you will run across this every day. If you are not, maybe you should consider working on an open source project (and/or reading lots of open source code). Occasionally, you might even learn something.

Legacy code and data structures

This is similar to the Other People's code issue, but you multiply time and complexity. Perhaps one of the libraries used has been end of lifed. Perhaps what you thought was necessary 6 months ago turns out not to be necessary now. Perhaps you are taking on a new role after the CEO fired the old development team. Or even worse, the people who built the old code are still there and don't realise there is a problem with it.

Moving code around should be relatively simple, but depending on how it's written, it can be tricky. Updating data structures is even trickier (as data is largely seen as immutable and there can be a lot of it to shift). However, nothing is impossible.

Personal projects are less likely to run into this. They tend to be short term and when you want to try something new, you will probably just abandon your project and start again with a new framework or language,

Working to a deadline

You know all those horrible programmers who wrote all that yucky code and those data structures that make no sense? Well, they probably did it because they had a deadline. You do get time constraints on personal projects, but they are more of a case of not having enough time to work on a project.

With a deadline, things have to get done, but corners sometimes have to be cut. Ideally it is scope that is cut and not quality, but in reality, something's gotta give. Even cutting scope can lead to half built features that might not make sense no matter how well the code is written.

As a whole though, it seems like businesses (at least the ones I have been in touch with lately) are more aware of the effects of deadlines on code quality and are using them more sparingly. In any case, this is something you are unlikely to encounter on a personal project.

Industry standards

Just as there are many different ways to implement something, there are also standards on the other hand to limit their use in practice. For example, it is a standard in Rails to use ActiveRecord for your models and to interact with them in a specific way. There are actually lots of other tools you can use (and there may be good reason for using them) but for the most part standards ensure some groups of programmers will implement things in a similar consistent manner. If standards are used properly, you should not be able to tell which member of the team wrote a particular piece of code without using git blame.

Again I recommend reading open source projects (or even joining one) to get a good idea of what the industry standards are like. Code does not happen in a vacuum and if no one else can understand your code, it does not matter how good its performance is.

What they can teach you

Ok, now I am going to do that thing where I list the same items, but from the opposite side...

How to communicate your ideas

You know that saying "A picture is worth a thousand words"? Well, the same holds true for code. A working example is worth many hours of discussion. If your personal project can be shared with your coworkers, it can give you a common vision and also a reference point.


While it is true you cannot tell exactly how things will scale on a personal project, there are tools out there which can hit an endpoint multiple times and at least give you an idea of how to handle load. Doing this against a personal project can help you identify potential pain points that you can avoid doing on production.


Personal projects are not by nature complex, but can be used to help break down a complex problem. You might be able to break a complex issue into multiple personal projects each solving one problem to help you tackle the big problem you are having at work.


Essentially personal projects make for great prototypes. You can solve a problem relatively quickly without the baggage of existing code and data or even try out new concepts and languages.


Popular posts from this blog

Master of my domain

Hi All, I just got myself a new domain ( ). The reason is that is offering cheap domain via and I thought after having this nickname for nigh on 10 years it was time to buy the domain before someone else did (also I read somewhere that using in your domain is the equivalent of an or email address...shudder...). Of course I forgot that I would have to re-register my blog everywhere (which is taking ages) not to mention set up all my stats stuff again. *sigh*. It's a blogger's life... In any case, don't forget to bookmark the new address and to vote me up on Technorati !

Speeding up RSpec

So today I have been looking into getting our enormous battery of tests to run faster. I have yet to find anything that works for Cucumber, but I did find an interesting way to speed up RSpec which is detailed here. Basically, it seems that by not collecting garbage too frequently, you can make your tests run much faster (at the expense of memory management of course). We observed a 30% reduction in the time it takes to run an RSpec test suite. I did try to implement this on Cucumber, however because we need to store much more in memory to set up and tear down our objects, it meant that I kept running out of memory when I wasn't using the default Garbage Collection and the tests took even longer (so, buyer beware). I suppose if you had a small set of features though you might see some benefit.

Responsive Web Design

I wanted to go over Responsive Web Design using CSS. In the old days of web development, we had to code to common screen sizes (i.e. 800 X 600, 1024 X 768) and we patiently waited for people to upgrade their computers to have a decent amount of screen real estate so we could design things the way we really wanted. We also took on semi stretchy web layouts etc to expand and contract appropriately. Then about 2 or 3 years ago, Apple released this little device called an iPhone with a 320 X 480 resolution which took the world by storm and suddenly a lot of people were viewing your website on a tiny screen again... Anyways, as it can be difficult to design a site which looks good on 320 X 480 AND 1680 X 1050, we need to come up with some kind of solution. One way is to sniff the client and then use an appropriate stylesheet, but then you are mixing CSS with either JavaScript or server side programming and also potentially maintaining a list of appropriate clients and stylesheets. Also, you