Skip to main content

What you can and can't learn from personal software projects.

Ok, controversial click bait title aside, personal software projects do an enormous amount of good in teaching you how to learn new languages, write better code, architect code and may even make you a buck or two on the side (if you are lucky). However, there are some things that you are not likely to learn when writing personal software projects and I wanted to discuss a few of them here.

What they can't teach you

How to communicate your ideas

By definition a personal project is personal. You are probably the only one working on it (though you can ask a friend or two to help, which is highly advisable if you can). When you want to add a new software pattern or framework, you just do it. While that's incredibly liberating, one of the benefits of working in a team is having to explain your decisions will make you more reflective and will give you a better sense of if you are doing things the right way or not. If you meet with resistance to the idea, it means you have to think through all the pros and cons. Also, working is probably about 80% communication and 20% coding in any case (or even less coding and more communication depending on your role).


It would be great if your personal project grew to the size of Uber or Google and had to handle millions of hits per second or billions of rows of data. Heck, even most non personal projects (e.g. professional ones) don't have to deal with scale on that level.

These problems of scale will force you to deal with even simple issues in a complex manner. You may need to tweak your SQL code. You might need to denormalise your data. You might need to add a caching layer or use eventual consistency. You might need to add load balancers and use a cloud service to deliver your static assets.

All of these are probably not worth spending the time implementing on your personal project.

I even read somewhere that Facebook allows you to log in with the caps lock version of your password (e.g. if your password is "PassWord123" you can also log in with "PASSWORD123"). The reason being is that Facebook gets so many hits that redirecting users back to the error page if their caps lock is off costs them more money than letting them in.

The other issue with scale is dealing with edge/corner cases. If you have a corner case affecting 1% of your users, it's not worth dealing with it if your project gets 50 hits a day. However if your website gets hit a million times a day that's 10 thousand people who will run into your 1% edge case.


When you are writing a personal project, you are the "business" as well as the programmer. As the programmer, you probably will suit the business needs based on what is easiest to code. In non-personal projects you will get asked to do some pretty whacky things that probably don't match your model of the system. Sometimes you can push back, but it's really a process of negotiation and you will probably end up putting something together that may be good, but not perfect (what is perfect anyways?). 

Not only that, but you know that other bit of funky code you wrote a few months back? The one you just optimised? Yeah, well, we don't need that anymore. You can remove it at your leisure. Ah, you had to change the base models to get it to work and now that affects the new code you have since written? Well, good luck with that!

Product managers aren't trying to mess you around. Most of the time they just don't know what is needed until something has been running for a while.

Other causes of complexity can be added by the other topics below.

Other people's code

Let's face it. Not everybody sees the world in the same way. Also, not everybody is going to approach (and solve) a problem in the same way either. Working on a personal project does not lend itself to seeing new ways of problem solving in the same manner (though you might come across some old code and say to yourself "What was I thinking?").

If you are working with a group of programmers, well, you will run across this every day. If you are not, maybe you should consider working on an open source project (and/or reading lots of open source code). Occasionally, you might even learn something.

Legacy code and data structures

This is similar to the Other People's code issue, but you multiply time and complexity. Perhaps one of the libraries used has been end of lifed. Perhaps what you thought was necessary 6 months ago turns out not to be necessary now. Perhaps you are taking on a new role after the CEO fired the old development team. Or even worse, the people who built the old code are still there and don't realise there is a problem with it.

Moving code around should be relatively simple, but depending on how it's written, it can be tricky. Updating data structures is even trickier (as data is largely seen as immutable and there can be a lot of it to shift). However, nothing is impossible.

Personal projects are less likely to run into this. They tend to be short term and when you want to try something new, you will probably just abandon your project and start again with a new framework or language,

Working to a deadline

You know all those horrible programmers who wrote all that yucky code and those data structures that make no sense? Well, they probably did it because they had a deadline. You do get time constraints on personal projects, but they are more of a case of not having enough time to work on a project.

With a deadline, things have to get done, but corners sometimes have to be cut. Ideally it is scope that is cut and not quality, but in reality, something's gotta give. Even cutting scope can lead to half built features that might not make sense no matter how well the code is written.

As a whole though, it seems like businesses (at least the ones I have been in touch with lately) are more aware of the effects of deadlines on code quality and are using them more sparingly. In any case, this is something you are unlikely to encounter on a personal project.

Industry standards

Just as there are many different ways to implement something, there are also standards on the other hand to limit their use in practice. For example, it is a standard in Rails to use ActiveRecord for your models and to interact with them in a specific way. There are actually lots of other tools you can use (and there may be good reason for using them) but for the most part standards ensure some groups of programmers will implement things in a similar consistent manner. If standards are used properly, you should not be able to tell which member of the team wrote a particular piece of code without using git blame.

Again I recommend reading open source projects (or even joining one) to get a good idea of what the industry standards are like. Code does not happen in a vacuum and if no one else can understand your code, it does not matter how good its performance is.

What they can teach you

Ok, now I am going to do that thing where I list the same items, but from the opposite side...

How to communicate your ideas

You know that saying "A picture is worth a thousand words"? Well, the same holds true for code. A working example is worth many hours of discussion. If your personal project can be shared with your coworkers, it can give you a common vision and also a reference point.


While it is true you cannot tell exactly how things will scale on a personal project, there are tools out there which can hit an endpoint multiple times and at least give you an idea of how to handle load. Doing this against a personal project can help you identify potential pain points that you can avoid doing on production.


Personal projects are not by nature complex, but can be used to help break down a complex problem. You might be able to break a complex issue into multiple personal projects each solving one problem to help you tackle the big problem you are having at work.


Essentially personal projects make for great prototypes. You can solve a problem relatively quickly without the baggage of existing code and data or even try out new concepts and languages.


Popular posts from this blog

Elixir - destructuring, function overloading and pattern matching

Why am I covering 3 Elixir topics at once? Well, perhaps it is to show you how the three are used together. Individually, any of these 3 are interesting, but combined, they provide you with a means of essentially getting rid of conditionals and spaghetti logic. Consider the following function. def greet_beatle(person) do case person.first_name do "John" -> "Hello John." "Paul" -> "Good day Paul." "George" -> "Georgie boy, how you doing?" "Ringo" -> "What a drummer!" _-> "You are not a Beatle, #{person.first_name}" end end Yeah, it basically works, but there is a big old case statement in there. If you wanted to do something more as well depending on the person, you could easily end up with some spaghetti logic. Let's see how we can simplify this a little. def greet_beatle(%{first_name: first_name}) do case first_name d

Speeding up RSpec

So today I have been looking into getting our enormous battery of tests to run faster. I have yet to find anything that works for Cucumber, but I did find an interesting way to speed up RSpec which is detailed here. Basically, it seems that by not collecting garbage too frequently, you can make your tests run much faster (at the expense of memory management of course). We observed a 30% reduction in the time it takes to run an RSpec test suite. I did try to implement this on Cucumber, however because we need to store much more in memory to set up and tear down our objects, it meant that I kept running out of memory when I wasn't using the default Garbage Collection and the tests took even longer (so, buyer beware). I suppose if you had a small set of features though you might see some benefit.

Freezing Gems

What is a gem and why would you want to freeze it? In Ruby, there are times when you want to access pieces of functionality that other people of written (3rd party libraries) and you normally have 2 options. You can install a plug in or install a gem. Normally the method you use is determined by which ever is made available by the author. Gems are installed on the host machine and are pretty handy when you want to run things in the command line or else across lots of projects, but their downside is that if you use a gem in a Rails project there is no automatic publishing mechanism when you deploy your site. You will need to log onto the remote host machine and install the gem manually. Plugins are specific to Rails and are similar to gems in that they are also 3rd party libraries. However they are associated with your Rails project as opposed to your machine so they will get posted to the server on a regular deploy. Freezing a gem is the process of transforming a gem into a plug in. Es