Skip to main content

What you can and can't learn from personal software projects.

Ok, controversial click bait title aside, personal software projects do an enormous amount of good in teaching you how to learn new languages, write better code, architect code and may even make you a buck or two on the side (if you are lucky). However, there are some things that you are not likely to learn when writing personal software projects and I wanted to discuss a few of them here.

What they can't teach you

How to communicate your ideas

By definition a personal project is personal. You are probably the only one working on it (though you can ask a friend or two to help, which is highly advisable if you can). When you want to add a new software pattern or framework, you just do it. While that's incredibly liberating, one of the benefits of working in a team is having to explain your decisions will make you more reflective and will give you a better sense of if you are doing things the right way or not. If you meet with resistance to the idea, it means you have to think through all the pros and cons. Also, working is probably about 80% communication and 20% coding in any case (or even less coding and more communication depending on your role).

Scale

It would be great if your personal project grew to the size of Uber or Google and had to handle millions of hits per second or billions of rows of data. Heck, even most non personal projects (e.g. professional ones) don't have to deal with scale on that level.

These problems of scale will force you to deal with even simple issues in a complex manner. You may need to tweak your SQL code. You might need to denormalise your data. You might need to add a caching layer or use eventual consistency. You might need to add load balancers and use a cloud service to deliver your static assets.

All of these are probably not worth spending the time implementing on your personal project.

I even read somewhere that Facebook allows you to log in with the caps lock version of your password (e.g. if your password is "PassWord123" you can also log in with "PASSWORD123"). The reason being is that Facebook gets so many hits that redirecting users back to the error page if their caps lock is off costs them more money than letting them in.

The other issue with scale is dealing with edge/corner cases. If you have a corner case affecting 1% of your users, it's not worth dealing with it if your project gets 50 hits a day. However if your website gets hit a million times a day that's 10 thousand people who will run into your 1% edge case.

Complexity

When you are writing a personal project, you are the "business" as well as the programmer. As the programmer, you probably will suit the business needs based on what is easiest to code. In non-personal projects you will get asked to do some pretty whacky things that probably don't match your model of the system. Sometimes you can push back, but it's really a process of negotiation and you will probably end up putting something together that may be good, but not perfect (what is perfect anyways?). 

Not only that, but you know that other bit of funky code you wrote a few months back? The one you just optimised? Yeah, well, we don't need that anymore. You can remove it at your leisure. Ah, you had to change the base models to get it to work and now that affects the new code you have since written? Well, good luck with that!

Product managers aren't trying to mess you around. Most of the time they just don't know what is needed until something has been running for a while.

Other causes of complexity can be added by the other topics below.

Other people's code

Let's face it. Not everybody sees the world in the same way. Also, not everybody is going to approach (and solve) a problem in the same way either. Working on a personal project does not lend itself to seeing new ways of problem solving in the same manner (though you might come across some old code and say to yourself "What was I thinking?").

If you are working with a group of programmers, well, you will run across this every day. If you are not, maybe you should consider working on an open source project (and/or reading lots of open source code). Occasionally, you might even learn something.
;)

Legacy code and data structures

This is similar to the Other People's code issue, but you multiply time and complexity. Perhaps one of the libraries used has been end of lifed. Perhaps what you thought was necessary 6 months ago turns out not to be necessary now. Perhaps you are taking on a new role after the CEO fired the old development team. Or even worse, the people who built the old code are still there and don't realise there is a problem with it.

Moving code around should be relatively simple, but depending on how it's written, it can be tricky. Updating data structures is even trickier (as data is largely seen as immutable and there can be a lot of it to shift). However, nothing is impossible.

Personal projects are less likely to run into this. They tend to be short term and when you want to try something new, you will probably just abandon your project and start again with a new framework or language,

Working to a deadline

You know all those horrible programmers who wrote all that yucky code and those data structures that make no sense? Well, they probably did it because they had a deadline. You do get time constraints on personal projects, but they are more of a case of not having enough time to work on a project.

With a deadline, things have to get done, but corners sometimes have to be cut. Ideally it is scope that is cut and not quality, but in reality, something's gotta give. Even cutting scope can lead to half built features that might not make sense no matter how well the code is written.

As a whole though, it seems like businesses (at least the ones I have been in touch with lately) are more aware of the effects of deadlines on code quality and are using them more sparingly. In any case, this is something you are unlikely to encounter on a personal project.

Industry standards

Just as there are many different ways to implement something, there are also standards on the other hand to limit their use in practice. For example, it is a standard in Rails to use ActiveRecord for your models and to interact with them in a specific way. There are actually lots of other tools you can use (and there may be good reason for using them) but for the most part standards ensure some groups of programmers will implement things in a similar consistent manner. If standards are used properly, you should not be able to tell which member of the team wrote a particular piece of code without using git blame.

Again I recommend reading open source projects (or even joining one) to get a good idea of what the industry standards are like. Code does not happen in a vacuum and if no one else can understand your code, it does not matter how good its performance is.

What they can teach you

Ok, now I am going to do that thing where I list the same items, but from the opposite side...

How to communicate your ideas

You know that saying "A picture is worth a thousand words"? Well, the same holds true for code. A working example is worth many hours of discussion. If your personal project can be shared with your coworkers, it can give you a common vision and also a reference point.

Scale

While it is true you cannot tell exactly how things will scale on a personal project, there are tools out there which can hit an endpoint multiple times and at least give you an idea of how to handle load. Doing this against a personal project can help you identify potential pain points that you can avoid doing on production.

Complexity

Personal projects are not by nature complex, but can be used to help break down a complex problem. You might be able to break a complex issue into multiple personal projects each solving one problem to help you tackle the big problem you are having at work.

Prototyping

Essentially personal projects make for great prototypes. You can solve a problem relatively quickly without the baggage of existing code and data or even try out new concepts and languages.

Comments

Popular posts from this blog

Freezing Gems

What is a gem and why would you want to freeze it?

In Ruby, there are times when you want to access pieces of functionality that other people of written (3rd party libraries) and you normally have 2 options. You can install a plug in or install a gem. Normally the method you use is determined by which ever is made available by the author.

Gems are installed on the host machine and are pretty handy when you want to run things in the command line or else across lots of projects, but their downside is that if you use a gem in a Rails project there is no automatic publishing mechanism when you deploy your site. You will need to log onto the remote host machine and install the gem manually.

Plugins are specific to Rails and are similar to gems in that they are also 3rd party libraries. However they are associated with your Rails project as opposed to your machine so they will get posted to the server on a regular deploy.

Freezing a gem is the process of transforming a gem into a plug in. Essen…

Unit/Functional Testing RubyAMF

One of my current projects is using RubyAMF to communicate with Flash (http://rubyforge.org/projects/rubyamf/). On the whole this is really nice because it allows you to transfer Ruby objects directly to ActionScript ones (as opposed to translating the object into XML, sending the XML and then recreating the object in ActionScript).
However, Rails does not provide a built in transport mechanism for AMF, so we cannot run functional testing directly on the data call (as we could for an XML or HTML transport layer). This is a show stopper for a lot of people (Rails w/o Unit testing = a big mess of trouble when something goes wrong).
We can though serve both the HTML and the AMF formats depending on the request format. This means that we can test the object instantiation logic and make sure there are no errors in the controllers (though we cannot check the actual format of the data being served). In the controller, instead of rendering AMF alone, do the following respond_to do |format|

Comparing Rails' Active Record Pattern with Phoenix/Elixir/Ecto

Rails has a very well established Active Record pattern for dealing with the database. You have an Active Record model which maps to the database table, the schema of the model comes directly from the database schema and you place your model specific methods on the Active Record model. This file is also where you set your model relationships (e.g. has_many, has_one, belongs_to). Your instance of the model has all the methods built in.

In Ecto/Phoenix it's a little different. First of all, the database schema doesn't automatically map to the "model". In fact we don't really have models (as Elixir is a functional paradigm). What happens in one file in Rails, happens in essentially two (or more). You have a schema file (where you have to list out all the attributes and relationships). Using the schema file, your "instance" is essentially a data structure (with no methods on it). If you want to transform the data on your struct, you would use a context modu…