Mark Needham

About 8 or 9 months ago I remember having a conversation with a colleague where I asked him where he had got his almost encyclopedic knowledge of all things software development.

His reply at the time was that he read a lot of blogs and that this was where he had picked up a lot of the information.

While subscribing to different blogs remains a useful way of learning about different aspects of software development, I think Twitter is now becoming a very useful complementary tool to use alongside the RSS reader.

I originally thought of Twitter merely as an extension of Facebook status, but several bloggers, most noticeably Roy Osherove, have been using Twitter almost as a cutting floor for blog articles or upcoming books.

Several blog aggregations have also started posting updates onto Twitter including Los Techies, Elegant Code, Code Better and Planet TW, which I set up at the end of last week. While this doesn’t remove the need for subscribing to the feed it provides a constant stream of blogs to read rather than the batch reading process I tend to use when reading posts from Google Reader.

Following software development authors is another way to keep in touch with what the best in the field are working Jurgen Appelo has removed the need to find them all by creating a top 50 list of software development Tweeters. This use of Twitter is also encouraged in the upcoming book Apprenticeship Patterns.

Since I started using Twitter I have come across quite a lot of interesting content that I may not have come across otherwise:

• Frequent conversations about REST and DDD between serialseb and colinjack. I’ve not used REST as much as these guys have but it’s interesting to follow the conversation and it provides a potential future reference if and when I do work in this area.
• Jeremy Miller explaining the ‘one in one out’ part of his Thunderdome approach to using ASP.NET MVC which I did not quite understand from reading his blog
• A link to a post about natural talent by Guy Kawasaki – I find theories of learning intriguing so it was interesting to read an angle on the subject which talked about how being good at something might actually work against you.
• Learning about Malcolm Gladwell’s new book Outliers from following Steven ‘Doc’ List and Jason Yip’s Twitter feeds. I probably would have come across this eventually but the process was shortened thanks to Twitter.

There has been some discussion on the Alt.NET mailing list with regards to whether Twitter is killing its use. I haven’t followed it for long enough to say whether that’s the case but one of the more valid cases against Twitter is that it is hard to follow message trails after the event – you tend to need to be there at the time the discussion is happening to get the most value from it.

Trying to find the right balance of noise to signal ratio is also something that has to be managed but overall I think Twitter is a useful platform for learning and getting to know what is happening in the rest of the software development universe.

I’ve noticed a recurring trend in the way that I learn new concepts which doesn’t seem to fit exactly into any of the models of learning that I have come across so far.

It seems to me to be a learning cycle which goes something like this:

1. Don’t know what is good and what’s bad
2. Learn what’s good and what’s bad but don’t know how to fix something that’s bad
3. Learn how to make something that’s bad good

The two areas I have noticed this have both been related to concepts about writing clean code.

When I first started writing code I couldn’t tell the difference between testable and untestable code. After my first project I became more aware of patterns that led to untestable code but I wasn’t able to make untestable code testable. Finally after working on a couple of projects I had worked with enough people and shown enough tips that I have a reasonable idea of how to get difficult to test code into a state where we can put some kind of tests around it.

I noticed the same trend more recently when discussing how to write truly object oriented code. Initially I wasn’t aware that suffixing classes with ‘Manager’ or ‘Helper’ was a code smell until I came across Peter Gillard-Moss’ post about this. Having become aware of this I started noticing this in code bases that I worked on but I still couldn’t see how to fix it. I’m not completely in the 3rd part of the learning cycle yet but I am more aware when creating classes to focus on what they are not what they do when it comes to naming, to think about the classes responsibility and so on.

This doesn’t seem to fit precisely into learning models that I have come across such as the Dreyfus Model, Four stages of competence or Shu-Ha-Ri.

I think it is closest to the Four stages of competence, closely relating to each of the first 3 stages. I’m not sure how the unconscious competence stage would fit in though.

At the moment I’m using the pattern to raise my awareness of where I currently am for different skills and to help me move from stage 2 to stage 3.

We decided to run a technical retrospective on our code base yesterday afternoon but apart from one blog post on the subject and a brief mention on Pat Kua’s blog I couldn’t find much information with regards to how to run one.

We therefore decided to take a fairly similar approach to our weekly retrospectives in terms of having one column for ‘Like’ and one for ‘Dislike’. In addition we had columns for ‘Want To Know More About’ and ‘Patterns’. We kept this retrospective purely about the code base because we tend to cover development best practices and process in our normal retrospectives.

Since the code base is only a couple of months old and has been kept in fairly good shape with regular paying off of technical debt, there weren’t that many areas of the code best that people didn’t like.

We did have some interesting discussions around the best way to get data onto the page.

We are currently getting domain objects to render themselves into a ViewData object which is then available for consumption from our Freemarker templates.

The problem expressed with this approach is that when we are writing the code in the Freemarker template we need to know exactly how the domain object has rendered itself to the ViewData object in order to display the data on the page.

The alternative approach is to expose the data by adding getters to the domain objects but this seems wrong to me because it means breaking encapsulation and even if we mean to only use these getters from the view, the fact that the data is now exposed increases the chance of it being misused elsewhere.

The majority of the retrospective was taken up discussing the items in the ‘Want To Know More About Column’. Although we have been pairing rotating quite frequently there are still some areas of the code base where some people are stronger than others so this gave them an opportunity to share their knowledge.

One useful thing about this discussion was that a pair were able to explain why they had done something in the code base rather than just what they had done which is often the case when explaining things in stand ups for example. Hopefully this will help to create a greater understanding of the code base.

I don’t tend to notice patterns in code bases so I put that column into the retrospective more out of intrigue to see what others had noticed. We managed to come up with 5 or 6 although a lot of them were around the use of Pico Container, Servlet Filters and Restlet Filters although we do have some of the Domain Driven Design patterns appearing in the code as well.

Overall this was an interesting exercise to have undertaken and one which I first came across from Sarah Taraporewalla. It will be interesting to see how things change in the code base if another one is run in a month’s time.

I recently came across this post which speaks about the desire of recruiters to put candidates into technology specific boxes when it comes to describing their experience.

I guess this desire is backed by humans’ need to see the patterns and similarities in data and having someone who doesn’t quite fit into a generalised box makes it more difficult.

I have worked on projects in Java, C# and a bit of Ruby so I do agree with most of the points with regards to language specialisation and as Jay Fields points out it is actually beneficial to diversify your experience to improve yourself. I have certainly come across some interesting ways of doing things from my so far limited experience with Ruby.

The one part of the post I disagree with is the following:

Never mind that years of experience will never really make you a better developer. Either you can code or you can’t. If you’re a good developer, you’ll be just as good with one month of Java experience as with 1 year.

While I still have a long way to go on my software development journey, looking at what I can do now compared with what I could do two years ago is not even a contest. It also doesn’t do justice to colleagues of mine who are orders of magnitude better than me, having constantly found ways to improve themselves over time.

I think perhaps the author was thinking more along the lines of ’1 years experience repeated 9 times does not equal 10 years experience’. I think it’s fair to say that some of the best people I have had the chance to work with have not repeated this pattern.

If we are constantly looking for ways to close the gaps in our knowledge and looking for better ways to do things then we are certainly going to get better. On the other hand doing the same thing over and over again for several years probably won’t lead to a significant improvement.

Using different languages is certainly one way to improve but I don’t think we necessarily need to change the main language that we work with in order to gain all the benefits of doing so.

I have used both Ruby and Erlang, albeit briefly, and there were certainly lessons learned from my experiences that I can apply to my work in Java and C#. For example, using Ruby has shown me the value of convention over configuration and keeping code simple where possible, while my experienced with Erlang have shown me the benefit of having no state and have led me trying to keep my objects as immutable as possible.

As long as we reflect on our experiences and improve the way we work from as a result, having experience in several different languages is certainly beneficial.

There are some other ways that we can improve as well.

Gaining experience with projects at different stages is another way to sharpen the tools in our software development toolkit.

I have worked on projects which have been going for a couple of years, rewrites of existing systems and projects which are just starting out. The skills are similar but not exactly the same.

Early on we have the opportunity to shape the system and indeed the code and a lot of the work revolves around getting environments set up, configuring 3rd party APIs to work with our system, setting up the build etc.

Later on in projects the skills are around working out how to add new code without breaking existing functionality. This may involve working out how to test around legacy code, which is a skill unto itself.

Gaining experience in different roles within a team is another way to improve ourselves.

While I am not sure that this extends to everyone on a team trying out each of the other roles, I have certainly heard developers speaking of the benefit of working in a QA role for some period of time and personally I have spent time working primarily on build and deployment and seeing the world from that angle.

A colleague of mine on an earlier project always advocated the benefits of developers having to work in production support so that they could appreciate the value of putting useful logging into the code. I believe that this is something that is done at Amazon – you are expected to support your application in production, it doesn’t just get thrown over to production support.

I’m sure there are other types of experiences that we can gain, but the best people I’ve worked with have demonstrated a superior ability to solve problems, integrate systems, explain ideas to others, write code and so on.

It’s not the number of years, it’s what you do with them that makes the difference.

One pattern I’ve noticed over the last couple of years with regards to my own learning is that I find it very difficult to learn new things unless I can directly apply what I have learnt to a real life situation.

I feel this was part of the reason I found the way material is taught at universities so difficult to understand – nearly every course I studied was taught on its own without any reference to the others, and rarely did I get to use the ideas I learnt in a practical context.

To give an example from the professional world, last year I was working on a project for an investment bank and I became very interested in the domain and started reading a lot of finance books including Den of Thieves, Traders Guns & Money, When Genius Failed, and several others.

I finished on the project and started working on a project in a completely different domain. I tried to continue reading the books but my interest had waned considerably now that I had no context to apply my reading to.

I noticed a similar trend with software books – I read Refactoring to Patterns when I reached the stage of knowing unclean code when I saw it, but being unable to fix it; some of Working Effectively with Legacy Code only when I worked on a project which had a lot of untested code; and Domain Driven Design when I became interested in how to interact with 3rd party systems cleanly on my next project,

A similar idea is described in Apprenticeship Patterns, in the chapter titled ‘The Right Book At The Right Time‘. The authors talk about working out which book is right for you at the time and not just reading the most prestigious book.

The barometer I am using at the moment for knowing whether it is the right book to read right now is how strong my motivation is to read it – for example I have been struggling to read Interface Oriented Design for the past 3 weeks but as soon as I saw we had a copy of Test Driven Development By Example in the office I picked it up and finished it within a day.

What do I gain from this approach?

The main advantage of learning in this way is that it helps concentrate your focus on what will be most useful for your current situation.

The benefits of this approach are also pointed out in My Job Went to India in the chapter titled ‘Be Where You’re At’ which talks about the benefits of focusing on the here and now rather than always thinking of the future:

Focusing on the present allows you to enjoy the small victories of daily work life: the feeling of a job well done, the feeling of being pulled in as an expert on a critical business problem, the feeling of being an integral member of a team that gels.

If we can focus our learning around what we’re actually working on then it certainly becomes even more enjoyable from my experience.

For me personally it also plays directly to my learning strengths – I understand material I read much more easily when I can apply it in context soon after if not immediately.

Drawbacks of this approach

The main problem I can see with this approach to learning is that you become somewhat reliant with regards to your learning on the projects that you are working on.

I started to realise while writing this that what I’m describing might more accurately be called project driven learning and it is certainly the case that projects can provide a great deal of context for learning.

I don’t think that learning needs to be totally guided by projects though – there are areas of software development that I am very interested in and I look for opportunities to learn more about these areas on my projects as well as from conversing with my colleagues.

Admittedly my motivation to learn coding wise comes from projects I do at work – I haven’t yet generated the enthusiasm to work on open source projects although as an avid open source user, I am very grateful to those who do take this approach.

I understand that this approach may not work for other people and certainly those who ran with Ruby & Rails early on in their own time were learning ideas that they didn’t necessarily have a context for and were perhaps creating their own context.

I also appreciate that there are many people who are able to just read books or papers on a vast array of different subjects at the same time and understand the concepts perfectly, but me – I need a context!

Early last year I became very curious about what it was that made the best developers in the industry so good at what they do.

Jay Fields points out some things that he believes indicate that a developer is good at the end of this post but a former colleague and I tried to come up with a list of areas that any Developer needed to be skilled in to justifiably consider themselves good.

This is not an exhaustive list but it was just some ideas we brainstormed and I’ve expanded on them a bit.

Languages and Tools

The first skill any developer needs to have is an ability to work with programming languages and the main tools that are used with them. These could include build tools, IDEs, web frameworks, messaging APIs.

Although Martin Fowler talks about how he prefers design skills over language skills, I think it is important that you do have a good level of knowledge of at least one language.

These skills can best be acquired through practice, learning APIs, playing around with different frameworks etc. There are also infinite books available which go through the features of different languages which I won’t list here.

In a Java development environment you might need skills in some of the following, for example:

• Java language
• Ant
• Jetty
• Spring
• Spring MVC
• JBoss
• WebSphere
• Active MQ

Clearly the more languages and tools you are familiar with the more useful you become on any team you’re on. Knowledge of different types of languages (e.g. static/dynamic/functional) can help bring different points of view in understanding how to solve problems.

Programming Paradigm

This concerns the design skills which Martin talks about. Martin covers it in much more detail, but one area this would cover is having a good understanding of object orientation which is vital for writing maintainable code with imperative languages.

An understanding of ways to approach different problems you might encounter in enterprise development by knowing different patterns and when to apply them is also useful.

The following books would make good reading in this area:

• Agile Software Development Principles, Patterns and Practices – Robert Martin
• Clean Code – Robert Martin
• Code Complete – Steve McConnell
• Design Patterns – Gang of Four
• Head First Design Patterns – Kathy Sierra & Co
• Enterprise Integration Patterns – Gregor Hohpe, Bobby Woolf
• Object Design: Roles, Responsibilities and Collaborations – Rebecca Wirfs Brock, Alan McKean
• Patterns of Enterprise Application Architecture – Martin Fowler
• Refactoring – Martin Fowler
• Refactoring to Patterns – Joshua Kerievsky
• The Pragmatic Programmer – Andrew Hunt, Dave Thomas
• Working with Legacy Code – Michael Feathers

Domain Specific Knowledge

Specific knowledge about the domain (e.g. Investment Banking) is vital for writing a system which is closely linked to the problems it is trying to solve.

Although a lot of this knowledge is acquired by the Business Analyst on the project if Developers can gain it too then conversations with users will be much easier to deal with as amongst other things the terminology they use will make sense.

Beyond this it is useful to understand the goals the business people have, the processes they use to achieve those goals. An understanding of the history of their business so you can see where their industry is going and why is useful for context.

Some domain knowledge can be gained by reading books – this is especially true once you start reading the books that practitioners are reading – but a lot of the knowledge can only be learned by speaking to or watching people who do that job for a living. For example I found out that Options, Futures and Other Derivatives is a very popular book in the investment banking world from my time working in that environment.

Eric Evans’ Domain Driven Design details a way of writing code specific to the domain that you’re working in. Specific domain related books are probably best located with an Amazon search.

With regards to learning domains, the approaches one can take vary from focusing very heavily on one domain and learning it inside out to getting an understanding of a broad set of different domains.

People Skills

One of the most important skills in software development is the ability to work effectively with other people – fellow developers, Quality/Business Analysts, clients, users, the list is endless. If you can do this effectively then you go a long way to ensuring your success.

All the skills of Emotional Intelligence as Daniel Goleman has termed it.

Some useful reading around this area:

• How to win friends and influence people – Dale Carnegie
• Emotional Intelligence – Daniel Goleman

Problem Solving

The ability to solve problems that don’t have an obvious solution is key in software. Coding wise it could be debugging a class-path issue when deploying your application to JBoss or finding a tricky bug that a test reveals.

I found Debugging: The Nine Indispensable Rules for Finding Even the Most Elusive Software and Hardware Problems to be very useful reading around this area as it helps to keep you honest when solving coding problems and not over thinking the problem.

The best problem solvers logically go through the problem logically using small steps, investigating potential causes until they solve the problem.

Personal Management

The art of managing your time effectively, prioritising your work correctly and keeping focus on what’s important. GTD is a famous book in this area although I think it comes down to experience and understanding the way to best exploit the way that you work.

Communication

This links to people skills but includes written communication, how you transmit ideas and your ability to persuade people.

These skills come in very useful when taking part in exercises such as pair programming for example.

I have read a lot of information around Neuro Linguistic Programming (NLP) in this area – Introducing NLP is a useful book to start with on this subject. NLP amongst other things is a gathering of techniques which effective communicators have been seen to use. NLP Weekly is a useful online resource.

Gaps

Acknowledging when the facts don’t match your theory and keeping room for possiblity. In other words approaching things with an open mind, keen to learn other approaches to solving problems.

Go Meta

The ability to learn how to learn – the skill of skill acquisition – as well as knowing which skills to learn next and which to abandon.

Andy Hunt’s Pragmatic Thinking and Learning strikes me as a book which covers most of these things. It has only recently come out but I will review it when I get my copy.

–

There are bound to be things I haven’t considered for this list. I would like to hear other ideas. My underlying desire is to hear the skills that people are developing to make themselves a better software developer.

Construx have the idea of a Professional Development Ladder but I think you need to be a member to see a proper version of it.

While reading some of the rough drafts of Apprenticeship Patterns online I started thinking about the stages I have gone through on my Software Development journey so far.

I have worked in the industry for just over 3 years; 1 year at Reed Business and 2 years at ThoughtWorks. Over that time my thoughts, opinions and ways of doing things have changed, and no doubt these will continue to evolve as I learn more and more.

My time at RBI

I started working at RBI in August 2005 a few months after I finished University. My experience up to this point involved several years coding PHP in a very procedural way and a little bit of Java.

I was hired by RBI as a C# Web Developer and my work there involved working on several internal projects and looking after one of their websites.

At this stage I was still very much convinced that the art of software development lay in learning languages, so I used to spend all my time reading about C# and playing around with all the different APIs.

At this stage I was using Visual Studio without Resharper so I didn’t have the ease of Refactoring or moving code around that I now take for granted.

One of my colleagues took me under his wing and started teaching me how to write better code – separation of code across presentation/business/data layers was my first lesson. Suddenly it became so much easier to make changes! All the code I wrote was still in a non TDD way and after one episode where I created a bug in production I started to think that surely there was a better way to develop software.

Eventually my colleague suggested to me that if I really wanted to learn how to write software then the best place to do so was at ThoughtWorks.

ThoughtWorks Days

I started working at ThoughtWorks in August 2006, hired through the TWU graduate program.

I thought I had a fairly good idea of how to write Object Oriented code but that theory was quickly disproved as I went through Object Boot Camp as part of my TWU training. The Single Responsibility principle was the overwhelming lesson learned as part of this. I also remember believing at this stage that it was all about Design Patterns.

I came back to the UK and did a couple of small projects where I first came across continuous integration and TDD before going onto my first big project.

I remember my first day on that project involved pairing with Darren Hobbs and being amazed at the speed with which he was able to move around the code using IntelliJ. It became clear to me that I had a long way to go.

Working on this project for the best part of the year I learned a lot, including how to write code in a Test Driven way, that everything you do in software is a trade off, but most importantly I learned how to master the IDE – if you can do this then you feel more confident and you can complete tasks much more quickly. This is always the advice I see given to new Graduates at ThoughtWorks – learn how to use your tools!

I moved onto my second project where I was immediately surprised at how much easier I found it to move around the code base than I had at the start of my first project.

We were designing a client side application so a big part of my learning here was around testing presentation logic. Jeremy Miller’s blog proved invaluable at this stage.

It was also the first time I came across the concept of Domain Driven Design – it was amazing how much easier it was to develop software when the developers were using the same language as the BA, QA and in fact the business. InfoQ’s cut down version of Eric Evans’ famous book proved useful in helping me understand the concepts that I was seeing in our code base. I remember thinking at the time that I didn’t need to bother reading DDD as it was all covered in this version – I was wrong!

We had an very lightweight version of Agile being used on this project – we tried to have minimal process and do as many things as possible just when we needed them. It was almost Lean in nature although this was never explicit. It was interesting to me how easy and fun software development could be when it was done like this.

My third project was the first time that I got the opportunity to work with legacy code – i.e. code that hadn’t been unit tested. My early lessons on trade offs came back to me here as I realised that not writing unit tests is a trade off – you can choose to go more quickly initially by not writing them but eventually it comes back to haunt you.

I was working with Alexandre Martins on this project, and his enthusiasm for writing clean Object Orientated code gave me a new outlook on writing code. Working with him got me in the frame of mind of hating exposing the internals of classes and constantly looking for other ways to solve the problem when I was considering doing so.

Halvard Skogsrud’s knowledge around concurrency was another eye opener for me around how non functional requirements should have an impact on the way that software is designed. It also introduced me to the way that other languages such as Erlang handle concurrency – and behind this the idea of having as much of your code immutable as possible to avoid threading issues.

During a debate at a ThoughtWorks Geek Night another colleague brought up Alistair Cockburn’s Hexagonal Architecture, which was the first time that I had come across an Architectural Design Pattern. This is a useful technique when thinking about the design of systems at a higher level.

On my next project I did a lot of work around build and deployment which gave me the insight that developing software is about more than just the code. This was a lesson first taught to me by Chris Read a year before but it finally made sense to me.

A big part of this project was inter process communication between different components of the system which introduced me to the idea of event driven messaging. I immediately saw the benefits of this over the RPC style messaging I had seen previously.

I also had the opportunity to do some work with Ruby on Rails and in particular around the use of Active Resource. This introduced me to the idea of RESTful web services which feels like a much more natural way to communicate over the web than any of the other approaches I have come across.

In Summary

The interesting thing for me is that I didn’t plan to gain any of these learnings, they came about as a natural progression from my interest in software development and from working on different projects with different people.

The biggest things I have learned since I started working in software development are that it is much more an art than a science and that there is no right or wrong, just trade offs that we should be aware of.

I still have a lot to learn but I thought it would be good to have a look at what I’ve learnt so far in the hope it can help others just starting out on their journey.

It would be interesting to hear about others’ journeys and the similarities and differences you have experienced.

I’ve always been the type of person who only gets the motivation to do something if there is some useful practical reason for doing so. It therefore probably doesn’t come as much of a surprise that I hated the majority of my mostly theoretical computer science degree.

I was talking to one of my colleagues last week and came out of the conversation convinced that the desire to know the theory behind concepts is amplified when you actually get to see it in action in a real life system.

My prime example is multi threading and concurrency. At university we had to write a program to calculate e to 500 decimal places using multi threading. In my mind this exercise was completely pointless, and I don’t think I came out of it any wiser with regards to threading.

Contrast that to the project I’m working on now where the main window of our application was freezing up due to an expensive operation being run on the UI thread. The solution of course was to execute the expensive operation on a background thread.

Although I know this is a fairly trivial example, it did provide a real life situation where threading was required. As a result of this I have become much more interested in the specifics of threading and the potential problems that can arise when doing so.

It would probably actually prove more useful to see the concepts in action in a real life system before being taught the detailed theory behind how it all fits together. For me at least when I’m learning about something I like to have a reference point and having seen the concept in action would provide this.

It seems obvious to me therefore that studying for a degree after having first done a few years in industry would probably lead to you having a much richer learning experience and a much improved understanding. Not that I expect that to catch on!