Oh no, why is my software broken again? - A scary IT story

#dsp #blog #dajsiepoznac #software-dev

Forenotes from 2023

This post was originally published on my blog on March 25th, 2017. This was the first entry in my Daj SiΔ™ PoznaΔ‡ 2017 blog post series πŸ–‹οΈ that was not strictly about F# and programming. Instead, I wanted to explore a more general topic - failures in software development projects. I use the word "project" in a purposely vague manner here. It could represent any effort or initiative to develop and release software to customers / stakeholders / users. Reading this piece again after 6 years brought back some fun and less-fun memories πŸ˜…. Spoiler: although most of the industry focuses on "developer productivity", the success (or failure!) on your "project" is rarely about technical problems.

"Oh no, why is my software broken again?" - A scary IT story

Today I am going to leave the technical stuff on the side and take a wider look at the Software industry. By reading the news you might get the impression that everything is nice and well in IT Land. Between success stories of startups worth millions or even billions of dollars and computer guys making a six-figures just by typing on a keyboard, it might seem like Software is the New Eldorado. Coding bootcamps and other programming schools are on the rise, promising that you will become a Web Developer in under 3 months! and live the millionaire life in just double that time.

Well, today I want to focus on the other side of the coin, and talk about a scary, yet very real aspect of our industry: project failures. If you type "software project failure rate" in your favourite search engine, you'll find a lot of interesting statistics that seem to repeat over the last 10 to 15 years. Here are some of the crispy bits that I found:

That doesn't look good at all. if the sight of those statistics makes your legs weak and palms sweaty, perfect. That was the desired effect. If not, you might want to read those once again. The message is pretty clear: Software development is hard. Anyone telling you otherwise is lying. It requires a lot to get it right. It takes very little to get it wrong.

Tell us why, Johnny! Why is this so hard to make software?!. Rest assured, for the past decades Mankind has spent a tremendous amount of time, energy, and money trying to answer this question. So what are the usual suspects? First, Let me give you my personal take on the matter (caution, touches of satire ahead).

Business vs IT

Those two main actors of almost any significant Software project seem to find an endless source of joy in making each other's life miserable. The business will often report production issues at 5pm on Fridays and make constant and meaningless requests for things they won't be using after all. Meanwhile, IT will plan a full upgrade of all servers right in the middle of the month-end closing and devise vague error messages because... Well because they can.

How IT views Business...

users_1.gif

... and how Business views IT!

Pasted image 20231111125010.png

Methodologies

- My Scrum is better than your Kanban! - Nah, my Waterfall is stronger and it will kick your Agile right in the n...!

Software development methodologies have been seen by many as Silver Bullets, the Holy Grail of IT that would solve all our problems. In practice though, they do not always prevent projects from failing.

Management

This is a tough one. Where to start?

And the list could go on... But more seriously though, let's have a look at some other figures I extracted from this source:

Most Common Causes of Project Failure:

Changing priorities within organization – 40%
Inaccurate requirements – 38%
Change in project objectives – 35%
Undefined risks/opportunities – 30%
Poor communication – 30%
Undefined project goals – 30%
Inadequate sponsor support – 29%
Inadequate cost estimates – 29%
Inaccurate task time estimate – 27%
Resource dependency – 25%
Poor change management – 25%
Inadequate resource forecasting – 23%
Inexperienced project manager – 20%
Limited resources – 20%
Procrastination within team – 13%
Task dependency – 11%
Other – 9%

There are a few interesting observations we can make based on that list.

Project failures seem very rarely caused by technical problems! Isn't it surprising, considering the very nature of Software development? Those damn developers watching cat videos during work hours might not be totally responsible for that disaster, after all.

Project management issues are omnipresent. Failure to plan, adapt, or communicate, is all over the place. According to this article from Joseph Gulla, a former IT specialist at IBM, "54 percent of IT project failures can be attributed to project management, whereas only 3 percent are attributed to technical challenges".

The lack of discipline in capturing requirements - is curiously something that people very often tend to forget about. Look again at those items:

This last point is the most striking to me for two reasons. First, requirements are the foundations upon which a project is built up. You can probably guess what would be the expected consequences of building your house on a swamp. The results are not much different in the Software world.

Secondly, it is astonishing that such a fundamental part of the project is so frequently overlooked, and sometimes even ignored altogether. People keep repeating the same mistakes again and again.

Now, it would be very presumptuous of me to criticise without at least suggesting a few ways to improve the current state of things. This is why I will be following up shortly with another post dedicated to capturing requirements and helping your programmers get the Software right: Help yourself and your programmers by writing better specs!

Did you also encounter IT project failures during your career? Don't hesitate to share them with us in the comments below!

Till next time!