Documentation, another case of rapidly diminishing returns

No body writes documentation in startups.

Writing documents is a luxury only established companies can afford. One might ask: are companies successful because they write documents or do they write documents because they are successful? But I’d be hard pushed to find anyone to argue the latter.

Back in 1996 I worked on the ill-fated UK rail privatisation. I was part of the team writing a new computer system to create a new rail timetable – one that would allow competition in train services. In the average week I spent half my time coding and half my time documenting.

For every program there was a program specification – what the program had to do. And a functional specification – saying what the program did. I had to update both. And these had to align with the system architecture, which was a multi-volume beast I wasn’t allowed to touch.

Additionally I had to create a unit test plan for each program or set of changes. Unit tests were manual and the test plan was made up of two Word documents. One was a big table and the other had one test per page. As I conducted the test I had to update the plan with success or fail – and fail of course mean rework so we alwys made sure it worked before hand.

When my program was ready to check-in I had to fill in a source code control form so my team leader knew which files to check in. If I added a new code file to the system I needed to complete an additional form to explain what the file was. On one occasion I got fed up of a 2,000 line C++ file and refactored it as 4 smaller files. Being C++ his meant 4 pairs of .h and .cpp files, so 8 new files each with a form. My team leader was quite clear: I was never to do this again, it made too much work for her.

Did it help? – I find it hard to imagine it did. In fact I started my own hidden documentation, the “Rough Guide to …” which told me (and other devs) things we actually needed to know.

There are two types of companies: those that don’t write documents – and where many many people are asking for documentation. Documentation is seen as a solution to (almost) any problem. And yet people don’t write documents, then they feel guilty about not writing them.

The second type of company is overrun with documentation. There seem to be armies of people writing documentation: architects and business analysts seem particularly keen to write documents. These are often prose, they are about as readable as your average Shakespeare play is to a 15 year old and read about as often. Project managers are also prone to documentation but they don’t write prose; instead, project plans and progress presentations are documents in another medium.

In the first type of company nobody reads documents because there are none. In the second type of company its debatable how many of those documents are read. Many of them are so utterly boring that it is hard to stay awake reading them. As I’m often heard to say:

The bigger a document is, the less likely it is to be read.

And if it is read, the bigger a document is the less the reader will remember.

In one eye-and-out-the-other.

It’s probably just as well because documentation rapidly becomes out of date unless copious amounts of time are invested in keeping it up to date. Actually, it is good that documentation is seldom read because reading it is almost as expensive as writing it and, in theory at least, it should be read far more often.

True, “documentation” covers a very wide range of things. From project plans to release notes, from user guides to architecture diagrams. Some has more readers more than others – user guides compared to functional specifications. But then, who has read their iPhone manual? Does such a document even exist? Arguably the best products don’t need documentation.

In both types of companies I hear complaints about the lack of communication – actually, I don’t think I’ve ever visited a client were people didn’t complain about the lack of communication. But only in the first type of company do people think that documentation will cure the problem.

In such companies documentation is seen as the solution to almost every problem. Programmers complain they haven’t been given written requirements and specifications, they complain the user designers aren’t giving them documents of is expected, and most of all they complain the developers who came before them did not document what they did. Equally testers demand the same requirements and specification but also want programmers to provide written descriptions of what the program does. Project managers want written reports of what was done and so on.

Nobody ever got fired for asking for more documentation, but I’m not so sure about writing it.

While I have sympathy that these people want information I don’t believe documentation will solve the problems – perhaps because I’ve never seen a development effort with “Goldilocks documentation” – not too much and not too little. One thing I do know is that if everyone wrote the documentation they thought was needed, and what others wanted from them, then little would get done.

What I fear is a descent into documentation as everyone sets about communicating through documentation and not talking.

Because documentation takes time and money to create, it takes time and money to read, it takes time and money to update and keep current. And all the time and money spent on documentation is time and money not being spent on developing products and testing products in the market.

Worst still documentation becomes a hinderance to change. On multiple occasions I have met companies that do not want to change their products or processes because the cost of updating the documentation is too great.

I’ve nothing against documentation itself, I simply lament the time and money that could be better spent elsewhere, I regret the missed opportunities for real communication and belief that something has been communicated; and I fear the limitations that documentation brings once in place.

To answer my own question above. I don’t believe companies that write documents are successful, documentation does not determine success – if it did many many projects would have succeeded instead of failing. That documentation is so often lacking but products are successful actually goes to prove that is not essential.

Nor do I really believe companies write documents because they are successful. They write documents because they are successful enough to be able to afford to write documents but those same documents inhibit future success.

Before I close, I can almost hear people rushing to their keyboards to tell me of occasions were a system document saved their life,

“If Sam hadn’t left behind a document that told me the function was connected to the reactor core…”

While I’m sure such cases exists I’m not convinced the they justify the vast amounts of cost of writing a document against doing something else. If Sam hadn’t written that document what might Sam have done with the time instead? Possibly something even more valuable.

As with planning documentation exhibits rapidly diminishing returns on investment.

Photo by Glenn Carstens-Peters on Unsplash


Subscribe to my blog newsletter and download Continuous Digital for free – normal price $9.99 / £9.95 / €9.95

The post Documentation, another case of rapidly diminishing returns appeared first on Allan Kelly Associates.

The Minimum Viable Wiki

Documentation is usually considered a necessary evil. The truth is probably closer to it being necessary and evil. For most of us it's one of those things we feel we should have more of (or any at all) but never have the time. And how much documentation do we need anyway?

Most project documentation is worse than useless. Why? For the same reason that most comments in code are useless-to-dangerous - only more so. Documentation, when written, is rarely kept up-to-date. It is rarely complete. It was often never "true" to start with.

In short the docs are just not reliable

Oh, there are exceptions, of course. API docs generated from source code, for example. They will be consistent, right? Perhaps. Assuming they are kept maintained within the source. It is still possible for them to lie.

If we are Agile we don't do documentation, do we? After all the manifesto states:

Working software over comprehensive documentation

Of course it doesn't say we don't value documentation - just that we value working software more. And even then it talks about comprehensive documentation.

There is the idea of "Just Enough Documentation". I hear people talk about this and see it referred to, along with some personal interpretation. But I have yet to see a definitive write-up of this idea (please let me know if you know of one). Perhaps this is meant to be deliberately ironic? In any case, my understanding is that this relates to the sort of documents that are often seen as deliverables of a project - for example a requirements doc.

I'm not going to talk so much about those, although the principle I'll describe may often be applied. I want to concentrate on the internal, developer-oriented, documentation that describes things such as how a project is organised, what bits do what, what to expect in certain situations etc. This is often captured in a wiki that is specific to a team.

This sort of documentation is often seen as unnecessary in a Agile team. This is where face-to-face communication between team members is sufficient, isn't it?

That view seems to be borne out on projects that do have a wiki. The wiki is rarely complete or up-to-date. It quickly falls out of use altogether, or is approached with trepidation as it is as likely to mislead as inform. If it is kept up-to-date it is likely due to a Mandate From Above, which leads to more time invested in its maintenance than is worth it.

So is there a way this can be made to work? Can it ever be worth it? Why would we need it at all?

Well that first depends on your project and team. Maybe the conversations alone really are sufficient. My experience is that this reaches a limit pretty quickly. We need something. How many times have you, or someone on your team, wasted a day looking for something or going down a wrong path, only for someone else to then say, "oh, that's over here", or, "that's done differently for x reason". Yes the conversation was had - but time was already wasted

So the problem is knowing when and who to ask - especially for things you would have thought were "obvious"

In terms of the Four stages of competence this is about moving from Unconscious Incompetence (you don't know what you don't know) to Conscious Incompetence (you at least know what things you don't know). This is much more valuable than it sounds at first! If you don't know that adding a setting to a config file dumps the information you need to a log you won't spend six hours writing a whole chunk of ad-hoc logging code to do the same thing, for example. Instead you'll remember, "there was something about being able to enable logging in a config file". Now you know to ask the question and finding the answer should be quick and accurate.

Having a full specification, perhaps along with examples, of every attribute that could go in the config file might sound better. And if that config file was meant for end-user customisation you'd probably need that. But if it's some internal thing then any attempt at such documentation is likely to be incomplete and/ or out of date - if written at all

But a line that says, "logging is configurable", somewhere that people will read leaves out those noisy, unstable, details and simply informs you of what is possible. You now know what you don't know (i.e. how to enable logging), whereas before you didn't even know it was possible. Details can be asked for.

So my recommendation for maintaining team information on a wiki is to stick to the briefest possible comment that moves the reader from Unconscious to Conscious Incompetence. Unless you really need it avoid the level of detail that will leave it unmaintained and incomplete.