The top-ten tenets of software quality assurance, part two: documentation

Not many people love documentation, but it's an essential element of any project. However, too much is just as bad as too little, warns Mark Wilson

Documentation. I love it. It's the sane view of the labyrinthine quagmire of code; the layman's window on the developer's Kafka-esque world. I know people mistakenly believe that Agile methods ban the production of documents, and that developers feel liberated and out-of-school if they can ditch documents but, believe me, without them it's the road to ruin.

To work, though, you have to do them properly: concise, pertinent, traceable. The minimum size and the minimum number possible. Doing too many documents that are too long and full of padding and blather (people still have the childish belief in quantity over quality) is just as bad as chucking them out altogether.

Nothing signals failure to me more than pie-bald, scraggy, partial, incomplete, missing, untraceable, duplicate and ambiguous documentation. This problem was made considerably worse by the advent of Agile. There has been a documentary diaspora; an explosion in the use of ticketing/social media-inspired communication tools.

People mistakenly believe that Agile methods ban the production of documents

The downside of using ticketing/defect logging as depositories for, or replacements of, documents, is that despite their nesting/parent-child spawning logical advantages, there is never a complete logical picture, in one place, of the state of the system (or sub-system) in question. By that, I mean a uniquely-identified, version-controlled, reviewed and authorised specification of what, when why, how and by whom.

Of course, if we are talking about a bug or small change, then the defect or testing tool, or service desk ticket, (and spawned replies/history) will be more than sufficient for that change. But major changes to designs or requirements result in the need to update the requirements, designs and tests.

This rarely gets done, even with a proper document. But it's a hell of a lot easier if there is such a document in the first place. Because writing a document is a whole-thought process. It requires you to consider the item in question in its totality, and not in bits. A document compared to appended notes is like a roast dinner compared to several packets of crisps.

It's possible, even desirable (if the circumstances are right; see ‘Method' in a subsequent article) to express requirements and designs as Stories, held in some tool. But the data is still then dispersed throughout the tools and not contained in logical chunks. In all cases where I've seen Agile development work well, there has been, at the heart of it, a design specification document.

All of my experience dictates that writing down in one place, in correct English, the specification of requirements, designs, tests and processes - and then assiduously reviewing that document (see next week's instalment) - is the single best way of improving the quality of the thing you are making.

Documentation do's and don'ts

Do use a simple, strong structure.

When I was about 14, we were shown by our ‘O'-level Biology teacher (Mr Fletcher) how to write-up experiments. The headings he told us to use were as follows:

Aim - Apparatus - Method - Results - Conclusion

The fact I can still recall them after 47 years tells you a lot about them.

We wrote up our experiments using them. But if you think about it, those headings are also a plan: you know in advance what you need to do. So a document with half-a-dozen - no more - major section headings has given you a logical structure and a route-map.

Make documents like code: capable of only one course of interpretation and action

Do use logically-decomposed sub-structures:

1. Nest your subsections and sub-subsections (but don't go lower in granularity than that);
2. Nesting paragraphs is like nesting code. Dependencies and sequence are clear, you are far less likely to miss something; orphaned thoughts and expressions are easier to spot. Each sub-section is like a signpost when you're reading it, or reading it back. Each sub-section is like a pebble or piece of bread in Hansel and Gretel; they show the path you're on. One of the worst examples of poor document structure (actually leading to the deaths of seven people) is the Space Shuttle disaster of 2005 (see Tools, below);
3. Soon you will find the text almost writing itself. That's why I'm such a stickler for document templates and standards. I once did an audit where there was a template, but the writer had deleted all of the sections he reckoned didn't apply. He consequently avoided producing good designs (as he'd missed out stuff cleverer people than him deemed mandatory). A team at a building society I worked at simply left all of the boiler-plate text in place, and never bothered writing anything else. In fact, I really don't like boiler-plate in the template: it should appear only in the standard;
4. Signal hierarchy and sequence clearly by using enumerated, not bulleted, lists. Use short sentences. Don't use the passive voice. If you want to know how a technical document should be written, look at an ISO or British Standard, or ITIL or SFIA definitions.

Do be unambiguous

Make documents like code: capable of only one course of interpretation and action. Don't have duplicate or redundant sections. Don't use words like "like" or "should". The reader of your document should be in absolutely no doubt.

Don't waffle or pad.

Be terse; be succinct. It's far better to present a sparse document that can be used as-is than artificially inflating it with waffle, repetition, redundant phrases, faux-formalities, attempts at legalese; platitudes no-one would disagree with. If your ignorance shows by this, maybe that's what the project needs. There is great precedent for tight document-writing not from engineering, but from science, legislation and even Word War One (where forms were first introduced widely i.e. cross out the options that don't apply.

They even had form Letters Home ("Dear Mother/Father/Sister/Brother, I am Well/Poorly/Wounded/ etc"). In World War Two they were sometimes perhaps too terse: the only reason the Chance Vought Corsair naval fighter-bomber sported its famous gull-wings was because the spec called for an unparalleled rate of climb, speed and power, but using a single engine. The supplier responded with an engine whose propeller was so enormous, the only way the landing gear would reach the ground was to dip the wings in the middle.

The top-ten tenets of software quality assurance, part two: documentation

Not many people love documentation, but it's an essential element of any project. However, too much is just as bad as too little, warns Mark Wilson

Do use tools

I like Microsoft Word for writing documents. It has strong capabilities for headings and sub-headings, tables of contents and layout.

But I hate Microsoft PowerPoint.

PowerPoint is a visual aid for talking to large groups of people, and that's it. It's not a tool for writing technical documents. It's deadly. Literally. We've all heard of "Death by PowerPoint", well, the seven people who died in the Challenger Space Shuttle disaster literally did so (according to NASA's own analysis) because of the ambiguities that PowerPoint-as-document entails. NASA abandoned PowerPoint (yuck) as a way of conveying technical information in 2005; Jeff Bezos abandoned it in 2012 - and you should too.

NASA abandoned PowerPoint as a way of conveying technical information in 2005; Jeff Bezos abandoned it in 2012 - and you should too

Do include pictures

But don't fall for that old chestnut "A picture paints thousand words". It may well do, but it's no good if the words are wrong or the picture is some thoughtless piece of Word Art. The words must do the heavy lifting; the pictures are an aid. Unless, of course, you're describing a complex structure or infrastructure where you need to show multiple components at the same time. In very complex cases these would, in any case, be rendered in a modelling tool like Erwin.

The most annoying combination of a poor tool and pictures is, of course, a business presentation in a PowerPoint "deck" (PPD). I dread seeing those revolving arrows and exploding acronyms.

Cautionary tales about documentation

Too much

There are two extremes in really bad documentation: far too much, or not enough. In my career, the first prize for Too Much goes to a major systems integrator bidding for, and winning, some massive UK government contracts.

The CIO made a big announcement at a "Town Hall" meeting of the bid team. He was really pumped-up. He was very proud, he said, that the team had sent six BOXES of bid documentation. Yes, he actually measured the quality by weighing the quantity. It's a good job this guy wasn't running the D-Day landings. The Allies wouldn't have even got in the landing craft by now. (I was once told by an ancient and renowned project planner that the Operation Overlord Planning Manual was contained in a single ring-binder, three inches thick). They didn't win the contract, I'm sad to say, but at least a thousand trees went to a good home.

Doing too many documents that are too long and full of padding and blather is just as bad as chucking them out altogether

The same organisation also developed a comprehensive system development method (based, I have to say on something I devised) that meant, taken to its logical extent, the production of 230 discrete documents. Every one of them had to conform to a product standard, and every one of them had to be reviewed, approved and stored in the online document library. When I first proposed it, there were only 30 or so documents. When staff moved on they took the same method to a major supermarket chain, where it caused the same damaging overload. One of my friends there was hospitalised due to the stress.

Not enough

At a financial services client site I worked at, not one document was completed according to the published standard, and not one document was correctly identified. It took me a week just to find out what jobs were in the scope of my work (Digital UAT Process definition) because no work item was correctly identified either. Nor could I find what the test status of any product was. These people had so much low-hanging fruit I was making jam with every step of my Loakes brogues.

At a credit-scoring company I worked at, I didn't see one intelligent, sensible document in the six months I was there. I saw plenty of other things (staff burgling their own premises at weekends; the product of "nasal extractions" on the toilet doors) but no documents. They probably did that HPI check you just received…

Doing too many documents that are too long and full of padding and blather is just as bad as chucking them out altogether

Not authorised

At my first job in 1990, we were building a compiler validation service and required NAMAS [National Accreditation of Measurement and Sampling] accreditation to operate as a government-accredited test laboratory. Far too many documents were produced, but many were useful and in use to run the service. But because of a wrong-headed belief that a document must be perfect before it can "really" be used, the documents weren't signed-off or authorised for use. It was the only major flaw the NAMAS audit found. So whenever a useful version can be used, issue it at version one. So what if it goes through 50 versions before it's finally complete?

Invisible business

I was once UAT manager at the notorious ambulance-chasing firm The Accident Group. Their complicated business model was being implemented into a JD Edwards CRM system. Unfortunately, no-one had ever written any documentation for the existing processes. So JD Edwards simply unloaded their expensive system out-of-the-box. I left the company just before their business model detonated; they infamously sacked their 500 employees by text message.

School's Out

I was quality manager at an offshore company when they decided on a double-whammy: Not only would they embrace Agile development; at the same time they abandoned a rigid command-and-control management structure. The developers became like giddy teenagers, snorting with glee as they ran barefoot through their new Agile paradise, happily hacking without the safety-net/shackles of formal documentation and able to act without bothering any bosses. Within two years they gladly brought them back.

Next week….

More knockabout QA tomfoolery when I shall be looking at Formal Inspection!

Missed Part One, the Contract Review? Read it now.