Mars Curiosity Rover required 'most rigorous software testing'

A product from software verification provider Coverity was one of many tools used to remove defects

The Mars Curiosity Rover, a robot sent to Mars by NASA to investigate the possibility of extra-terrestrial life on the planet, had to undergo a rigorous software verification process to ensure that it could safely land on Mars.

The robot, which had its first test drive on Wednesday, was built by NASA Jet Propulsion Laboratory (JPL). JPL opted to use development testing software provider Coverity to ensure that flight control and on-board function software were thoroughly tested.

In an interview with Computing, Andy Chou, chief technical officer and co-founder of Coverity, explained what the project entailed.

"The JPL developed several million lines of code - which if printed would be a 6ft stack of paper - and part of it was for the landing process, but also for other experiments. Before the software was fully testable, the JPL eliminated a large portion of the defects they found," he said.

JPL then enforced a mandate to use static analysis tools, including Coverity, and fix everything that these tools found; both defects and "false positives", which are incorrect reports of the tools.

Full scans were completed every night on all of the code and this resulted in what Chou called a "near infinite detection" of the defects that were introduced onto the code base.

"This allowed JPL to remove defects and ensure quality as the code is being formed and written and as a result it got quality code upfront so that its later testing could be more rigorous and focused on more difficult issues," he said.

Chou explained that it was very difficult to get anything of that size to be defect-free.

"JPL tried everything it could and got it right and it shows that we can get it right even for large software systems," he said.

[Turn to next page]

Mars Curiosity Rover required 'most rigorous software testing'

A product from software verification provider Coverity was one of many tools used to remove defects

But although the Curiosity Mars Rover project is a large software system by the standards of space vehicles, Chou said that Coverity has customers with commercial software systems that have code-bases that are more than 10 times larger than JPL's.

However, he explained that a big difference between commercial software systems and the Curiosity project is that it cannot afford to fail.

"Commercial software systems don't go through the same rigour in terms of verification of software and that is only reasonable as it has a different standard for quality than something that has to work the first time - and absolutely must work," he said.

Chou said that by enforcing very tight coding standards, Coverity aimed to test the source code by examining every single path to the code and examining for potential defects that could cause crashes, or unintended behaviour of the software.

"By doing it this way we get a very thorough examination of cases that were very unlikely to appear but will at some point in production get executed," he said.

On a broader front, Chou said that the JPL performs other software verification tests alongside Coverity.

"It is about looking at the software systems verification as a Swiss cheese, and each slice has potential holes in it which lets in defects. However, if you put enough slices in place next to each other during the development process, then hopefully all of the holes do not line up and you get covers for different defect types as you pass through different gate stages and development processes.

"As a result you catch more and more defects all the way; that's why it is important to use different - but complementary - techniques, so that defects can be removed in multiple stages," he added.