"There was contemporary evidence in central government and the private sector that the NHS IC's contractual approach, combining agile with a fixed price, was high risk. Our report Shared Services in the Research Councils reviewed how research councils had created a shared service centre, where a similarly structured IT contract failed," the NAO wrote.
Such failings are not confined to the public sector. Diamond giant De Beers ended up in court in a similar dispute, again with Atos. Judges found fault on both sides but ultimately awarded De Beers £1.4 million compensation.
In the case of the NHS, the NAO highlighted a number of factors that increased risk - none of them news to project management professionals. These included:
Limited staff capacity at the customer - NHS ICTheir reliance on contractors for development and procurement expertiseHigh staff turnover in the project team. Ten project managers were responsible for GPES during the period from September 2008 to September 2013.
Once the query tool contract was signed the NHS and Atos struggled to agree the detailed requirements. "This delayed development, with Atos needing to start development work while some requirements had yet to be agreed. NHS IC and Atos agreed to remove some minor components. Others were built but never used by HSCIC," according to the report.
Warning flags were raised. The Gateway 4 external review in December 2012 said that problems deciding requirements were possibly exacerbated by an offshore development process.
The Gateway reviewers also warned about the difficulty of combining Agile development practices and traditional, waterfall project management techniques, noting: "The GPET-Q [query tool] delivery is being project managed using a traditional 'waterfall' methodology. Given the degree of bespoke development required and the difficulties with translation of requirements during the elaboration parts of R1, the Review Team considers that, with hindsight, it might have been beneficial to have adopted an Agile Project Management (PM) approach instead."
It went from bad to worse with Atos building core components of the system that were not adequately tested by the NHS itself before they were accepted. The NAO blamed the NHS directly for this:
"The NHS IC accepted delivery of the GPES query system in March 2013, after system testing which the NHS IC developed. This testing did not identify design flaws that meant it would be impossible to extract data from all GP practices. These problems were severe and required Atos and HSCIC technical staff to carry out remedial work, taking six months to complete.
"The test was not comprehensive enough to identify these problems. To work in a 'real life' situation, the GPES query system needs to accurately communicate with the four separate extraction systems and other systems relying on its data. The test NHS IC and Atos agreed was less complex. It did not examine extractions from multiple extraction systems at once. Nor the complete process of extracting and then passing GPES data to third-party systems."
Sign up for Computerworld eNewsletters.