Much early usability work used summative methods ,
but was not always supported by other user centred design activities.
It therefore gained the reputation for being an expensive way to
identify problems when it was too late to fix them! So the emphasis
moved to formative evaluation (so-called "discount" usability methods)
that could be used earlier in development .
It is essential to introduce usability early in the development
process, but without subsequent summative testing, it is difficult
to judge the effectiveness of the usability work .
Summative usability testing using the CIF has advantages to both consumer and supplier organisations during procurement. It is one of the most effective ways to enrich the consumer's requirements document with objective user performance and satisfaction goals (based on the existing system/product used). It provides a platform on which to evaluate potential competitive products from a number of supplier organisations during the procurement of a new system/product. By using the CIF test structure, suppliers are able to demonstrate that their product complies to the usability metrics defined in the requirements document.
The PRUE trials have demonstrated the potential value of the CIF in a wide range of European environments, and this experience with the CIF contributed to the final version of the text for the CIF standard.
The adoption of the format as an international standard should
provide a strong case for the wider use of summative testing reported
in the Common Industry Format. The PRUE case studies provide strong
evidence for the benefits of this approach. The CIF builds on the
methods developed by the EU MUSiC project [3,7],
and it is appropriate that it should be used in Europe.