At both IR and IAI, a usability maturity assessment
was used to identify any gaps in the organisations ability
to apply user centred design. The assessment was based on the Usability
Maturity Model, which consists of a list of activities that represent
good practice in user centred design. In the assessments, each activity
was rated on the scale:
Inland Revenue/EDS: full assessment
At Inland Revenue a formal assessment lasting
one week was carried out by two assessors from Lloyds Register,
assisted by two usability specialists from Serco Usability Services
who identified opportunities for process improvement. A total of
13 stakeholders associated with the trial project at different levels
in Inland Revenue and EDS were interviewed in twelve 3-hour sessions.
A conventional software process assessment procedure based on SPICE
(ISO 15504) was used. This produced a detailed profile and rich
information about where improvements would be beneficial. At a feedback
meeting on the last day, it provided the basis for an agreed set
of improvement activities.
Inland Revenue comments:
It was however a wary project team that was
brought together for the first maturity assessment, uncertain
what they had let themselves in for. The maturity assessment however
opened everyones eyes to:
- The different ways user could and should
be involved throughout the lifecycle;
- The benefits that could accrue to both the
project and IR/EDS;
- Professional support available from Lloyds
Register and Serco Usability Services.
Output from the assessment was not only a
clear eyed assessment of the level of maturity in this area but
it provided a straightforward model for raising that level aimed
at the heart of the development lifecycle, the facilitated workshops
which are the engine of design and development stages.
12 months later when the improvements had been
made, a second similar assessment was carried out to see whether
the improvement objectives had been achieved. Very significant progress
had been made. When the results were presented to a meeting of senior
stakeholders, the benefits were sufficient for the meeting to authorise
incorporation of most of the methods into the standard Inland Revenue/EDS
documented processes. The meeting also suggested that regular usability
maturity assessments should be arranged to monitor improvement.
By contrast, a simple one-day workshop provided
the basis for process improvement at IAI. The activities in the
Usability Maturity Model were used as a good practice checklist.
The Serco usability specialist rated each activity as not performed,
largely performed or managed, based on a short discussion with one
or two developers or managers who were most knowledgeable in each
area. Although some ratings may not have been completely representative,
they were sufficient to provide the basis for an agreed programme
A second workshop was held 16 months later.
As the Serco usability specialist who had been carrying out the
improvements also organised the second workshop, it was easy to
agree on the extent of the improvement with the IAI representatives
in less than two hours.
The one-day assessment format was appropriate
for LAHAV since it is a) a relatively small organization, and
b) it has a lasting culture, commitment and infrastructure for
The first assessment revealed many areas that
needed improvement including some organizational issues. These
were used to select UCD methods for trial. The second assessment
purpose was to evaluate the improvements made. The detailed results
are very valuable and will be used in further dissemination activities
in LAHAV and other IAI divisions.
Number of design and development staff
Use a fully documented process?
Importance of end user needs
Experience with usability
Attitude to process improvement
Number of stakeholders interviewed
Number of UMM activities judged relevant
Initial number of activities partially
or not performed
Final number of activities partially or
The maturity assessment was highly effective
in setting an agenda for both organisations. Could the simple one-day
workshop-style assessment have been used at IR/EDS? This question
was asked at the final stakeholders meeting at IR/EDS, and the opinion
expressed was that IR/EDS needed the confidence provided by the
thorough and impartial software process assessment procedure.
Under what circumstances can major benefits
be initiated by just a simple workshop? In a large and complex organisation
like IR/EDS, it is more time consuming to identify the current situation
and more complicated to obtain consensus on change. The formality
and structure of a SPICE assessment is consistent with the organisational
culture, and provides the necessary detail and authority for the
By contrast LAHAV at IAI has a smaller more
flexible group where it is easier for all the main stakeholders
to be involved and make rapid decisions.
Both organisations found the results so beneficial
that they have adopted the methods as a normal part of their development
process. At IR some methods will be applied by usability specialists,
while IAI found them sufficiently intuitive for use by members of
the development team.
Does this provide a model for how to introduce
user-centred design in other organisations? The common features
of IAI and IR were that they shared goals to:
- provide systems that meet user needs
- improve their processes
But even these organisations had initial difficulty
in understanding the potential benefits of the methods, which differ
in nature from other software engineering activities. It is still
not clear how best to present the proposed user centred design activities
in a way that can be understood and appreciated by designers and
In these case studies an extra incentive was
that the trials were subsidised by the European Union. It is to
be hoped that dissemination of the successful results will provide
an incentive for other organisations to invest their own resources
in carrying out similar improvement initiatives. There is every
reason to believe that the benefits obtained by IAI and IR could
be replicated in other similar organisations.
Last updated 13-Oct-00.