From large systems such as remote towers and ATM system, to individual functionalities such as MTCD, AMAN and safety nets, Commercial Off The Shelf (COTS) systems come with the promise of significant cost savings and faster and better system development. But not all ANSPs are born equal, hence being ‘Off The Shelf’ means the system will need to be adapted to satisfy the local needs of the ANSP, and this is where COTS programme risks can gather.
The role of Human Factors in de-risking COTS implementations
Published 15.03.19
3 mins read
The triple risk model of COTS system implementation reminds us of the basic risks that need to be mitigated during the COTS introduction path. Lack of acceptance lies with the poor perception of the change by the user population; addressing it is very important right at the start of the change, but it may not protect against lack of adoption or partial adoption. These refer respectively to situations where the ATCO does not use the system it because s/he does not receive adequate support from it, or uses only a small part of its functionalities. For these risks, the main consequence is that benefits are not realised, investment is ‘wasted’. Last, but not least, the third category of the risk model, HF-safety, covers the more classic areas of Human Performance (HP) safety risks associated with the use of automation and extensively discussed in aviation literature. Things like potential for confusion, overreliance on automation, and erosion of expertise.
Human Factors (HF) integration brings benefits right from the decision to buy, through system selection, adaptation validation and transition to operation.
However, all too often, HF considerations begin later in the process – after system selection in the best cases, during transitions in others—leaving ANSP and manufacturer open to costly redesigns and the increased possibility of poor human performance, human error, and/or rejection.
The triple risk model of COTS system implementation reminds us of the basic risks that need to be mitigated during the COTS introduction path. Lack of acceptance lies with the poor perception of the change by the user population; addressing it is very important right at the start of the change, but it may not protect against lack of adoption or partial adoption. These refer respectively to situations where the ATCO does not use the system it because s/he does not receive adequate support from it, or uses only a small part of its functionalities. For these risks, the main consequence is that benefits are not realised, investment is ‘wasted’. Last, but not least, the third category of the risk model, HF-safety, covers the more classic areas of Human Performance (HP) safety risks associated with the use of automation and extensively discussed in aviation literature. Things like potential for confusion, overreliance on automation, and erosion of expertise.
Overall the mitigation of these risks requires effective human factors integration already at the COTS procurement phase. Here, an early task is to understand the gapbetween the systems available on the market and the ANSP’s own operations. HF/HP specialists can help define Terms of Reference as well as develop the Operational Requirements and HMI/CWP requirements specifications. They can help benchmark the various market solutions and ensure that HP criteria are included in the COTS system selection process.
In the COTS adaptation phase the focus is on closing the gap between the available system and the ANSP’s own operational context. Here, HF specialists can liaise with the manufacturer to ensure user-centred adaptation, leveraging on HMI and HF product and process standards.
In the final validation phase, HF will want to assess if the system is mature from an operational-HP perspective, and whether appropriate mitigation means have been developed. Typically, this will involve evaluations such as HMI/CWP inspections, Real Time simulations, shadow mode/ghost mode evaluation and HP-safety assessments, mitigation assessment for human-based hazards. At its most basic, we are answering the question, can this system be used effectively and safely by your ATCOs?
Having outlined the logic of HF integration across the COTS introduction process, it’s perhaps worth reminding ourselves why an HF specialist is needed alongside. HF experts understand how to apply HF methodologies cost-effectively depending on the local ANSP context. This means saving time and effort for instance when analysing cognitive and physical tasks and the work environment, when organising requirements identification workshops with end users and manufacturers, or when evaluating the system with end users. An HF specialist will also have knowledge of previous incidents and accidents demonstrating the significance of specific HF issues. Finally, the HF expert knows how to identify user needs, will be familiar with relevant HMI/equipment standards/guidance materials (for example ISO 9241) and this will help you to save time when discussing potential solutions and to move forward towards a successful COTS introduction.
The COTS introduction path is exposed to at least three main risks related to COTS usage: acceptance, adoption and HF-safety risk. Over the years, we have assisted/are assisting different ANSPs, such as DGAC, AVINOR, SKEYES and AEROTHAI, in mitigating these risks and maximising benefit realisation through cost effective integration of HF into all three phases of the COTS introduction path, i.e. procurement, adaptation and validation.
This blog was adapted from a presentation made by Dr Simone Rozzi at World ATM Congress on 12th March 2019.
Author.
Simone RozziSenior Human Factors consultant
