ASSESSMENT FOR AN ATM SYSTEM CHANGE : A METHODOLOGY FOR THE ANSPs

Air Traffic Management (ATM) is a continuously evolving process, where many current system elements derive from a time when ATM characteristics were very different from today. Nowadays, the provision of ATM services has to design new solutions and adapt to new scenarios. Although ESARR 4 and EU Regulation 1035/2011 define the need of evaluating ATM system risks before implementing any change, they do not define a practical tool to support the decision-makers. The aim of this paper is to fill this gap, proposing a systematic methodology; the Preliminary System Safety Assessment Tool (PSSA-T) capable of helping the decision makers in evaluating safety implications due to system changes. PSSA-T relies on the definition of two Indexes, which have been built according to the Aerospace Performance Factor (APF) methodology, and allow safety assessment of any proposed change. In detail, the former Index compares the evolutionary scenario with the current one and the latter evaluates the evolutionary scenario in which there is a failure of intervention, in the hypothesis the system change has been implemented already. A preliminary study about the change from Flight Progress Strip (FPS) to the Electronic-FPS clarifies the outcome of the study.


INTRODUCTION
Even though the recent forecasts on air transport [1,2] offer quite different predicted values, they all reflect a fast increasing trend.The causes of both cargo and passenger air travel growth go beyond simple macroeconomic factors and regard also structural parameters such as distance, populations, activities and ideas of mutual interest [3] in addition to environmental taxes [4], regulations and emission trading schemes [5,6].The most important contribution to the air transport growth, however, relies on the global economic growth and on the increasing competition and liberalization, which reduced the average fares and expanded the service in terms of route development and frequencies.Note that the technological evolution in the last decades allowed supporting this ever-increasing air transport demand.In this ever-growing scenario, the Air Traffic Management (ATM) system increases its importance and acquires the main role, evolving from the modest beginnings of the 1950s to a sophisticated command and control system, capable of taking into account and ruling an integrated vision of airports, volumes of airspace, facilities, equipment and people [7].It is therefore necessary to support the management of the subsequent complexity, adopting significant changes in the current ATM system, e.g. in services and procedures, human resources, physical infrastructure, systems and technology, regulation and standardization, as emerged in the recent SES-AR Master Plan [8].It is inevitable that the implementation of these changes should consider not only the performance enhancement but also relevant targets in other areas, taking into account the 11 Key Performance Areas (KPAs) [9], i.e. safety, security, environmental impact, cost effectiveness, capacity, flight efficiency, flexibility, predictability, access and equity, participation and collaboration, interoperability.Note that safety has the highest priority in aviation [10] and changes in any KPAs cannot leave safety performance G. Di Gravio, R. Patriarca, F. Costantino, I. Sikora: Safety Assessment for an ATM System Change: A Methodology for the ANSPs out of consideration.In Europe, the process of change started with the Single European Sky (SES) initiative, integrated with the ATM modernization project SESAR that gives priority to the developments of significant performance gains within KPAs [11].

SYSTEM CHANGE IN THE EUROPEAN CONTEXT
The core idea that lies behind ATM changes is that they provide significant performance benefits and/or form a pre-requisite towards the implementation of a target concept.
This paper examines the safety implications of changes that an Air Navigation Service Provider (ANSP) has to analyse, evaluate and develop.In detail, taking into account the perspective of an ANSP, it is possible to classify the essential changes in operational changes (operation and procedure) and technological changes (equipment), considering their direct contribution to the system performance.Human factor and organizational factor generally constitute a parallel effect deriving from operational and technological changes.When a technological or an operational change is accomplished, it implicitly involves changes in the way human beings and institutions as a whole interact with other subsystems.Note also that changes in human resources management are operational changes that can present a significant contribution to the human factor.With respect to the characteristics of the change itself, it is possible to obtain different effects.A change may be simple and easy to implement with noticeable operational or environmental impact [12].Otherwise, another change may be complex and involve significant alterations to the current situation.Thus, it is necessary to evaluate any change by a systematic process, which should give sufficient confidence that the system change is feasible from a human, operational, technological and safety perspective.
ICAO Annex 11 in conjunction with ICAO 9859 [13] highlights the importance for the ANSP to develop and maintain a formal process to identify changes which may affect the level of safety risk associated with its aviation products or services and to identify and manage the safety risks that may arise from those changes.Several national institutions develop mandatory process to follow for the ATM airspace change, e.g. the UK Civil Aviation Authority (CAA) [14], or manuals for setting out the change management process, e.g. the North European Functional Airspace Block (NEFAB) [15,16] and the Airports Authority of India (AAI) [17,18].CAA, NEFAB, AAI reports describe the phases of an ATM change, integrating it in the lifecycle of the system.Even though the representations differ in the taxonomy and in some details of the processes, they present common features and it is possible to extract a common core.In Europe, it may be helpful to highlight the contribution of ESARRs, i.e. the safety regulatory requirements that aim to represent an element of a harmonized framework for the ATM safety regulation.
In this paper, particular attention is paid to ESARR3 and to ESARR4.ESARR3 mandates the implementation and use of Safety Management Systems (SMS) by ATM service providers in order to ensure a prompt address to all the safety issues and risks within the provision of the ATM service.ESARR4 develops further ES-ARR3 requirements on risk assessment and mitigation and on the documentation of the process, its results and conclusions.ESARR4 exploits the use of a quantitative risk-based approach when introducing and/or planning changes to the ATM system from a safety perspective, covering human, procedural and equipment elements, in order to analyse the safety consequences of the change.ESARR4 does not address the technological assessment of introducing and/or planning organizational or management changes to the ATM service provision [19]; it aims to support the implementation of a systematic assessment and control of the safety impact of any ATM system changes.Within the overall objective of ensuring safety, the objective of ESARR4 is to ensure that the systematic and formal identification, assessment and management of risks associated with hazards achieves safety levels that, as a minimum, meet those approved by the designated regulatory authority.ESARR4, however, constitutes an ideal requirement, rather than a practical tool to support the decision-makers [20].In 2011, the EU Regulation 1035/2011 "Common Requirements for the Provision of Air Navigation Services" [21] transposed ESARR4 into the European Community law.Among other features, this Regulation sets specific requirements for risk assessment and mitigation with regard to changes to the functional ATM system: an ATM service provider shall provide a systematic risk assessment and mitigation process to ensure the hazard identification for any changes to the ATM system, addressing the involvement of the specific components and their relative interactions.
This general perspective defines the risk assessment process that develops along with the lifecycle of the system [22].In particular, it is possible to distinguish several phases of the two parallel processes, as shown in Figure 1.The first stage of the system lifecycle is the Definition, which includes the evaluation of the current system performance and the identification of an improvement.A prompt evaluation of this need should confirm the relevance of the evolving request.In this phase, the risk assessment perspective, it is necessary to develop a Functional Hazard Assessment (FHA).FHA analyses the potential consequence on safety resulting from the loss or degradation of system functions.FHA determines qualitatively the severity of each The following lifecycle stage, i.e.Design, defines and constructs the change in all its features, producing the formal documentation for the Regulatory authority.In this phase, the risk assessment develops a Preliminary System Safety Assessment (PSSA).PSSA examines the new system architecture and determines how failures and/or external events could cause or contribute to generate hazards and their effects, identified in the FHA, supporting the selection and validation of mitigations.PSSA consists of a formal operational and technical evaluation, verifying the requirements and assessing the impact on operational performance, cost efficiency and safety.
Following Design, in the system lifecycle there is the Implementation, which covers the development of the individual elements, and the Integration, which covers their integration in the system.The lifecycle continues then to the Transfer to Operations, which covers the installation and integration of the change in the operational environment, in addition to its validation.Then the Operations Maintenance reflects all the preventive and corrective actions to be performed in order to maintain the desired service and safety level and the Decommissioning provides the system withdrawal from the operation.
During all these stages, it is necessary to develop a System Safety Assessment (SSA) to collect arguments, evidences and assurances to ensure that each system element, as implemented, meets its safety requirements and the system as a whole meets its safety objectives throughout its lifetime.A continuous process of comparing current performances and safety objectives should confirm the achievement of the targets.
Note that FHA, PSSA and SSA reflect the core concept of ESARR3 SMS even in the case of a system change process.In this sense, PSSA acquires the crucial role because it authorizes the future implementation and integration of the change in the system structure, in a systematic way as prescribed in ESARR4 [19].

PSSA-T: PRELIMINARY SYSTEM SAFETY ASSESSMENT TOOL
PSSA is a complex task [23] to be executed during the Design phase of the system lifecycle and at any time when there is a change in the system framework, the PSSA should ensure the ability of the new architecture to meet the old and/or new safety requirements and targets.The tool developed in this paper, i.e. the PSSA-T, helps the safety managers to: -demonstrate to third parties, e.g. the regulator, that risks have been reduced to an acceptable level; -maintain a record of decisions and ensure that further changes will not invalidate the assessment or will not lead to unnecessary repetitions; -support the hand-over of safety responsibilities.
The Preliminary System Safety Assessment Tool (PSSA-T) analyses and organizes in a systematic, clear and user-friendly way, the prescriptions of the PSSA for an ATM system change.PSSA-T proposes a two dimensions evaluation, in order not to neglect the consequences of an ATM system change.It firstly proposes a comparison between the current and the future scenarios, about the different features that contribute to ATM safety performance (Safety Impact of Change -SIC).In addition, to ensure that the change would not jeopardize the safety in the future scenario, a second evaluation compares the impact of a potential failure of the proposed change, with a failure in the current scenario (Safety Impact of Failed Change -SIFC).
By this process, PSSA-T produces two different metrics; one for SIC and one for SIFC, that represent the expected variation of safety performance after the implementation of the change and the variation of safety performance in case of a failure of the change.If both evaluations are positive, PSSA-T confirms the opportunity of implementing the change, enabling further steps of the system lifecycle and risk assessment.If any evaluation is negative, it is mandatory to identify the risks that cause the safety performance degradation and determine mitigating action to an acceptable level, based on the Indexes outcome.If mitigations can restore the safety performance, the safety manager can authorize the system change; otherwise, they have to reject it.Both evaluations take into account the main features of the ATM system, developing performance indicators capable of describing its complexity.Filling a still open gap in the factor analysis [24], this research evaluates the technical, human and G. Di Gravio, R. Patriarca, F. Costantino, I. Sikora: Safety Assessment for an ATM System Change: A Methodology for the ANSPs organizational factors, according to EUROCONTROL prescriptions [23].
According to a general approach, it is possible to sketch the system as a multi-level process, according to the detail level of interest.In particular, the SIC (and SIFC) Index has to consider technical, organizational and human contribution.In this scheme, the Equipment (software and hardware components of the whole architecture) that enable communication, navigation, surveillance and information (traffic and weather) systems, represent the technical factors.Procedures, including e.g.flight, operational, maintenance and airspace sectorisation, represent the organizational factors; Human factors reflect the contribution of the human resources and their relations to the ATM system (e.g.coordination, responsibility, human performances), the human-machine interface (HMI) and the ergonomics of the workstation.In order to support PSSA-T, a more detailed description and further contextualization of the three macro-factors, identifying the sub-functions that each factor considers are necessary.Note that the interactions among sub-functions could be very complex to be described and they could be different in the SIC or SIFC evaluation.To this extent, Aerospace Performance Factor (APF) methodology [25] has been adopted for the safety assessment in ATM.The APF, widely explained in its theoretical steps in [26,27], has proved itself useful both for reactive analysis [28] and for forecasting the airspace safety level [29].APF could help defining the system safety performance and providing a balance measure of each sub-function, in order to support a more consistent assessment and thus, better decisions.

The Aerospace Performance Factor
The five general steps of APF can be adapted, for SIC and SIFC purpose, as follows: Determine the factors that influence the performance.
Starting from the three macro-factors, derive the list of sub-functions through a process of system decomposition.This can be accomplished by convening a panel of experts from different divisions within the organization (senior management, flight operations, dispatch, training, maintenance, flight crew, safety team), including also people with fresh eyes and no bias towards one or more particular sources of information, i.e.Subject Matter Experts (SMEs).

Determine the information available on those factors.
Repeat the APF analysis, each time a PSSA-T is required.Note that it is important to evaluate the additional workload on the panel of experts, minimizing it by using general frameworks and sharing common knowledge, if possible.

Organize the influencing factors.
Divide the sub-factors into categories in order to constitute the APF Mind Map, a graphical depiction of the relationships among the elements in analysis.The Mind Map provides the foundation to develop the APF and aids the company as a whole simply by going through the APF development process.Note that it is possible to set a desired number of levels, and that the number of factors in each level can be different, in accordance with the target of the study.Additional levels characterize a more detailed Mind Map.The Mind Map drives the questions to ask the SMEs.

Determine the relative importance or weighting of the factors.
What makes the APF methodology different is the way it treats each element and identifies its contribution to safety performance.Decision-makers need to know the significance of each element in order to assess the impact of any intervention.For that reason, APF suggests the Analytic Hierarchy Process (AHP) to provide a formal weighting process, ensuring at the same time the validation of the information.This process relies on SME judgments and represents the core idea of the APF.

Display information for decision-makers.
Provide a comprehensive and intuitive picture of the results, in terms of safety performance, graphically displaying the weighted Mind Map values and their potential changes, iterating the SIC or SIFC process, if necessary, to evaluate the adoption of modifications to the initially proposed system change.
APF relies on the Analytic Hierarchy Process (AHP), a multi-criteria decision-making tool developed by Saaty in the early 1970s.AHP permits to obtain a hierarchical structure that combines different functions of a system and translates SMEs' evaluations into quantitative weights.

The Analytic Hierarchy Process
AHP fundamental steps [30] can be adapted for evaluating PSSA as follows: Define the problem and determine the kind of knowledge sought.
SMEs have to determine the factors that influence safety performance with respect to the system change itself (for SIC) and with respect to a potential failure of the change on the future system (SIFC).

Structure the decision hierarchy.
It is necessary to build the hierarchy from the top with the goal of the process, through intermediate levels (factors and sub-factors) to the lowest level of the hierarchy.Note that the structure may be different for SIC and SIFC.

Build a set of pairwise comparison matrices.
For each level, compare sub-factors with respect to their impact on the upper factor in the hierarchy.Pairwise comparisons express SME's relative judgment between two factors in a 9-degree scale of importance (1=equal, 3=moderate, 5=strong, 7=very strong, 9=extreme) and a reciprocal value is assigned to the inverse comparison (a ij =1/a ji where a ij (a ji ) denotes the importance of the i-th (j-th) element).

Extract the relative weights of each factor from the pairwise comparison.
The AHP mathematical modelling follows the subsequent stages for each pairwise comparison matrix A: -calculate the eigenvalues λ of the matrix A, along with the consistency index: If I C <0.1, the matrix is consistent and the judgements are not contradictory; -calculate the eigenvectors x: where the one with λ max identifies the priority vector; -normalize the priority vector to 1 to obtain the relative weights of the different factors of the hierarchic structure; -repeat an analogue procedure to define relative weights of each level of factors and sub-factors; -combine the weights of factors and sub-factors to determine their absolute weights.To obtain the absolute weight of the sub-factor, it is necessary to multiply the relative weight of a sub-factor by the relative weights of all the factors (connected to the sub-factor in its upper hierarchy).
To create the SIC (and SIFC) APF Indexes, the owner of PSSA-T, i.e. an expert of the ANSP safety department, needs to assess the current and future scenarios according to the factors that influence safety.Considering the AHP framework, the only judgements to be expressed are the comparisons of the two scenarios (the alternatives) for each sub-factor of the lowest level of hierarchy.While in SIC the alternatives represent the current and future scenarios, in SIFC they represent the effect of a failure on both scenarios.In the hypothesis that the hierarchy has number I of levels, the I-th Index is obtained by multiplying the direct judgments of the expert of the ANSP safety department by the I-th AHP absolute weight of sub-factors, as shown in Equation 4. The generic (I-i)-th Index is obtained by rolling up the lower Index as shown in Equation 5.

L Index L AHP weight L Jugment
where i=1,… I; L I judgment represents the judgment of the ANSP safety department experts and L I AHP weight its absolute weight; k represents the k-th Index and J (I-i+1)(k) the number of sub-factors in the lower level with respect to the k-th Index, i.e.L (I-i) Index (k).If i=I, the PSSA-T generates the L 0 Index that sums up the overall risk of the system change (SIC and SIFC Indexes).According to APF methodology, this risk can be broken down into its components to analyse the specific factors in different level of detail by checking the corresponding L (I-i) Index.It is now necessary to make an in-depth analysis of the L I judgment with respect to SIC and SIFC purpose.
Note that SIC represents essentially a comparison of the safety characteristics of the system change envisioned in the future scenario with the current system in the current scenario.In order to facilitate the SIC assessment, the ANSP expert has to rate the alternatives on a -9/+9 scale.In this scale, the more negative the judgment, the safer the expert considers the performance in the current scenario without changes.Otherwise, the more positive the judgment, the safer the expert considers the system change envisioned in the future scenario.
On the other hand, although the structure of the APF Index does not change, the scale for L I judgment is substantially different for SIC and SIFC.In SIFC, there is no need to compare the scenario before and after the change.It is only strictly necessary to make a safety assessment in which the change just implemented may fail.Therefore, a scale -9/0 represents the system safety reduction in case of a failure of the proposed change where the lower the value, the higher the safety reduction and vice versa.Furthermore, even though the structure of the SIC and SIFC APF Mind Map are generally different for different system changes, it could be useful to define a general framework, valid for each type of change.The advantage, in this case, consists of not compromising the APF weights.For this purpose, it is necessary to build a dynamic APF, deleting those sub-factors that are not affected by the specific system change and make them non-dimensional, based on their relative weights.This process would be quicker, if compared to the entire AHP weighting process.Figure 2 shows an example of a generic SIC APF Mind Map, as a reference point to develop more detailed analyses.More specifically, Figure 2 describes SIC-Level0-Index, SIC-Lev-el1-Index and SIC-Level2-Index.

PRELIMINARY APPLICATION
This Section shows a preliminary application of the PSSA-T, concerning the validation process of a technical system change, which has impact both on the procedure and on the human factor: the evolution from traditional paper flight progress strip (FPS) to electronic FPS (EFPS).The change consists of implementing EFPS in an Aerodrome Control Tower (TWR) with low traffic demand.
The first step consists of defining the SIC and SIFC Mind Map, taking into account all the factors the analysed system change may affect.In this case study, by the contribution of SMEs, it is possible to develop a three-level Mind Map both for SIC and for SIFC, based on their expertise in the field and on a broad literature review.The Mind Maps acknowledge factors which take into account the relevant role of user interface [31], as well as the ergonomic aspects of the FPS emerging from the experience in the field in [32,33], and the typical potential sources of error for Air Traffic Controllers (ATCOs).
Then, the ACC safety manager and five ATCOs fill a specific questionnaire to express judgments on the change.Figure 3 describes the complete SIC Mind Map at Level 2, while Table 1 depicts the respective Index values.For example, with reference to the complete SIC Mind Map in Figure 3, the specific questions related to Surveillance (Equipment) would be: -Service Quality: How do you judge the variation of safety level in terms of service quality?(noise, data integrity, delay on transmission for information) -Data Elaboration: How do you judge the variation of safety in terms of data elaboration?(quantity, efficiency and correlation of information) It is possible to present some common points, emerging from the questionnaires.Many ATCOs, handling the strips, rearrange them and mentally register the evolution of traffic by these movements [34].Some ATCOs could prefer the paper strip just because they prefer to have unlimited possibilities to annotate them according to personal style.Although this feature could simplify personal comprehension, it makes the coordination more complex and exposed to misunderstandings.In addition, different ATCOs' expertise could cause different detail and accuracy levels in filling in the strips.This factor complicates the coordination, too.In conclusion, the general scepticism in case of a power failure or, more general, EFPS system failure, makes ATCOs generally more favourable to paper strips, which would not fail, at least not in the standard sense.Otherwise, in case of large or critical traffic volume, EFPS could help in managing complex situations with particular reference to the ease of coordination.EFPS also permits to have historic data on movements, in case it would be necessary to analyse past data, either for investigations or for statistics.Note also that EFPS system is compatible with other airport systems (e.g.A-SMGCS, FIDS, De-ice, AFTN) and could ease the management of maintenance procedure of line operations through an easier intercommunication between ATCOs and maintenance operators.
In conclusion, SIC-Level0-Index positive value reflects a situation in which the safety level in the ACC is not reduced with the introduction of EFPS.Note that the only SIC-Level2-Index feature offering a negative value (i.e.Ergonomics) reflects the ATCOs' point of view, which generally prefers paper FPS.The positive safety contribution of other factors mitigates this negative contribution.SIC Index allows a positive evaluation of the EFPS implementation consequences in terms of safety.
On the other hand, SIFC-Level0-Index results in a negative value, which determines the rejection of the change.In this case, it is necessary to go back up to the causes of this value, and define, if possible, corrective actions to decrease the risks and evaluate again SIFC.
Table 1 shows that Procedure has no contribution to negative SIFC Level0-Index and it is thus necessary to focus on Equipment and Human Factor, which has negative SIFC-Level1-Indexes.For example, it may be appropriate to maintain the traditional paper FPS system in a dedicated framework, situated in an ergonomic position within the EFPS vicinity and consider additional training for the ATCOs.These interventions have to be translated into modifications of the proposal system change, in the Design phase, which is developed along the PSSA.Once accomplished the proposed modifications to the system change proposal, PSSA-T has to be applied again to check if the mitigating actions lead to a safety enhancement, proceeding then with the other steps of the system lifecycle.

CONCLUSIONS
Systems changes represent frequent needs for the current ATM system, which has to face increasing volumes and subsequent interactions complexity.Because it is not possible to design a system change without taking into account its safety consequences, this paper proposes a PSSA-T, a methodology for organizing in a systematic and customizable way, the Preliminary System Safety Assessment (PSSA).PSSA-T relies upon APF and allows, by SMEs' evaluations, to define two multi-level Indexes, i.e.SIC and SIFC.These Indexes allow a clear and user-friendly safety assessment respective of the change envisioned in the future scenario with the current system in the current scenario, and of a potential failure of the system in the envisioned scenario.
The good outcomes related to the preliminary implementation of PSSA-T for the safety implications of a change from FPS to EFPS, offer positive expectations related to this approach, permitting to identify risks associated with the process and guidelines for possible

Figure 3 -
Figure 3 -SIC APF Mind Map for the System Change: from FPS to EFPS