Identifying Operational Benefits of the Arrival Management System – A KPI-Based Experimental Method by Evaluating Radar Trajectories

The arrival management (AMAN) system is a decision support tool for air traffic controllers to establish and maintain the landing sequence for arrival aircraft. The original intention of designing the AMAN system is to improve the efficiency of air traffic management (ATM), but few studies are investigating the operational benefits of this system based on key performance indicators (KPIs) and evaluating actual data in a real-time environment. The main purpose of this paper is to propose a KPI based transferable comparative analysis method for identifying the operational benefits of the AMAN through radar trajectories. Firstly, six KPIs are established from a joint study of the mainstream ATM performance frameworks worldwide. Secondly, appropriate evaluation technique approaches are determined according to the characteristics of each KPI. Finally, a Chinese metropolitan airport is taken for the case study, and three periods are defined to form data samples with high similarity for comparative experiments. The results validate the feasibility of the proposed method and find comprehensive performance improvements in arrival operations under the effects of the AMAN system.


INTRODUCTION
Increases in aviation transportation demand and air traffic flow expedite the emergence and application of novel concepts and new technologies in the civil aviation industry. To manage arrivals to a multi-runway aerodrome or locations with multiple dependent runways at closely proximate aerodromes, and to efficiently utilise the inherent runway capacity [1], arrival management (AMAN) system, a.k.a. aircraft arrival management system (AAMS), is created and applied as a decision support tool (DST) for air traffic controllers (ATCOs). The system sequences landing aircraft arriving in terminal airspace (TMA) by optimising the given objectives while subject to a variety of operational constraints.
the CSX characterisation and the preparation of the data set are summarised in Section 4. Section 5 presents the evaluation results and discussion, followed by conclusions in Section 6.

PREVIOUS RESEARCH ON AMAN
The theoretical core of the AMAN is aircraft landing problem (ALP) [5], which has been studied extensively as one of the typical problems of resource allocation in operations research. The main concerns of ALP mainly include calculation environments, solution algorithms, and optimisation objectives [6]. Regarding calculation environments, some studies set the problem scenarios to static [5] and others were carried out in a dynamic context [7]. For solution algorithms, in terms of small-scale ALP, the problem is formulated as a mixed-integer linear programing (MILP), and CPLEX was used to solve it [8]; in terms of large-scale ALP, dynamic programming (DP) [9], branch and bound (BB) [10], heuristic algorithms [11], and meta-heuristic algorithms including genetic algorithm (GA) [12], simulated annealing (SA) algorithms [13], and particle swarm optimisation (PSO) [14] were implemented. As to optimisation objectives that appeared in previous studies, they were not the same and were presented as the following: (1) minimising the total penalty [8,9,13,14]; (2) minimising the total delay [12,15]; and (3) minimising the last flight's landing time, or maximising the throughput of runway [16]. Nowadays, state-of-the-art research has been switching from single-objective to multi-objective optimisation [17], which combines several single objectives together according to the demands of the related stakeholders. Thanks to solid and continuous theoretical research, a number of AMAN systems have been developed, such as the Centre-TRACON Automation System (CTAS) of the NASA Ames Research Centre, the MAESTRO co-developed by Egis Avia and the French Air Navigation Service Provider, the queue management software known as OSYRIS of Barco, 4D-Planner developed by the cooperation of DLR and DFS, Selex ES's AMAN, and a multi-airport multi-runway AMAN system called OPTAMOS of AviBit.
Most of the AMAN benefits evaluation works were taken through modelling, simulation, and/or subject matter expert surveys [2]. Prevot et al. [18] investigated an efficient AMAN utilising air traffic control and aircraft automation and addressed potential benefits from simulations. Andersson et al.
Although each emerging technology has gone through multiple pre-implementation validations, i.e., theoretical tests, evaluation works based on actual practice are still non-negligible since the gaps between theory and practice are always ubiquitous. The same is true in previous evaluations on operational benefits of the AMAN system. They are almost implemented through modelling, simulation, and subject matter expert surveys [2]. Besides, even if practice proves that the only way to achieve visible progress in performance management is to establish a transparent and objective system consisting of key performance areas (KPAs) and key performance indicators (KPIs) [3], KPIs are still considered difficult to define, assess, and quantify for AMAN [4] and have not been applied on a large scale to evaluate the benefits of the AMAN. Whether an AMAN meets the expectations set for is usually regarded as a matter of subjective opinions and reflected by the feedback from ATCOs and other users.
In view of this, the paper aims to contribute to quantitatively discovering AMAN's operational benefits by evaluating radar trajectories and proposing a KPI-based transferable comparative analysis method for identifying AMAN's performance in this kind of experiments conducted under a real-time operating environment. Through jointly studying the mainstream air traffic performance frameworks worldwide and optimisation objectives within AMAN, six indicators are established to monitor system performance changes brought about by AMAN. Furthermore, the experiment focuses on three data collection periods: Period I (P. I) when the AMAN system was not assembled, Period II (P. II) when the AMAN had just been assembled and there was a running-in phase between ATCOs and the system, and Period III (P. III) when the AMAN had operated steadily for quite a time. This comparative experiment setting induces one of the main challenges that is to ensure the representativeness when selecting the periods and the similarity of environmental and other systemic conditions among the three periods. In this study, every attempt has been made to control the rationality of period selection. Changsha Airport (CSX) is taken as the case to verify the feasibility of the proposed method.
The remainder of this paper is organised as follows. Section 2 provides descriptions of previous research regarding the AMAN system. Section 3 highlights the method for establishing KPIs and study approaches for evaluating them. The description of 3. METHODS

Establishment of KPIs
This paper proposes to evaluate the benefits of AMAN from air traffic management (ATM) performance improvement point of view. The results of ATM work are related to the duties of the ATCOs, which are different in the work complexity according to different airspace geometries and traffic demands. The greater the work complexity is, the more probable it is to lead to increased workload and poor performance. In addition, the workload of ATM is also influenced by the abilities and habits of the ATCOs. Therefore, a hypothesis could be made that if a DST is introduced, it could assist the ATCOs to make decisions more fluently under the same work complexity, so as to alleviate the workload and improve the performance. Then DST becomes an indirect factor faced by the ATCOs and eventually acts on their work performance. Consequently, it is reasonable to confirm the hypothesis and discover AMAN's benefits from the perspectives of monitoring the ATM performance. Furthermore, evaluating in this way brings about an important convenience: the documented indicators for reference. Global leaders in the civil aviation such as the International Civil Aviation Organisation (ICAO), the Civil Air Navigation Services Organisation (CANSO), EU-ROCONTROL, and the Federal Aviation Administration (FAA) of the USA have established KPI systems for ATM since 2005. However, it is evident that sufficient harmonisation in the use of indicators has not been achieved so far [22]. Therefore, what should be done is select relevant indicators from the mainstream KPI systems and condense the information about them.
Based on documents published by the ICAO [23], CANSO [24], EUROCONTROL [3,25], and FAA [26], an initial KPI set can be formed. To cut down the number of indicators and delete those of high couplings, a criteria selection method is established for determining KPIs from five aspects: design expectations of AMAN in aspect 1, optimisation objectives of ALP in aspect 2, rationality of indicators in aspect 3, couplings among indicators in aspect 4, and concerns from the front-line work in aspect 5. By this selection process, the initial KPI set will be reduced and focus on the performance of the ATM regarding the arrival operations. [19] used an integer program model to estimate potential benefits of the Collaborative Arrival Planner, a part of the CTAS. Meyn and Erzberger [20] developed the Stochastic Terminal Area Scheduling Simulation to identify benefits of improved scheduling accuracy for an arrival traffic rush period in Dallas/ Fort-Worth airport. Miyazawa et al. [21] used dynamic programming to optimise both trajectory and arrival time assignment, and introduced potential benefits of the optimised flights. These evaluations are theoretical demonstrations of the AMAN's benefits, which were implemented under ideal environments and ignored interference factors from reality (e.g. human factors from ATCOs, the effects of severe weather, unbalanced traffic flow, etc.), leading to the results lacking objectivity. Consequently, evaluation works based on actual operation data are essential. However, since AMAN is under the continuing process of popularisation, there are few studies on evaluating benefits from AMAN based on real-life data. The European Organisation for the Safety of Air Navigation (EUROCONTROL) [4] declared AMAN's post-implementation benefits. Improvements in flight time/distance, as well as capacity utilisation of runway and TMA were found. But the results came from the feedback received from ATCOs and other users, there was no detailed description of how the benefits were evaluated quantitatively, and evaluation perspectives were quite limited. Guzhva et al. [2] assessed the AAMS's benefits in a single-airline environment, by comparing the performance in two periods when the airport operated without and with AAMS. These two periods contain different numbers of days, 88 days without AAMS and 13 days with AAMS. The authors eventually concluded that no observable improvements in the overall system performance were found. Since only the flights from a single airline were included and AMAN refers to all arrival aircraft, this conclusion may be quite one-sided and less convincible.
The aforesaid research studies leave gaps in evaluating AMAN's operational benefits based on actual operation data and through a large-scale experiment, which may make air navigation service providers (ANSPs) and ATCOs confused about whether AMAN should be popularised. In contrast, this study takes effort to propose a transferable data-driven comparative analysis method and identify what benefits AMAN can make in real-time environments.
Secondly, having been aware of AMAN's intended effects in the Aspect 1, we can be inspired more by probing into what objectives are concerned in the ALP combinatorial optimisation inside the AMAN. ALP reflects the interests of several stakeholders, resulting in different objectives having been taken into the optimisation. Zhang et al. [6] developed a criteria selection method to determine appropriate objective functions of the ALP and finally got the suitable subset of criteria, which are as follows: minimising total flight delay (∑T j ), minimising total dwell time (∑F j ), and minimising maximum dwell time (max F j ). It can be noticed that the determined objectives are concerned with flight time (a.k.a. dwell time in TMA) and flight delay, which are highly correlated with these initial selected KPIs: "additional time in TMA", "total minutes actual gate arrival time exceeding planned arrival time", "arrival flight time/distance efficiency", Firstly, AMAN is listed as an important method to improve air traffic flow in the aviation system block upgrades (ASBU) [1], and ICAO regards KPAs including capacity, efficiency, environment, predictability, and flexibility as intended performance improvements of its effects: (1) time-based metering will optimise the usage of terminal airspace and runway capacity; (2) efficiency will be increased through harmonised arriving traffic flow; (3) reduced holding and low level vectoring has a positive environmental effect; (4) flexibility will be improved by enabling dynamic scheduling; and (5) uncertainties will be decreased in terminal demand prediction. These statements offer insights into AMAN's design expectations. Except for the "flexibility" KPA, the other four KPAs all contain KPIs related to arrival operations. Accordingly, the initial selected fourteen KPIs are presented in Table 1, and the source organisations where these KPIs come from are marked in abbreviations. reduced by changing the measurement focus of the former from "time" to trajectories. The coherence of arrival trajectories is proposed here. High coherence of arrival trajectories means the past trajectories stick together, indicating that ATCOs' control methods are standardised, and the uncertainty, as well as the variability of flight time, decreases. Therefore, "flight time variability/travel time variation" is transferred into "the coherence of trajectories". Finally, concerns from the front-line work are considered. (1) The Civil Aviation Administration of China (CAAC) has formally proposed the delicacy management on final (the final here means the flight phase when aircraft makes the final adjustment of speed and configuration before landing) to improve the operational efficiency since 2017. Delicacy management is a kind of management mode aiming to change the original way of extensive management and reduce resource cost. CAAC has put forward to apply this mode to practice, especially in airspace management, in recent ten years. Thus, the "final approach interval" KPI is taken into consideration.
(2) As discussed in the aforesaid content about the correlation between the AMAN effects and the ATM performance, AMAN is expected to reduce the workload of the ATCOs by decision-making supports. It can be assumed accordingly that AMAN can help the ATCOs to build strong situational awareness to reasonably control the arrival aircraft, in order to smoothen the air traffic flow and balance the workload. Under this condition, the "instantaneous workload" KPI is proposed to measure the number of arrival aircraft in TMA per unit time, which is the main source causing the ATCOs' workload.
In summary, by jointly considering AMAN's design expectations, ALP's optimisation objectives, evaluation rationality, coupling extents among KPIs, and concerns from front-line work, six KPIs are adopted for evaluation, which are "airport peak arrival throughput", "capacity utilisation", "instantaneous workload", "dwell time in TMA", "final approach interval", and "coherence of trajectories".

Evaluation of KPIs
Detailed descriptions of the evaluation focus of each KPI are as follows.
Airport peak arrival throughput. It measures the arrival flight support capability by calculating the number of landing flights during peak/rush hours. Good performance in airport peak arrival throughput "arrival punctuality", and "flight time variability/ travel time variation". The above KPIs will be assigned a priority in the following discussions.
Thirdly, not all the initial KPIs are suitable to be evaluated due to disturbing factors in operations and the feasibility of doing this. (1) In the "capacity" KPA, KPIs care mostly about the capacity and throughput. The capacity emphasises the theoretical design while the throughput stresses the actual operation situation. Therefore, the throughput is obviously more appropriate for this study. However, the "Airport arrival capacity utilisation/Capacity utilisation" KPI is literally related to capacity but is still worth being observed as it assesses how effectively capacity is managed, compared to the available capacity. So only the "airport peak arrival capacity/capacity variation" KPI is deleted. (2) KPAs including "capacity", "efficiency", and "predictability" all display attention to whether delays occur compared with plans, but these KPIs may not fit to evaluate the ATM performance due to the propagable characteristics of flight delay. Thus, the initial selected KPIs "additional time in TMA", "total minutes actual gate arrival time exceeding planned arrival time", and "arrival punctuality" will not be picked even if they are prioritised by the optimisation objectives in the Aspect 2.
Fourthly, some indicators have a lot of overlap in the field of performance evaluation, i.e., the extent of coupling between them is very high and they can be merged into more concise ones. (1) The remaining KPIs in the "capacity" KPA are nearly all about throughput, differing in the time and space range of measurement. Since there is only one civil airport in the CSX's TMA, the TMA throughput can be considered the same as the airport throughput, the KPIs are consequently merged as for space range. What is more, AMAN is expected to be more effective during peak hours, thus, peak hours should be observed more carefully. Eventually the rest four KPIs can be merged into one "airport peak arrival throughput".
(2) As dwell distance has a strong linear correlation with dwell time, the "arrival flight time/distance efficiency" KPI in the "efficiency" KPA can be merged into "dwell time in TMA". (3) For the KPIs measuring fuel consumptions and pollution emissions in KPA "environment", they are time-related variables and can be viewed as the derivatives of dwell time; so these indicators will not be taken into account. (4) The "flight time variability/travel time variation" KPI in the "predictability" KPA is close to "dwell time in TMA", but the extent of this coupling can be Descriptive statistics provides approaches for collecting, sorting, describing, and giving information to organise the original data into useful forms. Descriptive statistical methods are introduced for KPIs including "airport peak arrival throughput", "capacity utilisation", "instantaneous workload", "dwell time in TMA", and "final approach interval". Inferential statistics rely on means of sample surveys to infer the general population: the probability density function of the data can be estimated by fitting, and the unknown probability density function can also be reached by non-parametric methods. Inferential statistical methods, specifically the kernel density estimation (KDE), are used for the "dwell time in TMA" and "final approach interval" KPIs.

Clustering through the DBSCAN algorithm
Clustering through the DBSCAN algorithm is for evaluating the coherence of trajectories, investigating the percentage of abnormal trajectories. In previous studies, trajectories clustering problem was always solved by using the Douglas-Peucker algorithm [27] to compress trajectories, and then the similarity matrix was constructed based on the normal distance of trajectory points. However, due to the too enormous dimension of the similarity matrix between trajectories, it is easy to fall into the curse of dimensionality. This paper adopts clustering through feature selection to improve calculation performance and reduce the time cost. Specific steps are as follows. a) According to the research results of the frontline work, the initial feature set is constructed to characterise trajectories, as shown in Table 2. b) Combine Laplacian score [28] and Spearman's rank correlation coefficient [29] for feature selection. c) DBSCAN algorithm is applied to cluster trajectories by the normalised distance matrix and identify abnormal trajectories. d) Using the idea of contradiction, the higher the percentage of abnormal trajectories, the worse the coherence of trajectories. What needs illustration is that we use longitude/latitude and heading as features because they record the entire flight process of a plane and can distinguish trajectories well. But taking these values at each timestamp into calculation can result in dimension curse. Therefore, the mean values are taken to represent each trajectory with the same dimension.
can result from two aspects: strong traffic demand and effective control work. This is a preliminary evaluation of AMAN's effect on throughput.
Capacity utilisation. The ratio of the number of actual landings (acceptance) to the number of flights that need to be controlled (demand) per hour. It is a measure of accommodated demand compared to the available capacity of the facility, evaluating the working effectiveness. This KPI gives more details about whether the good performance in throughput comes from the improved ATM work capability.
Instantaneous workload. This KPI calculates the number of arrival aircraft flying in TMA per unit time (10 minutes in this paper), assessing whether AMAN is playing a role in reducing the overall AT-COs' workload. AMAN is expected to provide more tactical air traffic control (ATC) work plans to prevent the influx of arrival aircraft within a short time.
Dwell time in TMA. The time it takes for the arrival flight from entry fix (EF) to runway, which is considered as one of the main benefits of AMAN [2]. Arguably, shorter dwell times indicate landing aircraft spend less time at low altitudes and then lead to systematic benefits in terms of fuel and other direct operating cost savings.
Final approach interval. The flight time intervals between approaching flights when reaching a designated position (set as horizontal 3,200 meters before the midpoint of the landing runway centreline). This KPI represents aircraft landing tightness and ATM work effectiveness. Under the premise of satisfying separation minima, final approach intervals are expected to be as small as possible.
Coherence of trajectories. The extent to which the trajectories using the same EF and arrival route are clustered together. Unified workflow and strategy are easy to lead to great efficiency. Scattered trajectories mean the aircraft operation is artificially deployed to a considerable extent, resulting in increased ATC pressure as well as reduced ATM efficiency.
In this paper, a comparative analysis method is established to support evaluations for indicators. The method substantially involves statistics and machine learning, and in details, descriptive statistics, inferential statistics, and clustering through the DBSCAN algorithm based on feature selection are used according to the characters of different KPIs.

Statistical methods
Statistics can be divided into descriptive statistics and inferential statistics.
The CSX's AMAN system is Barco's OSYRIS. Specifically, a relatively basic version is used to provide optimised landing sequences with time to gain and time to lose of arrivals for the ATCOs.
One of the main obstacles in conducting comparative experiments is to ensure that the system conditions are identical or at least closely resemble in all data collection periods. Unfortunately, there are too many variations affecting the operation. Two criteria to identify the similarity of periods are established as follows. (1)  UFT is a dwell time that is not affected by other factors under the current ATCOs' working capacity and it is specific to EF. The 25% quantile of flight time under northbound operation in the P. I is defined as the UFT, which is 979, 874, 726, and 846 seconds via EF A, B, C, and D, respectively.
Although there are many changing factors in the environment, their impact on the arrival operations will ultimately be reflected in whether aircraft could land at the airport on time. We mainly judge the similarity according to the following criteria (1): if not much difference appears in the percentages of the three periods, there are no adverse weather conditions or unexpected events. Then we use the criterion (2): if the percentages are similar, it means that the workload of each control sector is similar among the periods.

DATA PREPARATION
There are two parallel runways in the CSX, operating in segregated mode. The TMA has five EFs, among which the western two EFs share the same standard terminal arrival routes; for easy calculation, these west EFs are regarded as one EF. Starting from the north side, EFs are simply marked as the letters from A to D in a clockwise direction. The ANSP has been delivered ATCOs training about AMAN since the summer of 2017. In June 2018, AMAN system was installed in the CSX and officially started operations from November 2018. This research has begun with the data collection and analysis work in November 2017.  instances and 47 instances of hourly arrival throughput exceeding the AAR in P. II and P. III, respectively, and the maximum throughput reaches around 24 flights. Higher frequency of hourly arrival throughput exceeding the AAR brings increments of 104 and 86 flights in P. II and P. III. The significant improvement in hourly arrival throughput during peak periods indicates that with decision supports from AMAN system, ATCOs are relieved to have more energy to deal with more flights and work during busy times becomes more efficient.

Evaluation of the capacity utilisation
The mathematical expressions of capacity utilisation are shown in Equations 1-4 [24].
where, CU i,j denotes the capacity utilisation of hour j on day i. A i,j and D i,j denote the acceptance and demand of hour j on day i. N i denotes the number of arrival aircraft on day i. t i,n,f denotes the time when aircraft n flew over EF f on day i. T f denotes the UFT of EF f. CU k and N k denote the capacity utilisation and the number of arrivals in month k.
The result of capacity utilisation in P. I, P. II, and P. III is 97.45%, 97.65%, and 97.88%, respectively. Capacity utilisation increases slightly while remaining stable overall.
Moreover, the weather conditions in the same season are usually alike. As a result, December in 2017, 2018, and 2019 are included in the experiment. December 2017 is set as the P. I when the AMAN system was not assembled, December 2018 is P. II when the AMAN had just been assembled, and December 2019 is the P. III when the AMAN had been operating steadily for quite a time. Furthermore, since the runways of the CSX temporarily operated southward for one day, three days, and one day during P. I, P. II, and P. III, respectively, only northbound arrival aircraft is taken into account, as listed in Table 3.
In light of no adjustments in the airspace structure as well as in the departure/arrival procedures and no changes in the configuration of the number of AT-COs per control seat among the three years, it can be inferred that the selected periods are highly similar within an acceptable range, as the maximum percentage difference is 1.77%.

Evaluation of the airport peak arrival throughput
As of the fourth quarter of 2019, the ANSP announced the normal airport acceptance rate (AAR) in the CSX is 17 flights per hour (COVID- 19 had not yet affected the civil aviation industry). Table 4 shows flight amounts exceeding the AAR among the three periods. The situation that flight amounts exceed the AAR is safe. In China, the AAR announced by the ANSPs is a very conservative value and it is always lower than the actual values during peak hours. There are only 16 instances of hourly arrival throughput exceeding the AAR in the P. I, and the maximum hourly arrival throughput is 21 flights which occurs 3 times. In contrast, there are 57 make ATCOs unadapted or dramatically increase their workload because it is completely acceptable to deal with less than six arrivals in ten minutes. More exciting results can be discovered in Figure 3 when combing the instantaneous workload with the mean dwell time. For almost every value of instantaneous workload, mean dwell time in P. II and P. III is shorter than that in P. I, and the value in P. III is even shorter than that in P. II, except when the instantaneous workload is eleven. To figure out the reason, uncommon trajectories were found at the instantaneous workload of eleven on 2 December 2019, as presented in Figure 4. By verifying the information with the ATCOs, the ANSP issued restrictions on this day, causing more radar vectoring and holdings. Thus, the increased mean dwell time in P. III when the instantaneous workload is eleven does not

Evaluation of the instantaneous workload
Instantaneous workload concerns the work pressure of ATCOs. The statistics has shown that the value of this KPI ranges from one to twelve. The frequency of instantaneous workload among the three periods is summarised in Figure 2.
In P. II and P. III, the frequency of high instantaneous workload is basically the same as or a little lower than that in P. I, but the frequency of instantaneous workloads with values smaller than 6 rises significantly.  Descriptive and inferential statistics yield consistent results: with the operation of the AMAN system, dwell time in the TMA decreases significantly and time-related costs are reduced.

Evaluation of the final approach interval
The results of the descriptive statistical analysis calculating the final approach interval between arrival aircraft when reaching the designated position are presented in Table 5. The mean and the median in P. II and P. III are all lower than those in P. I, and the reduction is much larger in P. III. But the standard deviation in P. II as well as in P. III is larger than that in P. I, the difference between the data in these two periods being relatively large.
The smaller the width of the distribution fitting curve, the smaller the difference between the final approach intervals; and the more leftward the distribution fitting curve, the shorter the intervals. An obvious leftward tend of the distribution fitting curve can be found in both P. II and P. III, whereas P. III showed the best performance. represent the normal ATM performance in this period. It can be therefore concluded that the operation of the AMAN produces a significant reduction in the ATCOs' workload.

Evaluation of the dwell time in TMA
Counting dwell time in TMA, Figure 5 investigates the proportion of dwell time in different value ranges. It can be found in small ranges, the proportion in P. II and P. III is larger than that in the P. I, in contrast; as for large ranges, the proportion after the installation of AMAN is smaller. Moreover, it can be accordingly inferred that the reduction is directly proportional to the ATCOs' proficiency with AMAN since the changes in P. III are more obvious than that in P. II.
Taking Gaussian distribution as the kernel of KDE, the fitting curves of the dwell time are presented in Figure 6.
The cumulative frequency curves of P. II and P. III approach the higher cumulative probability faster, which means there is a significantly high probability of short dwell time in P. II and P. III. What is more, the approaching rate in P. III is the highest.  After data nominalisation, we set parameters of the DBSCAN algorithm as: the scan radius Epsilon=0.055 and the minimum number of samples in the scan area MinPnts=6. The total number of abnormal trajectories in P. I, P. II, and P. III are 935, 960, and 401 respectively, accounting for 15.15%, 15.14%, and 5.86%.

b) The uncommon trajectories
The percentage of abnormal trajectories in P. II is practically the same as that in P. I, but the value declines sharply when it comes to P. III. The abnormal trajectories detected by the DBSCAN algorithm are the scattered ones, lacking regularity. Using the idea of contradiction, the percentage of Descriptive and inferential statistics yield consistent results on the whole: the AMAN system provides shorter final approach intervals and makes the landing operation tighter.

Evaluation of the coherence of trajectories
Based on the initial features of trajectories shown in Table 2, the feature selection is implemented before clustering. The selected features are expressed in Equation 5.    aware that the experiment could still be conducted more precisely. These are the further research directions of this paper.