Dakic, Igor; Mladenovic, Milos; Stevanovic, Aleksandar; Zlatkovic, Milan Upgrade Evaluation of Traffic Signal Assets: High-resolution Performance Measurement Framework

Agencies that have large-scale traffic signal systems under their purview often have to face asset upgrade decisions. As one of the most advanced traffic control technologies, Adaptive Traffic Control Systems (ATCS) are among the options that must be taken into consideration. Having in mind the complexity of benefits and costs stemming from ATCS investments, there is a need for information-rich performance measures (PM) used in the evaluation and decision-making. However, individual PMs are often not suitable for evaluating the multidimensionality of ATCS operations, due the inherent variability of ATCS control parameters. To expand the range of PMs used in ATCS evaluation, this research develops a new PM, i.e., average arrivals on green ratio, and proposes a refinement of average delay PM to account for queue formation. The paper also presents an application framework for a multi-criteria analysis, assuming a combination of the proposed and existing PMs. In addition to presenting the analytical PM formulation, the evaluation methodology uses microsimulation for a case study comparison between actuated-coordinated and ATCS operations. The results include a comparison between previous and proposed PMs, based on the processed simulation data as well as field data. In conclusion, the proposed PMs have a high transferability potential, low data collection cost, and high data quality, thus being suitable for use in decision processes for signal asset investment. Finally, this research opens up further opportunities for advancing decision-support methods for traffic operations asset management.


INTRODUCTION
Intelligent Transport Systems (ITS) are increasingly important assets due to physical and economic constraints on large infrastructural investments, especially in urban environments. As an important element of ITS worldwide, traffic-signal systems are among the most cost effective investments [1]. In practice, agencies having to operate large-scale traffic-signal systems are facing the question of upgrade investments into these assets [2]. One of the options for system upgrade that agencies have to consider are Adaptive Traffic Control Systems (ATCS). As the most advanced traffic signal system technology for coordinated network control, ATCS continuously makes small adjustments of signal-timing parameters in response to changing traffic demand and patterns [3]. Thus, ATCS offer a promise of improving network-wide efficiency in dealing with fluctuating travel demand, reducing signal-retiming costs, and providing large amount of real-time performance data. Although most of these systems started as research projects of the academic community, today most of them are commercial products, having cannot be easily used to compare various traffic signal alternatives, as it does not provide information on how efficiently the allocated green time is being used [28]. Findings suggest that POG needs to be combined with percentage of green time (GT) and cycle length (CL) to provide a holistic picture of traffic progression. In addition, a simplified version of AD calculation from high-resolution data significantly underestimates true average approach delay. As a result, POG and AD by themselves cannot provide proper explanations as to whether the major contributions are related to offset setting, proper GT allocation, or any other parameter configurations. This, in turn, suggests that there is a need for complementary additional PMs, which can provide multidimensional understanding of traffic operations.
This paper aims to advance multidimensional understanding of ATCS operations by proposing a new high-resolution PM, Average Arrivals on Green Ratio (AAOGR), as well as by modifying the existing AD calculation, with an evaluation of both PMs. Although these measures are transferable for comparison of different actuated network traffic signal systems, in this paper we focus on comparison between ATCS (in particular, InSync ATCS) and actuated-coordinated control. To this end, the next section will present the formulation of PMs. The third section presents evaluation methodology. The fourth section presents case study results, including an exemplary multi-criteria analysis, performed combining the newly proposed AAOGR and modified AD with the currently available PMs. The final section presents discussions and conclusions, including directions for future research. many operational similarities but also many differences. Some examples of well-established ATCS include SCOOT [4], SCATS [5], and MOTION [6], but new ATCS are continuously emerging on the market. In practice, ATCS are usually vendor specific and delivered in the manner of "a black box", due to the proprietary in-built modeling and optimization algorithms [7]. Despite the promises of wide range of benefits, ATCS might not always prove beneficial for networks with predictable or oversaturated conditions, often have long setup time and steep learning curve, as well as higher maintenance/operating costs [8].

ANALYTICAL FORMULATION
Having in mind the complexity of benefits and costs stemming from investments in ATCS, the increasing number of agencies worldwide that are considering procurement of ATCS are facing the need for an improved knowledge base [9,10]. Previous research informs us that it is important to develop detailed functional requirements for investments into traffic operations assets [11,12]. In addition to the development of functional requirements, it is imporant to compare the operation of the existing with the proposed traffic signal system in order to support investment decision-making. The usual comparison of ATCS in previous literature is on a case-by-case basis, with a before-after analysis approach [7,[12][13][14][15][16][17][18][19]. Evaluation is usually done using hardware-in-the-loop or softwarein-the-loop simulations, with occasional pilot field studies, due to their investment requirements. However, one significant challenge for ATCS evaluation is the very nature of actuated control, having variable signal timing parameters. For adaptive operation, conventional performance measures (PMs) which are often based on conventional traffic flow measures do not provide enough details for capturing the multidimensionality of operational effectiveness [20,21].
Accounting for the need to produce information-rich PMs, recent advances in controller logging capabilities, high-level data storage methods, and communication technologies have enabled collection of high-resolution signal operations data [22,23]. Resulting data with time-stamped detector and phase state changes have enabled the development of novel PMs. Nowadays, there is a range of PMs in use (e.g. phase termination, yellow/red actuations, arrivals on red, approach speed, etc.), especially related to the quality of the main-road arterial progression [24]. However, having the possibility to establish new PMs, one has to account for common issues in performance measurement, including such aspects as data collection cost, data quality, transferability of comparison to other location or time, and liability for action [25,26]. Thus, previous research has suggested focusing on some of the PMs suitable for evaluating traffic progression, such as percent on green (POG) [27], platoon ratio (PR), and approach delay (AD) [22]. A previous study informs us that POG as a single performance measure allows one to clearly see the difference between the PMs, as cycle length and green time do not vary, i.e., {C i =C/g i k =g k ,6i!N}. One can observe that, in this simplified scenario, PR actually represents a product of AAOGR and cycle length (C) (Equation 7). Consequently, the AAOGR represents the percentage of vehicles that have passed through the intersection per second of green time, while PR is an index without units.
Furthermore, comparing the previous case with a more complex one, when cycle length and green time vary in almost each cycle, i.e., {C 1 ≠C 2 ≠ ... ≠C | N | , g 1 k ≠g 2 k ≠ ... ≠g k | N | }, as in the case of ATCSs, illustrates the potential of using AAOGR. From Figure 1 below one can conclude that PR fails to take cycle length change into consideration, while AAOGR reveals platoon efficiency. Figure 1 also illustrates the comprehensiveness of AAOGR when compared to POG, showing that POG does not provide information on efficiency of allocated green time used by an arriving platoon. Essentially, AAOGR is a PM that takes into account variability of cycle length and green time duration on a cycle-by-cycle basis, thus being suitable for ATCS evaluation. Nevertheless, the aforementioned comparison between AAOGR and PR does not preclude the possibility to use PR for assessing arterial progression, but instead indicates that several measures should be combined to thoroughly evaluate traffic operations along a given arterial, as different PMs might yield different findings. Note that it is evident that for the same cycle length (C 1 =C 2 ), but different green time distributions (g 1 >g 2 ), POG can take different values (POG 1 >POG 2 , assuming that V g ,1 >V g ,2 ). This is the reason why POG, as a single PM, cannot be easily used to compare various signal control strategies and why more comprehensive PMs have to be proposed. One of such measures is platoon ratio (PR) (Equation 3), which takes into account the percentage of green time within a cycle (  For the purpose of constructing a decision matrix, we define weights w m for each PM, where m!M={POG,PR,AAOGR,AD}. The total score for a given intersection approach and traffic control system can then be computed as: where v k m,g,r and v k m,g,r stand for the observed and normalized values of PM m for control system g!G and approach k at intersection r!I. Note that in case of POG, we use 100% as the normalization factor. For other high-resolution PMs, the normalization factor is defined as the maximum observed value across all intersections and traffic control strategies. This is because it is relatively difficult to determine a theoretical maximum value that can be achieved for these PMs, as it depends on the prevailing signal timing parameters and the traffic demand (Equations 3, 4, and 8). It is also worth mentioning that the normalization factor for AD is defined to be inversely proportional to the one for PR and AAOGR, given that a higher value for AD degrades the traffic performance. In general, the weights are used for discerning how significant each PM should be, and are usually assigned by a range of experts in transport engineering. Alternatively, one can perform a sensitivity analysis and see how different the results are when a different weighting system for PMs is used. However, such analysis is beyond the scope of this paper.

CASE STUDY SETUP
In order to perform a validity evaluation for the proposed PMs and multidimensional analysis of traffic operations, the methodology is based on a microsimulation model, having the capacity to generate high-resolution signal and detector data (i.e., VISSIM 5.3). The evaluation includes a comparison

Approach delay
In this section we propose modifications of the existing model for computing approach delay After the fifth vehicle, the headways leveled out at

Multi-criteria evaluation
Here we combine all aforementioned high-resolution PMs, showcasing the potential for a multidimensional analysis of arterial performance. The methodology follows a general multi-criteria decision matrix, including a set of alternatives, criteria, and their scores and weights (Figure 2). For an elaborated version of a decision-framework, the reader is referred to [2].
for afternoon peak period traffic. The resulting cycle lengths vary from 80 to 160 seconds. The setup of the detection system for InSync had to resemble video detection systems, represented as a series of detectors up to the point of advance detection. Thus, only one detection phase is activated at a time, based on what InSync calculates as optimal phase sequence and duration. Moreover, detector size, location, and between actuated-coordinated, time-of-day (TOD), and ATCS operations. The evaluation includes a comparison between AAOGR and PR, AAOGR and POG. In addition, the evaluation includes a validation of the refined PM for AD, in comparison with average delay per vehicle, as given by the microsimulation model, as well as with the actual field data provided by the Utah Department of Transportation (UDOT). For the comparison with the field data, only five intersections were compared, due to data limitations. In particular, adaptive operation is simulated as software-in-theloop of the novel market ATCS InSync. InSync has two major aspects of operation. First, InSync automatically adjusts local signal timings, and second, it coordinates signals along the arterial according to the demand. The spatial study scope was a 6.4 km signalized corridor of SR-421 in Port Orange, Florida, from Williamson Blvd to US-1 ( Figure 3).
In total, there are ten signalized intersections on the corridor. The temporal study scope was the afternoon peak period. Annual Average Daily Traffic varies between 25,000 and 35,000 vehicles/day, depending on the corridor section. Signal timing plans for actuated-coordinated control were developed accounting intersection (PR=1.10). Moreover, the AAOGR results show that for a given C and GT, TOD plans outperform InSync. Table 1 displays many contradicting results, showing that these three measures (POG, PR and AAOGR) generate a variety of conclusions. Consequently, their inclusion in a common decision-support framework would be beneficial, as it would provide evaluation of signal performance from multiple dimensions at once. For example, AAOGR has shown greater values at intersections that provided poorer progression in terms of POG (Clyde Morris and Nova in InSync scenario with 1.43% and 1.54% for AAOGR, 60% and 68% for POG, respectively; Spruce in TOD scenario with 1.72% for AAOGR, 63% for POG). A similar relation can be seen between POG and PR. In the InSync scenario, the maximum value for PR is at Nova, where a relatively small POG was recorded (68%) compared to the other intersections. On the other hand, the minimum value for PR is at Taylor Rd (1.10), where a high POG (87%) was obtained. The TOD scenario also shows that the highest value for PR was recorded at Nova (1.79), with the lowest POG (60%). I-95 NB and Taylor Rd were intersections with the lowest PR (1.10 for both intersections) compared to a high POG (78% and 80%, respectively). An interesting illustrative example is the intersection of Yorktowne and SR-421, where the same results (73%) for InSync and TOD were obtained for POG, with similar values for GT (67% for InSync; 65% for TOD). If PR was taken into consideration, TOD would have a small advantage over InSync (1.14 > 1.08), whereas the AAOGR shows a clear advantage for evaluating TOD operation (1.24% > 0.98%). Figure 5 shows differences between AAOGR and PR, while Figure 6 shows differences between AAOGR and POG, for InSync and TOD operations at various intersections. One can notice a higher level of compliance between AAOGR and PR ( Figure 5) than between AAOGR and POG ( Figure 6). function were modeled according to the NEMA standard. To enable such flexibility, it was necessary to set up controllers in VISSIM to operate in an uncoordinated mode. More information about the calibration and validation of the simulation model, development of TOD signal timings, and steps for integrating InSync with microsimulation software can be found in [31].
Multiple simulation runs with different random seeds were executed for both InSync and TOD scenarios. Each simulation lasted two hours and 15 minutes, with 15 minutes of warm-up time and two hours of evaluation time. To compute AAOGR and AD based on VISSIM outputs, we post-processed VISSIM's detection and signal phasing, as well as timing log files using the algorithm shown in the form of pseudo code above (Figure 4). One should note that, for evaluating the proposed AD model, VISSIM's AD from the simulation was obtained through the node evaluation, where appropriate node boundaries (for incoming vehicles) were placed to coincide with outer edges of the relevant approach detectors. Similarly, the node boundaries for outgoing vehicles were placed at the stop lines. This way, simulation outputs were adjusted to closely match conditions needed for high-resolution data collection and were shifted to avoid overestimating delay by considering conditions not observed by detectors.

Comparison between ATCS and TOD signal operations
The comparison between ATCS and TOD results in Table 1 indicates the variance in high-resolution PMs. For example, if one was to judge InSync and TOD performance at Taylor Rd based on POG, they would conclude that InSync (POG=87%) outperforms TOD plans (POG=80%). A quick look at the PR would show that these two systems perform equally well at the same 45% when compared to the discharge rate-based AD model, because it is not taking into consideration the possible queue build-up during red time duration.

Approach delay validation
To determine how well a new delay model computes AD, average delay per vehicle (as given by VIS-SIM) was compared with the proposed high-resolution AD as shown in Figure 7. This figure shows that AD derived from high-resolution simulation data closely matches VISSIM's own approach delay, as the model is quite reliable, R 2 is around 93%). In addition, Figure 8 demonstrates the relationship between high-resolution AD obtained by the current UDOT methodology and high-resolution approach delay obtained by the proposed discharge rate-based model. The value of the coefficient of determination (90.57%) indicates a strong correlation between the field data and the proposed model. However, further analysis shows that the current methodology underestimates delays by around Similarly, the Village intersection shows that, even though InSync achieved better results for AAOGR (0.20%) compared to TOD (0.16%), it provided higher AD, resulting in a lower overall score. On the other hand, at I-95 SB we can observe that both InSync and TOD performed the same in terms of AAOGR. However, when other PMs are also taken into consideration for computing the total score, the results are in favor of InSync. Last, one should note that by using the proposed approach, it is possible to draw the conclusions on which system performs better for a given arterial, rather than focusing on individual intersections/approaches (e.g. in this case InSync, as shown in the last row of Table 2).

CONCLUSIONS
Taking as a basis the importance of detailed ATCS evaluation for asset investment decisions, this study proposed and evaluated two PM models based on high-resolution signal control data. The first model for evaluating signalized intersection performance introduces a new PM, Average Arrivals on Green Ratio (AAOGR), further used for multidimensional analysis of arterial performance. This PM is suited for adaptive operations as its analytical formulation enables to take into account the ratio of vehicles that passed through the intersection per second of green time. The second model proposes a new methodology for computing approach delay (AD) by taking into account the possible queue formation during red signal. In addition to the analytical formulation, the evaluation methodology uses microsimulation for a case study comparison between actuated-coordinated and adaptive operations. The evaluation is based on the calibrated and validated signalized network including ten intersections, accompanied with a field data comparison. ATCS operation has been simulated as software-in-the-loop, while the

Multi-criteria analysis of alternative trafficoperation systems
For demonstration purposes, and given the limitations of using POG as a single PM, we define the weights for the decision matrix to be {w 1 ,w 2 ,w 3 ,w 4 }={0.15,0.3,0.3,0.2}. The results of this comparison are shown in Table 2. Remarkably, the obtained score does not completely follow the trend of any particular PM, as it takes into account all PMs at the same time. This confirms our premise that by taking into account only specific PMs one can reach different conclusions regarding arterial performance than by analyzing all metrics at once. Thus, new PMs should be combined with the existing PMs, in order to obtain a more holistic picture of traffic operations along a given corridor. The importance of combining these measures can be clearly seen at the Clyde Morris intersection, where POG and PR imply that TOD outperformed InSync, whereas the multi-criteria based score S suggests that InSync actually yielded better performance, due to substantially higher AAOGR and   Figure 8 -Comparison between proposed AD model and UDOT methodology applied to field data experts might have different backgrounds (e.g. control, maintenance, or design), and thus perceive differently the importance of same objectives. In particular, by discussing the relative criteria weights, a multi-criteria decision matrix can become a centerpiece for transparent and versatile communication, not just between experts, but also in relation to other stakeholders. Certainly, one can conclude that the benefit will not be just the suggested investment choice, but also the knowledge generated during the iterative decision-making process itself. The limitation of the proposed AD model relates to the advanced detector location, which limits the capability to infer delay for vehicles waiting in queue behind the advanced detector. Nevertheless, in such cases the already developed methodology for estimating queue length for the entire segment, also based on high-resolution event data, can be used [32]. Future research studies should consider accounting for the sensitivity of PM formulation to the queue discharge rate by taking into account the local values. In addition, further research should include similar experiments on other networks and with other ATCS, both to confirm the result of this study and to suggest further PM improvements. Finally, as there is a growing amount of investments into the ITS infrastructure, there is a need for further advancement of PMs and decision-support frameworks for investment decisions about ITS assets. actuated-coordinated operation included optimized timing. A novel algorithm was developed to derive high-resolution PMs from simulation outputs.
The results show that POG as a sole PM cannot be easily used for adaptive operations evaluation, as it does not provide enough information on efficiency of allocated green time used by an arriving platoon. In contrast, PR is an improved PM, but it does not take into account the duration of the cycle length, which is crucial for adaptive operation. Thus, a signal operating at a higher cycle could be evaluated as less efficient than the one with a lower cycle and the same green time and POG values. As AAOGR includes the necessary parameters, namely green time and cycle length, this PM can clearly depict the efficiency of a platoon of vehicles through a signal in adaptive mode. Furthermore, this PM is also suitable for evaluating signalized intersections operating at different cycle lengths on different network partitions. This, in turn, suggests that, for a comprehensive analysis and multidimensional understanding of traffic operations, different PMs should be combined, as shown in this paper. Moreover, the refined AD model shows that vehicle delay can be estimated very well, and the comparison with field data shows that the refined AD model can be applied to the existing field data. In general, both PMs are applicable for implementation in the field, even in the case of controllers not based on NEMA standard. For example, European controller standards can also provide high-resolution (10 Hz) logging data. In turn, this data can be used for performance evaluation, so the findings have transferability potential. In addition to the transferability potential, these PMs have low data collection cost and high data quality, and are thus suitable as standard PMs for asset investment decisions. Overall, the proposed PMs can provide crucial information for evaluation of ATCS considered for an upgrade.
In a larger trajectory of research in transport engineering, this study has provided an outline of potential benefits that traffic operations research can gain when interacting with asset management and decision-support research areas. In particular, an important point to underline in the process of traffic operation evaluation, particularly when an agency is considering large-scale investments into signal assets, is the need to center the process on multi-criteria methodology. Aside from the conventional before/ after studies that include a range of PMs, centering the process on multi-criteria methodology allows for transparent and uniform presentation of alternative decisions. Moreover, in combination with other methods for knowledge management, the use of multi-criteria methodology allows for an iterative discussion on the alternatives, until a mutually-agreeable solution is reached. Thus, the process can incorporate a range of perspectives, having in mind that traffic-operations