THE USE OF COOPERATIVE APPROACH IN RAMP METERING

To ensure higher Level of Service (LoS) at urban motor-ways, new traffic control concepts are being applied since in most cases there is no available space for infrastructural build-up. For urban motorways, the mostly used control methods are ramp metering combined with additional con-trol methods like variable speed limit control (VSLC). This paper gives a review of the current ramp metering approaches with special emphasis on cooperative control concepts be-tween ramp metering, VSLC, prohibiting lane changes sys-tem and the vehicle itself. Additionally, a learning framework for ramp metering proposed by the authors is described. The CTMSIM Matlab based macroscopic motorway simulator with ramp metering control support is used for the simulation of selected ramp metering approaches. The simulator is also augmented to enable the development and implementation of cooperative ramp metering approaches. The Zagreb bypass is used as test case for evaluation of several different ramp metering algorithms.


NTRODUCTION
Improved vehicle production technology combined with higher purchasing power of the citizens have induced a significant increase of the number of vehicles in the past decades.The increased number of vehicles has consequently produced high demand for exploitation of the existing road network capacities.Additionally, road networks of larger urban areas have to cope with the traffic demand of nearby smaller cities, central larger cities and transit traffic.The nearby cities are mostly connected directly by the so-called urban motorways.They are designed to provide larger maximal traffic capacity for a higher Level of Service (LoS) unlike urban road networks.Some of the urban motorways serve also as a bypass around larger cities.The overall traffic demand which affects such urban motorways consequently induces their overload.Traffic demand can have its origin in the adjacent urban traffic network or transit traffic.Additionally, urban road networks in most cases do not have any possibilities for physical enlargement of their capacity.
Despite high maximal capacity of urban motorways, they can be points of congestions.The reasons for congestions on urban motorways can be divided roughly into two major groups.The first group of reasons is classified as non-periodical and they are characterized by a sudden drop in traffic throughput of a particular motorway.They involve various types of traffic incidents, concerts, sport events, etc.Such traffic congestion reasons happen relatively independently from each other in a temporal context.The second group of reasons is in the temporal context dependent on the time interval that passes between them.Because of such behaviour, they are usually named periodical reasons.Daily or seasonal migrations are the main cause.
In a spatial context, periodically based congestions are most common on urban motorway sections with nearby on-ramps.Significant traffic flow from an on-ramp entering the urban motorway can significantly reduce the speed and increase the density of the mainstream flow.This can consequently lead to downstream congestions.Additionally, if an oversized platoon of vehicles originating from an on-ramp is stopped long enough it will produce a long on-ramp queue.On-ramp queues can grow to the proportion that they can eventually block a part of an adjacent urban road network.This is known as the spillback effect.An illustration of the mentioned problems is presented in To prevent congestion at on-ramps, a control of on-ramp flows with special traffic lights and appropriate control algorithm is implemented.Such a control system keeps the mainstream flow under its capacity and is known under the term ramp metering [1,2].Ramp metering is one of the traffic management services described in the currently actual European Intelligent Transport Systems (ITS) deployment strategy [3].can provide additional control actions to the ramp metering algorithm directly affecting the vehicle [25].This paper is organized as follows.Section 2 briefly describes the used LoS measures in the analysis of simulated ramp metering algorithms.Sections 3 and 4 describe the properties of local and coordinated ramp metering approaches including their difference.Cooperative ramp metering approaches are described in Section 5. Section 6 presents a comparative analysis between simulated ramp metering approaches including a description of the applied simulation settings.The paper ends with a conclusion and future work description.

LEVEL OF SERVICE MEASURES
Measures of quality for ramp metering are usually used as measures for the assessment of the overall motorway LoS.The basic measure of service quality for ramp metering is the travel time (TT).TT is a simple measure which describes the time one vehicle needs to travel through the observed motorway segment.It is measured in minutes.TT is computed using equation (1): , where v i (k) denotes the traffic velocity at motorway segment i, L i is the length of segment i, N is the total number of segments, k is the simulation step, and T is the simulation step length.An unusual high value of TT is a clear sign for the LoS quality drop for the examined motorway.
There are several other quality measures derived from TT.One of the simplest measures derived from TT is the Total Travel Time (TTT).TTT sums up the values of TTs on all of the observed motorway segments during the whole simulation run.Total Time Spent (TTS) is the most comprehensive measure derived from TT. TTS takes into account the mainstream density and on-ramp queues.It is expressed in vehicle-hour units.Equation (2) presents TTS [7]: where ρ i denotes traffic density at motorway segment i, and q rmi (k) is the queue on the on-ramp at motorway segment i.Total Travel Distance (TTD) presents another measure derived from TT.It represents the total travelled distance in vehicle-kilometres [veh-km] and can be obtained as: where q i denotes the measured flow on motorway segment i. Recently, numerous improvement measures of the basic TT have been developed to aid in the evaluation of motorway LoS [8].Most important are Vehicle Hours Travelled (VHT) and vehicle kilometres travelled In accordance with this, Croatia's ITS development strategy [4] is also strongly linked to the implementation of new traffic management systems and services, motivating the research described in this paper.Ramp metering can be based on local and coordinated control strategies regarding their algorithm working principles [5].Local strategies take into account only the local traffic situation, while coordinated strategies consider the overall traffic situation on the entire controlled motorway section.Cooperative control strategy, as one subcategory of coordinated strategies, is the main focus of this research.It is based on a lower control level, compared to the coordinated strategies.This is the reason why in several studies [5,16] cooperative strategies are generalized as coordinated strategies.
Coordinated strategies enable the selection between different local control activities to ensure that their global objectives are met by modifications of their original plans [6].The selection can be conducted using the higher level control module.The mentioned module governs the behaviour of all local control activities under particular circumstances.Cooperative strategies use direct communication between local control activities.By cooperating, the local control activities search for an action that is potentially suboptimal locally, but optimal for the overall system.A cooperative strategy can thus be understood as a subcategory of coordinated strategies, which resolves specific situation with conflicting interests between local control activities.This type of strategy selects a dominant control activity, and all other activities support the dominant one in order to achieve a common goal.
In this research, the cooperative ramp metering strategy is examined as a promising future development direction for ramp metering based traffic control systems.Cooperation can be also achieved between ramp metering and other traffic control systems applied on a particular urban motorway segment.Additional motorway control systems that can be part of such cooperative systems are: variable speed limit control (VSLC) [16], prohibiting lane changing, and cooperation with On-Board-Units (OBU) in vehicles.OBU (VKT).VHT is defined for a given unit of time and a given section of motorway.It indicates the amount of time spent by all of the vehicles on the motorway.Performance measure VKT is defined for a given unit of time and a given section of the motorway.It indicates the sum of kilometres driven by each vehicle on a motorway.Measure of travel Delay can be obtained as the difference between the actual VHT and the respective VHT value that the vehicle would travel at free flow speed [9].Measures such as: TT, Delay, on-ramp queue length and TTS are used in this research for the comparative analysis of the obtained simulation results.

LOCAL RAMP METERING STRATEGIES
Local strategies include ramp metering algorithms which take into account only the traffic condition on a particular on-ramp and its nearby motorway segment.The most important local strategies are ALINEA, Demand-Capacity and Percent-Occupancy.ALINEA is the most frequently used standard local ramp metering algorithm.This is because of the ALINEA's optimal ratio between simplicity and efficiency.The core concept of ALINEA is to keep the downstream occupancy of the on-ramp at a specified level by adjusting the metering rate.The specified level of downstream occupancy is called the occupancy set-point O. Its value is slightly lower or equal to the occupancy at the maximum downstream capacity [10].The resulting metering rate can be obtained by the following equation:  Demand-Capacity algorithm uses the downstream occupancy measurement data.If the downstream occupancy is above a specified critical occupancy, it is assumed that there is congestion and the metering rate is set to the predefined minimum value.Otherwise, the metering rate is set according to the difference between the downstream capacity and the measured upstream traffic volume.The basic working principle of the Demand-Capacity algorithm is shown in Figure 3.The Demand-Capacity algorithm's main disadvantage over the ALINEA algorithm is its inability to provide a proper reaction at the moment when the congestion begins to arise.ALINEA can provide reaction at the start of the congestion build-up due to their closedloop control structure and appropriate traffic sensor position.Percent-Occupancy does not require computation of the motorway capacity.This feature is important because it has direct impact on the low implementation cost.This algorithm uses two types of constants.The first constant (K 1 ) is the traffic flow and the second constant (K 2 ) can be obtained as an approximation of the uncongested part of the fundamental diagram.Parameter o in represents the measured upstream occupancy.The metering rate is computed using the following equation: Other local ramp metering algorithms used today are based on neural networks and fuzzy logic.Local algorithms based on neural networks use the learning capabilities of neural networks to produce metering plans.Learning data sets are generated using a traffic simulation model and an expert ramp control system.Neural network based control algorithms provide better results when they are used as part of coordinated ramp metering strategies [24].

COORDINATED RAMP METERING STRATEGIES
Generally, coordinated strategies involve all algorithms which take into account the overall traffic situation of the controlled motorway segment.Coordinated algorithms can be further divided into: cooperative, competitive and integrated algorithms [12].Cooperative algorithms involve additional computations between local control actions in order to resolve situations with deep conflicts between local control actions.The usual procedure is to detect the place of a bottleneck M. Gregurić, E. Ivanjko, S. Mandžuka: The Use of Cooperative Approach in Ramp Metering

COOPERATIVE RAMP METERING
The basic principles of cooperative ramp metering were already briefly mentioned in the introduction and in the previous section.Cooperation can be established between several on-ramps or it is possible to add several other motorway management strategies into cooperation with ramp metering.Higher number of management strategies included into cooperation with ramp metering contributes to a more comprehensive control over a motorway section.With better control it is possible to achieve high LoS and effectively mitigate congestions.

Cooperation between on-ramps
The merging of on-ramp flows was initially controlled in response to real-time local traffic conditions.The main problem with locally responsive ramp metering is their inability to resolve downstream congestions.To tackle this problem, cooperative ramp metering was proposed.In order to enable cooperation between several on-ramps, it was necessary to develop a centralized control module.It enables an override of the locally-determined metering rates when necessary.A comprehensive evaluation study done in Denver, Colorado in 1988 and 1989, has shown that the central override is effective in mitigating congestion on the mainline if the speeds are less than 90 [km/h] [14].
HELPER is a common cooperative ramp metering algorithm.It uses local traffic-responsive algorithms combined with a central override control connected as shown in Figure 4.Such a two-stage control architecture is common for cooperative control applied in ramp metering.To measure the local traffic conditions, the mainstream occupancy is obtained by detector stations.Furthermore, a central module also monitors the size of on-ramp queues.The override feature can adjust the metering rates in case of any excess queue development.If an on-ramp operates at the minimum metering rate and occupancy on the queue detector exceeds a pre-determined threshold value, the particular on-ramp is categorized as "master".The centralized module increases the rate at the "master" on-ramp by one level and reduces the metering rate for the upstream on-ramp by one level.The distribution continues one level at a time to other upstream on-ramps.These upstream on-ramps are categorized as "slave" on-ramps.The main idea is to exploit their queuing capacity in order to mitigate downstream congestions [14].

Concept of cooperative ramp metering learning framework
Recent work of the authors described in [22] and [24] includes the use of Adaptive Neuro-Fuzzy Inference System (ANFIS) to teach a ramp metering and enrol several upstream on-ramps to create virtual on-ramp queues.Virtual queues have the primary role to stop forwarding additional traffic flow into the mainstream in order to mitigate upstream congestions.Typical representatives of such algorithms are: HELPER and LINKED [12].The HELPER algorithm was the first developed algorithm based on cooperation.It includes several local traffic responsive metering algorithms which communicate with a centralized operational unit containing an override possibility.
Competitive algorithms contain two control logics: a local and a global control logic.During execution of the ramp metering algorithm, each local control logic provides an appropriate solution for the current traffic situation.The more restrictive ramp metering value is chosen as the final one.Typical representatives of these algorithms are: Bottleneck and SWARM [12].The Bottleneck algorithm has two components which provide two different metering rates.The first component calculates a local metering rate based on occupancy control that selects a ramp metering value from a finite set of discrete predetermined metering rates according to the upstream occupancy.The second component calculates the so called bottleneck metering rate.A section is identified as a bottleneck if it satisfies two conditions, i.e. the capacity condition and the vehicle storage condition.The bottleneck metering rate is calculated to keep the flow of traffic at a defined bottleneck below capacity.The System-Wide Adaptive Ramp Metering (SWARM) algorithm is the most effective competitive algorithm [13].It contains two types of control algorithms: SWARM1 and SWARM2B.SWARM1 algorithm conducts global coordination by taking into account the traffic state on each on-ramp.It can predict bottlenecks on the motorway using short-term historical data.SWARM2B is a local algorithm and defines the metering rate according to the difference between the current and the critical traffic density for a particular on-ramp [13].
Integrated algorithms contain a control module based on an optimization engine with defined boundaries and a goal that has to be achieved during the control period.Typical representatives of these algorithms are: METALINE, FHWA/BALL Space, DYNAMIC, and fuzzy logic based algorithms [12].Fuzzy logic based algorithms are the most sophisticated in this group.They can be described as one type of expert systems for ramp metering.Fuzzy logic based algorithms make their decisions by using converted empirical knowledge about traffic flow parameters and ramp control into so-called fuzzy rules.The rules contain inputs in logical relations and their impact on the particular traffic parameter is defined as the rule output.Fuzzy logic based algorithms are suitable for ramp metering because fuzzy logic is ideal for making decisions which are based on inaccurate input data e.g.inexact traffic model and noisy sensor measurements [11].algorithm.In [23] the application of ANFIS is described only for local ramp metering and in [24] the ANFIS approach is augmented to obtain a cooperative ramp metering learning framework.The ANFIS approach which provides a cooperative ramp metering learning framework will be denoted as ANFIS ramp metering cooperative (ARMC).
The ARMC concept is oriented towards mitigation of the congestions which vary in strength and in time.This is done by the use of ANFIS self-adaptation properties.The most noticeable property of ANFIS is its self-tuning of Fuzzy Inference System (FIS) control parameters [23].Primarily, the ANFIS algorithm uses an artificial neural network (ANN) to modify the parameters of a Takagi -Sugeno FIS according to the learning dataset.ANN optimizes its interconnection structures through an unsupervised learning method.ANFIS is trained using a hybrid learning algorithm (combination of feedback error propagation and least squares method).FIS is part of ARMC, which actually provides control over the metering rates on every on-ramp.
To teach an ARMC ANN it is necessary to acquire knowledge from the chosen ramp metering algorithms.They are denoted as teaching algorithms.ALIN-EA is chosen as a local, HELPER as a cooperative and SWARM as a competitive ramp metering teaching algorithm.The mentioned algorithms have mutually different control logics which enables ARMC to resolve different types of congestions.The next step is to create a learning dataset which contains different types of traffic parameters according to the simulation results of all the mentioned algorithms [24].After the creation of a learning dataset, the best solution between all the possible solutions provided by all the simulated ramp metering algorithms has to be selected.This is done by using the following criteria function: where f(r) is the metering rate function, TT is the travel time, and D is Delay.TT is multiplied by a higher ponder regarding Delay.This means that the ARMC algorithm should learn solutions which have an emphasis on lower TT values.Lower values of TT are suitable for urban bypasses which have the primary role to serve transit traffic.Suitable inputs and outputs among traffic variables relevant for ramp metering are selected for teaching by using brute force optimization [23].The proposed model has two input variables (e.g.x -density of main traffic flow, y -on-ramp demand).Each input variable has five membership functions and one output in the form of a ramp metering rate value.For fuzzification the classic fuzzy theory method is used.The middle of maximum method is used for defuzzification.

Cooperation between ramp metering and variable speed limit control
VSLC is in most cases used as a standalone traffic management system on motorways which use Variable-Message Signs (VMS) to inform the drivers.As a standalone application, the main goal of VSLC is to homogenize vehicle speeds.The reduction of speed differences among vehicles and the mean speed differences between lanes provides suppression of shock waves [15].Simultaneously, the traffic safety is increased.In [15,16] a possible cooperation between VSLC and ramp metering system including the benefits of such cooperation is described.These  M. Gregurić, E. Ivanjko, S. Mandžuka: The Use of Cooperative Approach in Ramp Metering ellite System (GNSS).Roadside beacons in case of EVSC can be replaced with advanced ramp metering control computers.Communication between the ISA in-vehicle system and the road infrastructure enables automatic vehicle speed control according to the relevant speed limit obtained using the current traffic condition on nearby on-ramps.The basic activity diagram of an On-Ramp Metering System (RMS) based on ramp metering-to-vehicle communication (r2v) can be seen in Figure 6.

Cooperation between ramp metering and prohibiting lane changes
The process of initiating change from the right (slower) to the middle (faster) lane in the mainstream traffic flow is called Prohibiting Lane Changes.The goal of this additional traffic control system is to clear the right mainstream lane (the lane closest to the merging lane) from vehicles, and to enable a quick and safe merging process.This control approach must be supported by appropriate traffic signalization or a driver information system.It can work stand-alone or in cooperation with a ramp metering system.The cooperative approach uses VMS to inform the drivers when they need to change from the right to the middle lane.The periods when VMS informs the drivers to initiate lane changing depends on the motorway segment traffic situation, the current controlled on-ramp properties and the applied metering rate.
Prohibiting lane changes presents the most restrictive traffic management measure.It can significantly approaches consider two types of cooperation.The first cooperation approach uses VSLC based on VMS in order to suppress shock waves which are created near on-ramps [16].The cooperative ramp metering engine uses VSLC to gradually slow down the mainstream flow on several downstream motorway sections.Additional slowdowns induced by VSLC reduce the number of incoming vehicles to the place of congestion.This is a typical scenario of cooperation between ramp metering and VSLC in which they work together in order to resolve or mitigate congestion faster.Illustration of a VMS-based VSLC concept can be seen in Figure 5.
The main problem of VSLC systems based on VMS is that the mainstream drivers do not fully obey the posted speed limits [17].The solution of this problem is to override the current vehicle speed by an in-vehicle imposed speed limit.This is the core concept of Intelligent Speed Adaptation (ISA), the second type of cooperation between ramp metering and VSLC.In order to enable such a system each vehicle has to be equipped with an appropriate on-board unit (OBU).The ISA system monitors the location and speed of a vehicle, compares it to the defined variable speed limit, and takes corrective action such as advising the driver and/or limiting the vehicle maximal speed [18].
Recently there has been a growing interest in the potential of ISA, also known as External Vehicle Speed Control (EVSC) [19].External control is achieved by a communication infrastructure in the form of roadside beacons or autonomous systems using an on-board digital map combined with the Global Navigation Sat-

Figure 5 -Concept of cooperation between ramp metering and VSLC based on VMS
decrease maximal mainstream capacity of a particular motorway segment with an on-ramp.It is reasonable to apply the prohibiting lane changes when there is generally low traffic load or when an on-ramp queue is extremely long.Such a scenario implies that the applied ramp metering algorithm cannot resolve the on-ramp queue by itself.Illustration of a traffic control system with prohibiting lane changes included can be seen in Figure 7.

Cooperation between ramp metering and vehicles
Cooperation between the vehicle OBU and the onramp control computer (RMS-r2v) is presented in this research only on the conceptual level.The aforementioned system is introduced only at the conceptual level due to its complexity.This research proposes the establishment of this system at the moment when a vehicle stops at the on-ramp end and is waiting for green light.At the moment when the green light is turned on, the on-ramp control unit obtains throttle control over the first vehicle in queue.The vehicle movement starts automatically preventing that an inexperienced driver fails to leave the on-ramp during the short green light phase.
Additionally, the vehicle OBU can receive also information about the mainstream merging manoeuvre.The types of mainstream merging manoeuvres depend on the subsystem which is in cooperation with the ramp metering control system.If selectively prohibiting lane change is in cooperation with ramp metering then the on-ramp computer forwards only simple merging trajectories to the vehicle OBU.When a vehicle becomes parallel with the mainstream direction in the rightmost lane, the on-ramp control computer terminates its control over the vehicle and the driver continues to control the vehicle.OBU also provides appropriate information to the driver when the remote automatic control over a vehicle is established and when it is terminated.The diagram of basic RMS-r2v activities can be seen in Figure 8.

Augmented CTMSIM Traffic Simulator
In this research the CTMSIM simulator is used.It is an interactive simulator based on macroscopic traffic models specifically designed for motorway traffic flow simulations.Macroscopic-based simulation tools are better and faster for evaluating coordinated ramp metering algorithms on larger motorway corridors.The reason for that is satisfying the ratio between simplicity (which enables faster simulation speed) and accuracy of their mathematical model.CTMSIM is based on the Asymmetric Cell Transmission Model (ACTM) and

COMPARISON OF RAMP METERING APPROACHES
This section presents the augmented CTMSIM motorway traffic simulator, and the results obtained from a simulation-based comparative analysis between stand-alone and cooperative ramp metering algorithms, and VSLC.The constructional parameters of the Zagreb bypass and traffic data from the Ljubljana bypass are combined in order to define the parameters for the motorway simulation test model.allows user-pluggable on-ramp flow and queue controllers [9].In CTMSIM on-ramp flow controllers are based on a collection of ramp metering algorithms which are already implemented in CTMSIM.The collection contains: ALINEA, SWARM (SWARM1 and SWARM 2B), proportional in congestion and ideal ramp metering algorithm.For this comparative analysis CTMSIM is augmented with several newly added features as can be seen in [22].Here only a brief description of CTMSIM is given and more details can be found in our previous paper [22].
The original CTMSIM simulation sequence goes only through defined cells in a particular time step.In the implemented augmentation, additional simulation step is added at the end of each time step [22].It is computed after all cells have been simulated.This additional simulation step is related to the cooperative ramp metering module and it has access to data from all cells.So, optimal local ramp metering rates including VSLC values used in the next time step can be computed.Final modification of CTMSIM enables the implementation of VSLC for any cell in the simulation model.VSLC is implemented through modification of the following expression: where is the traffic flow in i-th cell, and v i ff (k) is the free flow speed in i-th cell.

Configuration of the Zagreb bypass model
The Zagreb bypass is an urban motorway with seasonal overloads.The section between nodes Jankomir and Lučko, and the Lučko interchange have already become part of the Zagreb urban road network on which about 70% of traffic is generated by the nearby town Zagreb [21].This section is interesting as a case model due to the combination of long-lasting increased traffic load and significant influence of daily migrations.
Constant variables of the Zagreb bypass model are related to its physical parameters.The physical model of the Zagreb bypass is created based on 15 cells (11 cells have on-ramps, and 10 cells have off-ramps).Maximal capacity of every on-ramp is 600 [vph], while maximal capacity of every mainstream cell depends on its length, number of lanes, etc. Constant variables define the fundamental diagram for every mainstream cell also.The variables of the motorway model are traffic demand (presented as a traffic flow) on every onramp, and model input and output flows.All the mentioned variables have two peak values which describe two most common daily characteristics (morning and afternoon rush hours).Afternoon peak hour is more expressed compared to the morning peak hour.
On-ramp traffic demand characteristics of the Zagreb bypass simulation model are reconstructed using daily characteristics of the Ljubljana bypass traffic.Traffic data were transformed in the form of a traffic demand dataset for each on-ramp separately.In order to adjust the daily traffic demand characteristic, the average daily traffic values from [21] are used to ensure that the daily vehicle number describes the traffic demand of the Zagreb bypass realistically.

Zagreb bypass simulation
All data from the ARMC learning dataset are derived based on simulations of all teaching algorithms on the Zagreb bypass during 30 working days.The actual learning dataset, which will be presented to the ARMC ANN is created after the application of a criteria function.Table 1 shows the key learning dataset features after the application of the criteria function.
In Figure 9 it is possible to observe comparative analysis between ARMC learned outputs (r ARMC ) and outputs based on which ARMC is learned (r ld ).Both outputs are presented in the form of metering rates computed based on the same input sets.Input set contains the learning data from five working days.Higher Root Mean Square Error (RMSE) values are reported during the learning process due to the lack of real Zagreb bypass traffic data so only a relatively small set is used.
Several simulations using the model of the Zagreb bypass were conducted in order to perform a comparative analysis between several motorway scenarios with and without any control.The scenario with no ramp metering applied was simulated first.The results of the mentioned simulation run are compared with the results provided by the ramp metring algorithms such as ALINEA, SWARM, HELPER, ARMC and the scenario which involves cooperation between ramp metering (HELPER) and VSLC.
All the mentioned ramp metering algorithms are simulated using the same simulation model and traffic data for a typical working day (24 hours).The results of comparative analysis according to the average values of TT, Delay, TTS and queue length including maximal queue length are shown in Table 2.In Figure 10 comparative analysis according to the simulation results regarding TT (a) and Delay (b) are presented.
According to Table 1 ARMC and the SWARM algorithm have produced the lowest average TT.Competitive algorithm SWARM has achieved the best average TT value due to its restrictive nature that resulted in a high average Delay value.The ARMC approach has produced the highest Delay value.This is the consequence of the generally low TT values and the highest value of the average on-ramp queue.The aforementioned ARMC results are induced due to the higher value of TT ponder in contrast to the Delay ponder value in the criteria function (6).The results verify ARMC ability M. Gregurić, E. Ivanjko, S. Mandžuka: The Use of Cooperative Approach in Ramp Metering pacity is not exceeded.The on-ramp queue starts to build up only in case when the mainstream capacity is reached.Such behaviour induces smaller on-ramp queues but significantly increases the traffic density of mainstream traffic which also produces the highest TT.
It is possible to notice that standalone VSLC has the most significant effect on TT, since it is applied to the mainstream flow only.In Table 2 it is possible to observe that the application of standalone VSLC produces lower value of TT compared to the situation without any traffic control.Lower average value of TT is the consequence of vehicle platoons which travel slower.The mentioned scenario enables more space at mainstream traffic lanes due to slower arrival of vehicle platoons to the congested area.VSLC standalone application has achieved similar Delay as the situation without any control which is expected since both cases have a higher average TT value.The results presented in Figure 10 graph a) show that the SWARM and ARMC ramp metering algorithms, and the cooperation between HELPER and VSLC produce the smallest TT values.The reason for such results is the reduction of the metering rate on the "slave" on-ramps produced by the HELPER algorithm as well as the VSLC tendency to mainstream density upstream of the congested on-ramp by lowering its speeds.According to Figure 10 graph b) it can be concluded that VSLC has lower influence on Delay if the mainstream density is decreased by the HELPER's exploitation of on-ramp queues.The cooperation between HELPER and VSLC, which involves exploitation of the on-ramp queue capacities and mainstream speed reduction, produces higher values of TTS.ARMC has achieved a TTS value which is between values obtained by the SWARM and HELPER teaching ramp metering algorithms.SWARM has the highest TTS value among all involved motorway control strategies.The TTS value produced by the HELPER algorithm is the best among all involved ramp metering algorithms.
to select solutions which can produce lower TT values but higher Delay values during the learning process.
The teaching ramp metering algorithm HELPER provides knowledge of cooperative control between on-ramps which the ARMC-based algorithm should learn.The cooperative strategy of the HELPER algorithm maintains an increased mainstream throughput by distributing vehicles and consequently the waiting time to "slave" on-ramps queues.This behaviour causes longer queues at "slave" on-ramps increasing the Delay.
Due to non-existing on-ramp queues in most cases, the no control traffic scenario provides best TTS and Delay values.In CTMSIM environment the blending coefficient was set to 1 for every cell [24].This means that all vehicles from the on-ramps are immediately merged with the mainstream under the condition that in the current cell the maximal mainstream ca-

CONCLUSION
In this research the control systems such as VSLC, prohibiting lane changes and the vehicle itself are included into the consideration for cooperation with ramp metering.Special emphasis is on the concept of cooperation between the vehicle and ramp metering infrastructure.The proposed concept is designed to provide assistance to the driver or perform a complete automatic merging of the vehicle into the mainstream flow.The mentioned type of cooperation is presented on the conceptual level.It conceptually augments the ramp metering system with the ability of direct control over the vehicles at on-ramps.
The presented results regarding TT and Delay behaviour shows that ARMC has learned ramp metering control behaviour similar to the HELPER (cooperative), SWARM (competitive) and ALINEA (local) algorithm.The obtained control knowledge has potential to reduce mainstream congestions, which vary in intensity and time.The obtained results verify the potential of this approach to provide a framework for cooperation between on-ramps.
The aforementioned framework can be potentially used for establishing cooperation between ramp metering and other motorway control strategies.The concept regarding automated vehicle guidance can be implemented in cooperation with the ramp metering by using the proposed learning framework.

Figure 1 -
Figure 1 -Illustration of potential problems near an on-ramp where r(k) is the current metering rate, r(k-1) is the metering rate from the previous iteration, K R is the regulating parameter, and O out (k-1) is the measured downstream occupancy from the previous iteration.The recommended value for K R is 70 [veh/h][10].ALINEA has numerous enhanced versions and is used as part of many other local and coordinated ramp metering approaches.The basic working principle of ALINEA is shown in Figure2.

Figure 2 -
Figure 2 -Scheme of the basic ALINEA working principle

Figure 3 -
Figure 3 -Scheme of the basic ALINEA working principle Local decision: On ramp 3 Queue len.: long Motorway sec.density: high Metering rate: very short Local decision: On ramp 1 Qeue len.: short Motorway sec.density: low Metering rate: long Local decision: On ramp 2 Qeue len.: short Motorway sec.density: low Metering rate: long

Figure 6 -
Figure 6 -Basic activity diagram of on-ramp Metering and VSLC System based on ramp metering-to-vehicle communication

Figure 9 -
Figure 9 -Comparative analysis between ARMC learned outputs and outputs based on which ARMC is learned

Figure 10 -
Figure 10 -Comparative analysis according to the simulation results a) TT and b) Delay

Table 1 -
Key ARMC learning dataset features after the application of criteria function

Table 2 -
Results of comparative analysis between different ramp metering algorithms