MaxSUMO : A NEW EXPERT APPROACH FOR EVALUATING MOBILITY MANAGEMENT PROJECTS

There is need for better understanding of how mobility management interventions work and how they affect the individuals’ modal choice decisions, as well as need for robust evaluation techniques allowing any behavioural changes to be observed. Changing individual’s behaviour is not a one-step process and any evaluation methodology should account for this. A new standardized expert evaluation resource MaxSUMO takes this step-wise process into account. MaxSUMO is based on a new theoretical behavioural change model MaxSEM which measures individuals’ stage positions (their susceptibility to change behaviour) and stage movement (progression towards actual behavioural change). This paper illustrates the use of MaxSUMO by the evaluation of the mobility campaign “I keep moving, even without my car” undertaken by the City of Ghent.


INTRODUCTION
"Things you cannot measure cannot be changed" -this is an old management adage that is still accurate today.You cannot manage for improvement unless you measure what is getting better or worse.This principle also holds for mobility management projects.Mobility Management (MM), also known as "soft policy measures", refers to a concept to promote sustainable transport and manage the demand for car use [1]."Soft" measures such as information and communication campaigns and offering tailor-made mobility services, are used to change the travellers' attitudes and behaviour.Such "soft" measures are frequently used to support and strengthen "hard" measures like the construction of new tram lines or new bike lanes [2,3].
The interest in MM projects as a solution for mobility problems and associated environmental problems has undoubtedly increased in recent years [4,5,6,7].For example, the first annual European Conference on Mobility Management was organized in 1997, but afterwards there was still a necessity to have a platform to provide some continuity.Consequently, two years later in 1999, the European Platform on Mobility Management (EPOMM) was established.It started as a European platform, but soon developed into an international association (in 2006).
If MM projects were given greater policy priority, they could be much more effective than initially assumed.Based on a review of UK and international evidence, Cairns et al. [8] developed a "low intensity" and a "high intensity" impact scenario of the future implementation of MM projects in local and national transport policies.In the low intensity scenario, they maintain the interest and attention for MM projects at the current level.Scenario results indicate a reduction in peak period urban traffic of about 5% and a nationwide reduction in all traffic by about 3%.In the high intensity scenario, they assume much more interest in MM projects and many more funding and resources.In the high-intensity scenario, MM projects have the potential to reduce urban traffic during peak hours by about 21% (off-peak 13%), non-urban traffic during peak hours by 14% (off-peak 7%) and a nationwide reduction in all traffic of about 11%.They also estimated the potential effect of various individual MM projects: workplace travel plans can reduce car use between 10 and 30%, school travel plans between 8 and 15%, and personalised travel planning initiatives between 7 and 15% in urban areas and between 2 and 6% in smaller urban areas and rural areas.These projected changes in traffic levels are thus quite large and indicate that MM projects merit serious consideration in local and national transport policies.
However, other transport researchers (e.g.[9]) warn that results of review studies such as Cairns et al. [8] might be too optimistic.This is mainly due to poor quality of the data used in the studies that are reviewed and, subsequently, used as input for scenario development.For these reasons, Möser and Bamberg [10] critically re-evaluated 141 studies on the effectiveness of three types of MM projects (workplace travel plans, school travel plans, personalised travel planning).They found a much lower potential of MM projects and pointed out that car use can be reduced by only 7%.
These inconsistencies between findings of various studies call for the development of a rigorous evaluation method.Typical methods used to evaluate MM projects generally lack empirical vigour (e.g.small sample sizes, unrepresentative samples, overreliance on self-reported behaviour, lack of corroborative data to confirm self-reported data, …) and, thus, serious questions about the reliability of these methods remain [10,11,12].There is clearly need for the development of robust expert evaluation techniques.MaxSUMO is considered a suitable technique to evaluate MM projects.This paper illustrates, therefore, the usefulness of MaxSUMO to evaluate MM projects which were recently undertaken by the city of Ghent, Belgium.
The paper is structured as follows: Section 2 presents the MaxSUMO approach, and the usefulness of it is illustrated in the third section.This third section describes first the study area of Ghent, a medium-sized city in Belgium, before discussing the results of various MM projects undertaken by the city.Finally, the results are summarized and discussed in Section 4.

HOW TO EVALUATE MM PROJECTS?
There is clearly need for the development of robust evaluation techniques [13], but in order to accomplish this we must first understand what we are evaluating.Or, in other words, a better understanding of how MM projects work and how this affects the individuals' modal choices is needed as well.Carreno et al. [12] mention two key facts.
First, some people are more susceptible or ready to change their travel behaviour than others.For example, Curtis and Headicar [14] found that only a minority of car commuters is susceptible to change.This group is more likely to be males, in their 30s and, most importantly, travel short commuting distances (5 miles or less).More recently, Anable [15] segmented a population of day trip travellers into potential "mode switchers".Six distinct groups were extracted, but susceptibility of car users to switch modes was rather limited.These varying degrees of mode switching potential partly relate to differences in subjective factors such as peoples' perceptions, attitudes, level of confidence towards their current travel choices but also towards alternative travel choices, as well as their willingness to actually alter the travel choices.For example, if people have negative attitudes towards public transport, have little or no confidence in public transport or see no reason why to change their car use, they are less susceptible to switch from car to public transport.For other people the barriers to switch modes are more objective.For example, people will not switch to public transport if no adequate bus services are offered.
Second, politicians might finally be interested only in short-term changes such as targeted reduction in car use but changing peoples' behaviour is not a onestep process.Instead, changing travel behaviour must be seen as a series of transitional stages which individuals pass through [16].For example, it takes time to change the individual's modal choices and it usually starts with altering non-behavioural aspects such as attitudes.
Consequently, any MM project is likely to affect people in different ways based on (i) people's susceptibility to change behaviour, and (ii) their stage position within the behavioural change process.Any evaluation methodology must therefore not only focus on the behavioural change as such, but also on the more subtle changes in attitudes and perceptions underlying the behavioural change process.Researchers use a variety of pre-existing theoretical frameworks such as, among others, the Theory of Planned Behaviour, the Norm-Activation Model and the Social Cognitive Theory (for a more comprehensive review, see, e.g.[17,18].However, no consensus exists on which framework is the most appropriate.Each theoretical model conceptualizes other factors of behavioural change instead of the process as a whole, and often uses different terminology to indicate very similar (or even identical) factors [19,17].Evaluating the step-wise behavioural change process requires thus specific evaluation techniques.Moreover, we notice that these expert evaluation and decision support systems are increasingly being adopted in freight transport, such as in (multimodal) transport, logistics (routing) and traffic management (see, e.g.[20,21,22]), but not that much in relation to passenger transport management.Max-SUMO is such a new standardized evaluation tool that takes this step-wise process into account.

MaxSUMO
The evaluation strategy of MaxSUMO is based on the idea to measure effects at different levels (see Figure 1).The "gap" between the MM project and the expected effects is often large.MaxSUMO divides this gap into smaller steps, or assessment levels.Targets, indicators, and results can be specified at each of these levels, so that each level can be monitored and evaluated separately.This makes it possible to measure effects at an early stage in a project.
The different MaxSUMO levels are divided into four main categories: 1. Intervention framework conditions (although not symbolized in Figure 1) refer to external factors and person-related factors.External factors include background information of the location where the MM project is offered.These external factors are similar for all users.Person-related factors include information about the personal situation of different users.These person-related factors are "objective" factors such as the distance to the nearest bus stop as well as "subjective" factors such as the individual's stage of behavioural change and the travel behaviour before the MM project was offered.2. Services provided refer to the different activities of the MM project in order to achieve changes in travel behaviour (e.g.information meetings, distributing brochures and posters).After describing the project activities and output, researchers should also pay attention to (i) the degree to which people are aware of the MM project, (ii) the usage or interest in the MM project by people who are aware of the MM project, and (iii) how satisfied the users are with the services provided.3. Mobility options offered through the services provided refer to the new travel behaviour the MM project aims to encourage.For example, by offering free season tickets for public transport (= service provided) frequent car drivers might switch to public transport for some or all of their trips (= mobility option).One should also distinguish between (i) people who intend to change travel behaviour and are willing to accept the mobility option offered, and (ii) people who eventually test the new travel behaviour and take up the mobility option offered.Afterwards, the latter people might also be asked whether they are satisfied with this mobility option.After all, being satisfied with the new travel behaviour remains a pre-condition for long-term changes in attitudes and behaviour.4. Overall effects, finally, refers to the main outcomes of the MM project in terms of (i) new attitudes and behaviour (e.g.decrease in car use), and (ii) more general system impacts due to these new attitudes and behaviour (e.g.CO2 emissions saved by this decrease in car use).
The design of MaxSUMO is thus simple and the methods included are not significantly different from other guidelines for transport and policy evaluations.However, MaxSUMO is unique in how targets, indicators and results can be specified at different  assessment levels bridging the gap between implementation of the MM project and its expected effect.
MaxSUMO thus provides step-by-step guidance so that MM projects are effectively planned, monitored and evaluated.

MaxSEM
As mentioned above, the use of MaxSUMO starts with describing the intervention framework conditions.One of such conditions refers to person-related factors and describes the individual's stage of behavioural change.These stages can be determined using Max-SEM (Max Self-regulation Model).MaxSEM not only measures the individual's stage positions (i.e., their susceptibility to change behaviour), but also the stage movement (i.e.progression towards actual behavioural change).It utilizes the most important factors of "static" psychological models of behavioural change, such as norms and goal feasibility, and links those with the temporal dimension of the process of change by incorporating four key "stages" of behavioural change [23].This helps to analyze and segment the target group and thus to choose and design the most appropriate and effective MM projects for them.
Stage 1: Pre-contemplative stage.Persons in this stage are habitual car drivers who have no intention of reducing their current car use or feel that it would be impossible to change due to objective and subjective reasons.In this stage, travel awareness campaigns are necessary to persuade this group to consider travel alternatives other than the car.
Stage 2: Contemplative stage.Persons in this stage mainly use their cars, but are not content with their current car use and would like to reduce it.However, they are unsure of how to do so or lack the confidence to change their travel behaviour.Persons in this stage thus need tailor-made travel information.
Stage 3: Preparation/action stage.Persons in this stage still use their cars, but already know how to switch to another travel mode (e.g.public transport).Moreover, they also intend to switch to this alternative, have the confidence to do so and may have already tried this new travel mode for some trips.The aim here is to have the group actually try out new behaviour (e.g. by offering free season tickets of public transport) and to facilitate the maintenance of this new behaviour (e.g. a tool which visualizes the money saved while travelling by public transport instead of using the car).
Stage 4: Maintenance stage.Persons in this stage have successfully changed their travel behaviour and have formed a new habit.MM projects in this stage should reward the new habit so that no relapse to the old behaviour occurs.
The aim of MM projects is to move the persons to the next "higher" stage and prevent relapses to a "lower" stage.Critical threshold criteria must be satisfied before any stage-progression can occur (see darker boxes in Figure 2).For example, for progression from pre-contemplative to contemplative stages the individuals must first recognize their current car use as "problematic" (Perceived negative consequences).This might eventually result in the formation of a personal goal (e.g.reducing personal car use in order to save CO2) which must be perceived as positive (Perceived goal feasibility).Once in the contemplation stage, people seek the best alternative travel mode.People must first have a positive attitude towards this alternative (Attitude towards different behavioural change strategies) and/or need to feel confident that they could use this alternative by themselves (Perceived behavioural control).Once this is fulfilled, the previously formulated goal is translated into a more precise behavioural intention (e.g.intention to use the bus instead of the car for some trips next weekend).Now, people need to plan more specifically in the preparation/action stage: when, where and how to use the new travel alternative.It is important to use the cognitive planning abilities to retrieve relevant information (e.g.interpreting the timetable of the local bus) and to be able to cope with the implementation problems (e.g. using the upto-date web-service instead of an outdated timetable).If people make definite plans to test the travel alternative, the behavioural intention is translated in an implementation intention (e.g.going to the city centre for shopping by bus at 10 a.m. next Saturday).Finally, in the maintenance stage, before a new habit is formed, people need to repeat the newly tested travel alternative (e.g.going by bus for other work and leisure trips and at other times as well).Therefore, they must use their skills to resist temptation (e.g.fall back into old behaviour and use their cars).If they do not resist, they have to recover from relapse and take up the new behaviour again.
MaxSEM provides six so called "stage-diagnostic questions" (see Table 1) which objectively measure people's stage position and readiness to change.This set of question results from a series of validation studies within the MAX project [23].With the help of these questions, it becomes clear whether MM projects directly result in changing the actual behaviour or rather whether people move to the next stage and move closer to the behavioural change.MaxSEM is thus on the one hand a theoretical model explaining the process of behavioural change (see Figure 2), and on the other hand a practical tool to determine the different stages of behavioural change (see Table 1).
By asking the stage-diagnostic questions, people are grouped into different stages.This facilitates the design of appropriate MM projects according to which stage the individuals within the target population are currently in.For example, an appropriate travel awareness campaign might persuade pre-contemplators considering alternatives for their current car use.By asking the same questions after the travel awareness campaign as well, the effect of this campaign can be evaluated and it illustrates whether people progressed to later stages of readiness to change the behaviour [24].

MAXSUMO IN PRACTICE
This paper illustrates the use of MaxSUMO based on the results of an MM project recently undertaken by the City of Ghent, Belgium.[23] Question: Which of the following statements best describes how you feel about your current level of car use for daily trips (in city X / to your workplace1) and whether you have any plans to try to reduce some or all of these car trips?

Study area
Please choose which statement fits best your current situation and tick only one box.

Stage allocation
At the moment I use the car for most of my trips.I am happy with my current level of car use and see no reason why I should reduce it.

□
Pre-contemplation At the moment I do use the car for most of my trips.I would like to reduce my current level of car use, but feel at the moment it would be impossible for me to do so.□ At the moment I do use the car for most of my trips.I am currently thinking about changing some or all of these trips to non-car modes, but at the moment I am not sure how I can replace these car trips, or when I should do so.

□ Contemplation
At the moment I do use the car for most of my trips, but it is my aim to reduce my current level of car use.I already know which trips I will replace and which alternative transport mode I will use, but as yet have not actually put this into practice.

□ Preparation / Action
As I do not own / have access to a car, reducing my level of car use is not currently an issue for me.

□ Maintenance
As I am aware of the many problems associated with car use, I already try to use non-car modes as much as possible.I will maintain or even reduce my already low level of car use in the next months.□ consortium of cities. Ghent joined a consortium called CIVITAS ELAN which consists of the cities of Zagreb (Croatia), Brno (Czech Republic), Ljubljana (Slovenia) and Porto (Portugal).Ghent has already some experience within the field of sustainable mobility projects and measures.It is thus a "leading city" and an example for Porto and Brno which are the "learning cities".These five cities will exchange knowledge and experience (http://www.civitas-gent.be).
The City of Ghent implements 24 sustainable mobility measures which are grouped into five integrated packages.One of these packages specifically focuses on MM as a tool for changing mobility behaviour.This integrated package contains all types of "soft" measures that will be implemented to improve (i) citizens' awareness of different sustainable transport modes and (ii) citizens' commitment to change their non-sustainable urban mobility behaviour.The measures consist of new communication strategies (e.g.3D-model) and new concepts (e.g.school travel plans for secondary schools).Within this paper one specific campaign "I keep moving, even without my car" has been evaluated using MaxSUMO.

What preceded the campaign
The integrated package focusing on MM as a tool for changing mobility behaviour consists of six measures.One measure provides tailor-made information for citizens about public transport and bicycle or walking routes in their neighbourhood.By doing so, this measure aims at raising the citizens' awareness of the options for sustainable mobility so that also a modal shift can be realized from car towards more sustainable transport modes.
Information on sustainable transport modes was distributed among citizens in the first place by a tailormade brochure (mobility campaign entitled "Our district is moving" or "Onze wijk beweegt" in Dutch).The city of Ghent is divided in 20 residential neighbourhoods, each with very specific transport features.The neighbourhood-specific characteristics are therefore included in each brochure.
Citizens who received this brochure were afterwards invited to join the "mobiteams", groups of citizens per neighbourhood that would exchange ideas, information and experiences related to sustainable mobility with each other.However, it seemed that cardependent people were not interested in being part of such "mobiteams" since mainly people who already use sustainable transport modes responded to the invitation.This suggests that (car-dependent) people do not spontaneously seek information on sustainable mobility, and thus other initiatives had to be undertaken to inform people about sustainable travel options.
One possibility was to contact people through the system of "play streets".Play streets are closed for motorized traffic during specific hours or days during holidays so that children can play freely on the street, and are organized by the city at the request of the citizens.One might expect that the willingness to participate in a project about sustainable mobility is greater in these streets.Consequently, residents of these play streets were invited to participate in a competition between play streets to find the street with the highest modal shift toward more sustainable transport modes (mobility campaign entitled "Our street is moving" or "Onze straat beweegt" in Dutch).Residents were asked to use public transport, bike or walk for trips which are normally travelled by car.They could  register their sustainable trips and travel distances in a specifically developed website which also calculated the amount of CO2 saved, calories burned and money saved.This illustrates the environmental, health and monetary benefits of sustainable transport.Despite all efforts, only a few households (literally) wanted to participate.To gain insights in this total lack of interest, residents of play streets were personally interviewed two months after the start of the campaign.Only one third recalled having received the invitation to participate.Two thirds of them have actually read this invitation letter, but did not reply mainly due to lack of time.However, many residents became interested in the campaign after the interview.Thus, a very personal approach seems necessary, especially in campaigns aiming at changing attitudes and behaviour.From this notion, a third campaign entitled "I keep moving even without my car" (or "Ik beweeg ook zonder auto") was developed.

The campaign "I keep moving even without my car"
The campaign "I keep moving even without my car" aims at changing the travel behaviour of frequent car users by providing personal guidance and advice on sustainable travel options.
The city planned interviews with at least 300 citizens who frequently use their cars but are willing to switch to public transport, cycling or walking for some of their trips.From this group of 300 citizens at least 10 citizens were willing to participate in the campaign.This means that these 10 citizens are very aware of their travel behaviour during one month and use sustainable alternatives for each trip whenever possible.The city thus sets targets at different assessment levels according to the MaxSUMO approach (see Figure 3).

Results
In April 2011 two pollsters interviewed 454 citizens at various public places such as the shopping mall, library and sports centers.Forty-four citizens did not own a car and used public transport or walked and cycled frequently.These respondents were already within the final maintenance stage of the behavioural change process and, thus, did not belong to the target group of this MM project (i.e.frequent car users).The other 410 citizens all owned a car and might be interested in participating in the campaign.In order to determine their stage position, five stage-diagnostic questions were asked similar to the MaxSEM questions mentioned earlier (see Figure 4).
Almost one in ten car owners stated that they frequently used their cars and saw no reason why they should change this (9.0% in pre-contemplation stage).On the other end of the spectrum, one third frequently used sustainable transport modes (33.4% in the maintenance stage).These two groups clearly did not belong to the target group of this MM project.Consequently, more than half of all car owners can be described as frequent car users who might be willing to switch to sustainable transport modes but have not done this so far for various reasons: -7.1% want to use public transport and bike more frequently, but are not sure how they can replace their car trips by these sustainable travel modes (contemplation stage); -15.6% already know how to switch from car to public transport and bike, but have not put this into practice (preparation stage); -34.9% already use public transport and bike, but want to use these sustainable travel modes more frequently (action stage).These three groups of respondents (236 respondents) might be interested in personal guidance and advice on the use of sustainable travel options.Consequently, these respondents were questioned further about their susceptibility to change travel behaviour.After explaining the content of the campaign "I keep moving even without my car", they were asked how they evaluate this campaign.The majority (71.2%) considers this campaign as a great initiative.One quarter (25.0%) does not have a strong opinion about the campaign while only a minority (3.8%) thinks it is a completely useless initiative.
Despite a generally positive evaluation of this campaign, the willingness to participate is significantly lower.Only a dozen respondents (7.0%) were willing to participate in this campaign, but ultimately only 6 citizens actually participated.During the month of June 2011, these 6 participants were asked to consider sustainable transport alternatives for each car trip that they used to make.They were given personal assistance and detailed information (e.g.city maps, brochures, and websites on sustainable mobility).The consultancy bureau Traject was 24/7 standby to give necessary transport information (e.g. which bus or bike route to take to a specific destination).If needed, free bicycles and season tickets were also offered to the participants.During this test month, participants were contacted several times in order to enquire whether additional help or information was needed.
The 6 participants were asked to switch as many car trips as possible and to register their sustainable trips in a specifically developed website which also calculated the amount of CO2 and money saved.Table 2 illustrates that, during just one month, these 6 participants travelled more than 2,000 km using sustainable travel modes instead of using their cars.This equals almost 340 kg less CO2 and 600 euro less spent on travel.In November 2011 a follow-up was planned.The 6 participants are to be surveyed over the telephone, assessing their mobility behaviour after the campaign.This is to clarify whether the 6 participants formed new travel habits and really progressed to the final maintenance stage of the behavioural change process.

Summary of the evaluation method
Figure 5 summarizes the evaluation of the campaign "I keep moving even without my car", recently organized by the city of Ghent, Belgium.This campaign aimed at changing the travel behaviour of frequent car users who were willing to change but do not know how to or have not changed their car use so far.Participants received personal guidance and tailor-made advice on sustainable travel options so that they can switch car trips to more sustainable trips by public transport,

Figure 5 -Summary -results at different assessment levels according to MaxSUMO
cycling or walking as much as possible.The city targeted that at least 300 citizens are aware of the campaign, at least 10 citizens are willing to participate in the campaign which results in more sustainable trips and less CO2 and money spend on travel.Eventually, 454 citizens were interviewed on street but only 236 respondents belonged to the target group of frequent car users willing to change their travel behaviour.The campaign was presented to these respondents only.Consequently, the initial target of 300 citizens being aware of the campaign has not been fully achieved.Furthermore, 71.2% of the respondents considered the campaign as a great initiative.Although no initial target was defined about the interest or usage of the mobility services provided, the interviews revealed great interest in the campaign.However, 7% of the target group, or 16 respondents, were effectively willing to participate in the campaign.The willingness to participate was thus higher than targeted, but eventually only 6 respondents participated in the campaign.Thus, a large gap seems to exist between being interested in the campaign, the willingness to participate and actually participating in a campaign.Although only 6 persons participated in the campaign and received personal advice on how to switch their car trips to more sustainable trips, the results are quite positive.During only one month, these 6 persons travelled more than 2,000 sustainable kilometres and saved more than 300 kg CO2 and 600 euro.

CONCLUSION
This paper reported on the usefulness of MaxSUMO as a new methodology to effectively plan, monitor and evaluate MM projects.It breaks down the complex process of behavioural change into smaller steps which facilitates monitoring and evaluation.These steps are presented in MaxSUMO as different assessment levels.For each assessment level targets and indicators must be defined, but some levels can be skipped since in some MM projects it is neither possible nor necessary to monitor all levels.
This paper also reported many efforts that were needed to design a successful campaign.The campaigns prior to the "I keep moving even without my car" campaign illustrate that contacting the target group is not always that obvious.However, this was facilitated by using the stage-diagnostic questions of MaxSEM at the beginning of the on-street interviews.These stagediagnostic questions easily clarified that almost one in ten respondents will not change their car use, one in three respondents have already changed their car use to more sustainable travel options and half of all respondents belong to the target group of frequent car users willing to change their travel behaviour.This narrowed down the initial sample of 454 citizens in-terviewed on street to a specific target group of 236 respondents which facilitated further steps within the MM project.
Although a tendency exists to report only good practice case studies of MM projects [10], using the step-wise approach of MaxSUMO offers better insights in the positive but also the negative aspects of an MM project.For example, the final results in terms of more sustainable kms and the amount of CO2 and money saved are clearly described.However, the drop-out from great interest in the campaign to a limited willingness to participate and even more limited actual participation in the campaign is significant.This step-wise approach thus offers valuable insights for anyone organizing an MM project as it clearly illustrates in which specific steps the MM project was successful (or not).

Figure 3 -
Figure 3 -Defining targets and indicators at different assessment levels.

Figure 4 -
Figure 4 -Distribution of stage position according to the campaign "I keep moving even without my car"

Table 2 -
Results of the campaign "I keep moving even without my car"