QUALITY ASSESSMENT METHOD FOR MOBILITY AS A SERVICE

The ongoing development of the concept ‘Mobility as a Service (MaaS)’ along with Shared Mobility contributes to the integration of transportation systems. Several MaaS or similar services are already in operation. The perceived quality of MaaS by the users varies significantly, and no general method is proposed to evaluate the service quality. This scantiness is identified as the research gap. The objective of the research is to elaborate a quantitative method to assess MaaS services. The research question is how to assess the quality of MaaS, and how to transform the qualitative description into quantitative numerical values, namely, the quality index and the level of quality. Since user expectations towards the importance of criteria are taken into consideration, the modified triangular fuzzy analytic hierarchy process method is introduced to calculate the weights of criteria. A quantitative method to calculate the quality index and to assign the quality level has been elaborated. Ten MaaS services are assessed with the method. It was found that the journey comfort is regarded with significant importance among the respondents. Furthermore, the quality index of MaaS services is not high; accordingly, the service quality requires continuous improvement. Our method facilitates decision-making when planning MaaS to identify the expected service attributes.


INTRODUCTION
The integration in transportation system has been a widely studied topic, which has been analysed mainly from the economic, social, and technological aspects. The development of infocom-munication technology, especially smartphone and applications, have been embedded into new mobility solutions, such as car-sharing, ride-sharing, bike-sharing, ride-sourcing services, which belong to shared mobility forms. These private, semi-public, or public services are required to be integrated with conventional public transportation systems.
In Mobility as a Service (MaaS), a comprehensive mobility and related services are provided [1]. The MaaS operator is regarded as a broker between the users and the transport service providers. Bundled services from public and private transport providers are accessed with a single user account. As an alternative of private car ownership, the easy-touse, seamless and door-to-door mobility services bundled into packages are tailored to cater for the users' travel needs [1,2].
'User' and 'traveller' terms are used as synonyms in this paper. The perceived quality of MaaS by the users varies a lot and no general method has been developed to compare the quality. The perceived quality is derived from the evaluation of end-users. The terms 'quality' or 'service quality' are used as synonyms in this context. The quality of such innovative mobility service is a complex term depending on its subjective and objective attributes. The subjective attributes express the user experience-related service characteristics (e.g. comfort). The objective attributes are specific, measurable service characteristics (e.g. application functions).
The research question is how to assess and quantify the quality of MaaS; namely, how to calculate the quality index and the level of quality. The quality index is the aggregated value of quantified attributes. The level of quality is a pre-defined MaaS is a user-centric, shared mobility-oriented, real-time information-based mobility service. The objective is that all modes of travel are integrated into one unified system which is available via a digital platform (e.g. smartphone application) [3]. Seamless journey is experienced before (journey planning and booking), during (seamless transfers both in time and space) and after (e.g. payment and feedback). It was explored how public transport service will be affected by MaaS, since private operators would play a significant role to create public value, to facilitate 'the mobility for all'. Market-driven, public-controlled and public-private partnerships as predictive scenarios were identified for future public transportation [4].
The typical operation fields of MaaS are interand intra-city travels. However, the aim is to reach the international and global MaaS operation [5]. Sweden and Finland are pioneer countries of MaaS implementation [1]. The UbiGo of Sweden was the first MaaS trial launched in 2014. The participants had reported that they decreased the usage of private cars and increased alternative modes such as car-sharing usage [6]. The Whim pilot launched in 2016 by the MaaS Global has been operated in Helsinki, the West Midlands, Amsterdam, and Antwerp (https://whimapp.com/) . Along with Europe, MaaS has been also launched in North America and Australia. For example, the TransitApp of America involves the Uber service (https://transitapp.com/) . A stated choice face-toface survey was conducted in Sydney with 252 respondents, a preference model was estimated, to reveal how large the potential market of MaaS and how many potential users might value a MaaS plan. It was found that the infrequent car users would be the most likely adopters [3].
The quality assessment of public transportation services was studied more. The bus service quality was measured with AHP and Technique for Order Preference by Similarity to Ideal Solution (TOP-SIS), the AHP-TOPSIS method. AHP was applied to prioritize the quality attributes and TOPSIS was used to rank the bus transit routes. It was found that the decision-making was supported by the calculated and ranked quality score of each bus transit route [7]. The Artificial Neural Networks (ANN) method was proposed to analyse the perceived quality of the public transport service. It was found that frequency was the most influential attribute. In addition, speed, information and proximity were also important [8].
hierarchy which contains the same attributes in order to assign the appropriate level to the calculated quality index. Several MaaS services have been selected as 'MaaS representatives' to be assessed in this work. Compared to ranking and benchmarking the MaaS services, grouping them by the introduced levels is more adaptable in practice to support the decision-making (e.g. system planning and development, quality improvement), since the preferred service attributes could be collected from the same level of MaaS services.
The main research question is unfolded into three sub-questions: -What are the assessment criteria and grading rules? -How to assign weights of criteria and grades to MaaS services? -How are quality index and level of quality to be calculated? Our method is elaborated to assess both subjective and objective attributes of MaaS. The major parts of the proposed quality criteria and grading rules are objective, which are applied to assess the infocommunication background of MaaS. Since the feedbacks from end-users regarding the subjective attributes are not available, only the objective attributes are considered during the application demonstration. The attributes are considered as evaluation criteria. The corresponding websites and applications of MaaS services are assessed.
The remainder of the paper is structured as follows. State of the art is summarized in Section 2. In Section 3, the service quality assessment method is elaborated; namely, the criteria and grading rules are introduced, the calculation steps of weights are presented, the equation to calculate the quality index, as well as the equation to identify the level of quality, are developed. As a case study to apply the method, the assessment of ten MaaS services are presented in Section 4, in which the results of weight calculation are presented in detail. The paper is completed by Section 5 as a conclusion, including further research directions.

STATE OF THE ART
Literature has been reviewed, with the topics of MaaS, quality assessment of mobility services, methodology of combination of Analytic Hierarchy Process (AHP) with fuzzy theory. various and exact evaluation could not reflect the results. Therefore, the fuzzy AHP is introduced to reconcile and obtain rather objective weights from the questionnaire results.
For example, a rule-based decision-support mechanism was introduced to emerge these two methods to evaluate the influential factors in the Internet of Things [16]. A triangular fuzzy scale was proposed based on Saaty's approach to analyse the goal of companies. The intervals were applied to replace the exact values in pairwise comparison [17]. A fuzzy multi-criteria decision-making model was proposed to select the best alternative. The triangular fuzzy interval scale was also applied [18]. In addition, the fuzzy AHP is proposed as a better approach to deal with uncertainty, when the fuzziness of the decision maker or respondents is considered [19,20].
The reviewed literature is summarized regarding the applied methodology in Table 1.
It was concluded from the literature that many researchers studied the service quality of the public transport systems; however, MaaS is between public and private mobility service during its transitional period. Thus, new criteria should be considered, to involve private service attributes to assess the quality of MaaS. Few authors [11,12] have already studied the integration level of MaaS by qualitative approaches; however, the research about quality of this service is still missing. This research gap was identified and attempted to be filled in our work. In addition, the importance of criteria is determined by survey-based, calculated weights.
A fuzzy integral-based information fusion model was proposed to analyse the weight gap of criteria, in order to evaluate and improve the service quality of the transport systems [9].
A school transportation satisfaction index was estimated by applying linear regression analysis; the level of service was analysed but it was found that passengers evaluated the school bus service with low satisfaction level [10]. The integration of MaaS was reviewed from Ticket and Payment, Mobility package and Infocommunication technology aspects. An index was introduced to evaluate the level of integration based on the assumption that higher level of integration was more preferred by the users [11]. The level of MaaS integration was identified by applying a topological approach. Levels 0-4 were introduced to identify the phases of integration, how the service attributes were involved [12].
The AHP method developed by Saaty is to facilitate the decision-making process; the overall objective is separated into sub-objectives and solved in decreasing the hierarchy order [13]. AHP was applied in a vehicle routing problem to select the supplier [14]. Zadeh proposed the fuzzy set theory to model the uncertainty of the real-world problem [15]. The intersection of these two methods was applied as a new approach, such as fuzzy AHP method, which was already applied in other subject studies. Namely, AHP is widely applied as a semi-subjective method to calculate weights of factors; the fuzzy theory is introduced to handle the existing uncertainty. Considering the typical questions listed in questionnaires, the answers of respondents are The introduced quality criteria and grading rules are used in the questionnaire survey, to collect input data of weights calculation, as well as to obtain the grades of MaaS services. Steps 1 and 2 are established as preparation work and inputs of step 3, since the calculation of the quality index and assignment of the quality level are considered as our main task.

Quality criteria and grading rules
The quality criteria and grading rules are established first. The major part of criteria is introduced to assess the infocommunication background of MaaS, which is important to ensure high quality, because: several transitional services are only available to be booked via applications, such as ride-sourcing and car-sharing; the integration of transport modes and functions is perceived by users via applications; the seamless transfer is significantly supported by accessibility of real-time information.
The groups and hierarchy of criteria have been established and presented in Figure 2. We have introduced the criteria structure considering both sides: service provision from the operators and service perception from the users, namely, what kind of service is provided and perceived. Since MaaS is expected to be an integrated service, which is information-based, spatial seamless transfer-ensured, as well as comfort-providing; the criteria groups are determined.

METHODOLOGY
In an assessment method, typically, weights and scores (grades) of criteria are multiplied, and the objective is to rank the alternatives [25]. Both weights and grades can be subjective from the user evaluation. By contrast, we elaborated an objective quality assessment method. The weights are calculated with the modified triangular fuzzy AHP (TFAHP) method. AHP is selected because it provides semi-objective results. Taking the characteristics of questionnaire data into consideration, the modified fuzzy AHP method has been proposed. The grading rules are introduced to obtain objective grades.
The user of our method is a decision maker, e.g. MaaS developer or operator. Our aim is to group and analyse MaaS services being in similar development phase. In order to identify the common attributes, the potential improvement aspects, as well as the development consequences of the MaaS services within the same level, and to better support the decision making. The steps are introduced in The following steps are introduced: 1) quality criteria and grading rules are determined according to the literature review and the existing MaaS solutions (applications, websites, etc.); both public and private transportation characteristics are taken into consideration; 2) the modified triangular fuzzy AHP method regarding weights calculation is introduced, which is based on data from the questionnaire survey; 3) the equation to obtain the quality index is established, and also the equation to identify the level of quality is developed.  are introduced in public transportation [23]. However, these two terms with MaaS are emphasized on transfers between public and private modes. For example, the transfer may be between conventional stop/station and free-floating cars. The grading proposal of objective criteria is presented in Table 2. Criteria factors are introduced as the detailed grading rules of the criteria.
Values of 0 or 1 are assigned to criteria factors. The grades represent whether the target object (MaaS service) fulfils the criteria factor or not, e.g. a check list. The grades of criteria are calculated as the sum of values assigned to the criteria factors.
The C 4 Comfort group is used in quality assessment of public transportation service, which contains subjective criteria [23]. As shared mobility (e.g. car-or ride-sharing) is provided with smallsized vehicles, the service users may care more about the interior of vehicles C 41 [27]. The comfort at stop is considered through C 42 . User (customer) care as C 43 includes e.g. management of complaints or a guide for the disabled to car-sharing or bike-sharing facilities.
The grading proposal of subjective criteria is presented in Table 3. As 1 or 0 (equal to 'yes' or 'no') are not appropriate to describe the subjective attributes, a middle evaluation value (0.5) is also introduced. Accordingly, three values are assigned to subjective criteria. This proposal can be used in the user satisfaction evaluation as well (e.g. revealed preferences).
The C 1 Integration group is established according to specialties of MaaS. Multimodal journey-related passenger handling functions are considered through C 11 [12]. Typically, the application functions such as journey planning, booking, ticketing, reminder are taken into account. Transport modes integration and tariff are considered by C 12 and C 13 [11,22]. Integration with other services (e.g. relevant information provision, recommendation of food/drinks/Point of Interests) is to be assessed by C 14 .
The C 2 Information group, criteria C 21 and C 22 are applied by quality assessment of the public transportation service [23]. Real-time and estimated information are significantly important to the users, especially regarding multimodal journey, such as network condition, time of vehicle arrival. Since data are to be provided by the users as well (crowdsourcing), the C 23 is included, to collect e.g. real-time congestion information of the user position. In addition, the user's real-time feedback is applied as the recommendation information. C 24 is introduced for extra information, such as report of CO 2 emission and energy consumption, which facilitate the sustainable mobility [24].
The Connectivity (Transfer) is identified in conventional public transportation to enhance the connection between different public modes; for example, the smart stop concept is proposed and implemented in Singapore and in the Netherlands [25,26]. The C 31 accessibility and C 32    The triangular fuzzy number is used in TFAHP as an interval value to replace the single judgement value in AHP; the span of data is better processed with TFAHP [19]. The length of the interval reflects the coverage of the data, which is to indicate the fluctuation of importance towards the compared criteria. In theory, a triangular fuzzy number F=(l,m,u), where 0≤l≤m≤u≤1, l and u are introduced as lower and upper limit of m, the |l-u| is the size of F [29,30]. A triangular fuzzy number represents a numerical interval. Since the pairwise comparison and interval value are applied, the fuzzy comparison matrix M i is presented as:

Weights and grades
The importance of the criteria varies; therefore, weights of criteria have to be identified. AHP is a widely applied method to calculate the weights based on the questionnaire data. Thus, the user expectations are considered [19,28]. Because of the fuzziness of the surveyed data, a modified TFAHP method is introduced to calculate the weights of the criteria.
The grades are calculated according to the grading rules when assessing the selected MaaS services in application demonstration. Grades are aggregated according to the specific MaaS service.
The 1~5 scale representing weight importance is proposed to be applied in the survey replacing Saaty's 1~9 scale approach ( Table 4). Considering that the number of criteria is small in each group, the span of 1~5 scale is easier to be applied by the respondents.
The novelty of the method lies in the introduced triangular interval number in the data process. The characteristics of the collected data have been consid-  Local weight W ij of C ij with respect to the corresponding criteria group C i . In normalization step, W ij is calculated according to Equation 5. 0<W ij <1. k represents the row number in the specific matrix; i and j represent index corner mark of criteria C ij . 0<k≤n of M i . The corner mark i of bracket in Equation 5 is highlighted; it indicates that W ij is the local weight.
The global weight w ij of C ij with respect to the service quality Q; w ij is calculated according to The upper boundary of i and j depend on the number of C ij in the criteria table. In this work, 1≤i≤4, 1≤j≤4.

Quality index and level of quality
Quality index Q is introduced to transform the qualitative description of quality to quantitative numerical value for comparison purpose. Equation 7 to calculate Q is introduced first. where g ij is the grades of C ij , according to the grading proposal C ij.k ; g ij.k is assigned to the assessed MaaS service; n ij is the number of C ij ; n ij.k is the number of C ij.k in C ij .
The upper boundary of i and j depend on the number of C ij in the criteria table as well. In this section i≤3, j≤4, as C 4 group is eliminated. We propose the level of quality L (Figure 3) to group alternatives (MaaS services): The type, length, and colour of arrow present the criteria factors C ij.k involved in each level. The same level is assigned to similar MaaS services. The level is identified according to the emerging phase of infocommunication background and the implementation of the existing MaaS. For example, regrading C 12 , public transportation is integrated first typically, then taxi, then car-and bike-sharing, then other transitional service, and train/flight service. Levels , , ,   The modified data processing is described as follows. For one specific pairwise comparison x kt , l kt refers to the average value of the collected data which are less than or equal to 1; u kt is the average value which is more than 1; m kt is the average value of all values. The F=(l, m, u) is regarded as an interval number. For example, the triangular interval value x 12 =(l 12 , m 12 , u 12 ) in matrix M 1 is a result of pairwise comparison of C 11 and C 12 , which is determined from: l 12 =average (value ≤1 in collected x 12 ), u 12 =average (value ≥1 in collected x 12 ), m 12 = verage value of collected x 12 .
First, the comprehensive importance interval value of each criterion C ij compared with all other criteria within the same criteria group C i is calculated. The aggregated value is calculated as [29] Equations 1 and 2. where n is the rank of specific matrix M i . The importance interval value S i is calculated as [29] Equation 3; S i is a column vector.
Then the interval value is transformed into single average value s i with Equation 4; s i is the column vector.
Level 2: one or two private modes are included; application support travel; detailed information provision; Level 3: more private modes are included; application is regarded important in travel; E-payment is introduced; Level 4: road transport modes are integrated; application is essential support to travel; comprehensive, dynamic information provision; Level 5: modes are integrated; using application is regarded as a component of travel; comprehensive mobility information service (digital mobility). Based on criteria weights and the grading proposal, the equation to identify the endpoint value of L is presented as Equation 8. Each level is introduced as an interval with two endpoints. 1~5 are introduced according to literature review [1,11,12,24], websites and reports of MaaS schemes, e.g. MaaSiFiE [31], maas4EU [32].
Generally, the upper level fulfils the functionalities of lower level. For example, regarding C 11 Functions, C 11.1 , C 11.2 and C 11.3 are included in Levels 1, 2 and 3; ticket validation and application reminders are included in Level 4; the artificial intelligence-based function (e.g. C 11.7 and C 23.2 ) is included in Level 5. If a MaaS service is assigned Level 4, regarding C 11 , journey planning, booking and ticketing functions are available.
According to the increasing integration of modes, personalization of functions, flexibility of information provision from lower to upper level, the levels are established as follows: Level 1: integration of public transportation; application is available to plan travel; base information provision; C 11.1 C 11.2 C 11.3 C 11.4 C 11.5 C 11.6 C 11.7  The application of (a) case is detailed in the remaining part of the paper.

Level of quality
The input data were collected among 74 students and 12 experts at the Faculty of Transportation Engineering and Vehicle Engineering, Budapest University of Technology and Economics, Hungary. MaaS has not been launched in Budapest yet; however, the public transportation is highly integrated and operated by one unified service operator. Accordingly, the users have similar service experience. Eighty-six survey results have been received and the pairwise comparison of numerical values have been processed to obtain weights. For comparison purpose, the weights of criteria have been derived from three groups: student group (SG), expert group (EG), and together (TG).

Weights from questionnaire survey
Weights have been calculated according to Equations 1-6. Weights from SG and EG vary considerably. Weights of TG are not simply calculated as mean values of SG and EG; the data of SG and EG are used as input data in the calculation of weights of TG. The numerical values of weights are listed in Table 5.
The weights show the relative importance of criteria. It was found that Information is regarded as the most important criteria group among potential users, followed by Comfort group (aggregation of w ij ). C 14 , C 23 , C 24 are the obtained higher values of weights from SG. Since travellers can share their mobility data, value-added service (C 14 ) and personal data-sharing (C 23 ) are preferred with respondents where: l=0,1,2,3,4, ij are determined according to C ij.k included in each level, a ij.k is the number of C ij.k assigned in the same level, n ij =10 in this context. X l-1 is the endpoint value of the previous level. The upper boundary of i and j are i≤3, j≤4 in this section.
For example, endpoint value X l is determined as follows. The MaaS services are to be assigned to levels according to interval endpoints [X l-1 , X l ], X l-1 ≤Q≤X l . Since weights are regarded as variables in our method, the ranking of alternatives may change along with the change of weights. The identified equation of endpoints contains weights. The weights may be different, the assigned level remains the same.

RESULTS AND DISCUSSION
The demonstration contains two parts: weights calculation and assignment level L to the MaaS services according to index Q. The assessment method is to be applied in two ways: a) accept weights we calculated from our surveyed data; b) perform survey to collect data and calculate new weights.   Qixxit is introduced as the collector of website links regarding mobility services in Germany. Smile project is highly considered as the first MaaS in Austria. Since no smart device applications are developed from Ubigo project, it is not involved in this assessment; Ubigo and Whim are MaaS implementations with monthly subscription [6]. Transitapp is widely available in the USA, and in other countries as well, e.g. Australia, Canada, France.
Smartphone applications of 9 MaaS services are downloaded and tested by the authors in order to assess the objective attributes (application of Smile already stopped). The supplementary information on websites are also reviewed to complete the assessment. Based on the calculated weights and the grading proposal, the endpoints of the level of quality are illustrated in Figure 5:

Figure 5 -Calculated interval of level of quality
The assessment results are summarized in Table 7, by applying Equations 7 and 8, the value of quality index and level of quality are presented as well.
Only two models are assigned 'Level 3'. According to website information of Smile, this model is assigned the highest index value. The Whim application from MaaS Global is highly valued among the existing MaaS services, too. Compared with previous studies [11,12], the quality index with grades and weights has been calculated. It is not simply assumed that the importance of criteria is equal. On the contrary, the importance of criteria w ij is calculated from the collected data. from SG. In addition, feedback opportunity (C 24 ) is preferred in the service-oriented mobility system, too.
The last question 'what do you think about the importance of the criteria group? Please score the pairwise comparison values in the corresponding table' was surveyed. Hence, the weights of criteria group C i have been assigned two sets. The numerical values are listed in Table 6.
The comparison regarding two sets of weights of C i is presented in Figure 4.
One set contains values calculated from the questionnaire, the other contains the sum of corresponding w ij . The differences resulted mainly because of the number of calculations. The respondents scored higher values to C 4j ; namely, journey comfort is regarded significantly important (bars in FQ of C 4 are the highest). In a service-oriented mobility system, it is not enough to simply transport passengers from A to B, they require high-quality service. From the summation values, the criteria C 2j of C 2 Information are assigned higher values (bars in SUM of C 2 are the highest). It is concluded that the respondents consider journey information more.

Assessment of MaaS services
Because w 4j are eliminated, the global weights of the remaining criteria are re-calculated and re-assigned. Global weights of C 1j , C 2j , C 3j from TG are applied. Ten MaaS services are selected and assessed, which have been operated in a relatively wide area and over a long period: (I) Qixxit, Germany, https://www.qixxit.com/, (  MaaS operator is the key player in the operation. Public authority or private company is to be the third party MaaS operator, which performs better in practice; such studies are expected. It was found that private companies obtain higher quality index than public authorities in the transitional period of MaaS.

CONCLUSION
Our main contribution and work has been the elaborated quality assessment method for MaaS based on the surveyed data, which contains the weighted criteria, the proposed grading rules, as well as the identified level of quality. The novelty of the method is to process the questionnaire data in the form of triangular interval value, which is better for presenting the span of data, rather than using the mean value. Namely, we have considered the range of data and uncertainty of respondents' The comparison of Q values is presented in Figure 6, which is grouped by the assigned levels. Level 1 is presented in light grey, Level 2 is in medium grey, level 3 is in dark grey. The MaaS operator of (1) and (6) is public authority, presented with arrow. Private companies as the third party MaaS operators are more typical, which perform better according to the assessment results.
Because of the uncertainty of the questionnaire survey answers, the exact values of pairwise comparison could be replaced by fuzzy interval values. In our work, the triangular fuzzy number is selected to construct the comparison matrix. The novelty is that instead of calculating the average values of the collected data, the intervals are proposed to present the span of data. Therefore, our approach is based on the continuous data sets rather than the discrete data values. All the collected data are used during the calculation of the weights.
The assessment results meet the reality, e.g. higher index is available with more functions; namely, our proposed method works properly.   answers. In addition, semi-public MaaS is typical in the transitional period, wherein both public and private service attributes are involved. Our method is a combination of qualitative and quantitative approach compared with previous studies.
It was found that: the journey comfort is regarded as the most important criteria group among respondents; journey information provision is highlighted by respondents as well; typically, a private company acts as a third party MaaS operator in transitional period of MaaS; the service quality of the existing MaaS services is not high enough; continuous improvements are needed. The lesson that has been learnt is that performing a pure number-oriented questionnaire survey is complicated and time consuming. Respondents require time to understand the survey method and fulfil the questionnaires. It was concluded that the respondents with background knowledge are more preferred in such data collection survey.
Faced with the future research plan, we are going to develop methods to assess other aspects of new mobility services, e.g. calculation of indices for flexibility, personalization, and integration, in order to analyse MaaS in a comprehensive way. The index calculation method regarding integration is to be developed first. The assessment aspects as variables are determined and the development tendency is envisaged. The integration phases are to be introduced considering the weighted variables. This rather quantitative assessment may present a comprehensive overview of MaaS for the decision makers.

ACKNOWLEDGEMENT
This work was supported by EFOP-3.6.3-VE-KOP-16-2017-00001: Talent management in autonomous vehicle control technologies. It was also supported by the Higher Education Excellence Program in the frame of Artificial Intelligence research area of the Budapest University of Technology and Economics (BME FIKP-MI/FM).