The role of data in sustainability assessment of urban mobility policies

Abstract

Data have played a role in urban mobility policy planning for decades, especially in forecasting demand, but much less in policy evaluations and assessments. The surge in availability and openness of (big) data in the last decade seems to provide new opportunities to meet demand for evidence-based policymaking. This paper reviews how different types of data are employed in assessments published in academic journals by analyzing 74 cases. Our review finds that (a) academic literature has currently provided limited insight in new data developments in policy practice; (b) research shows that the new types of big data provide new opportunities for evidence-based policy-making; however, (c) they cannot replace traditional data usage (surveys and statistics). Instead, combining big data with survey and Geographic Information System data in ex-ante assessments, as well as in developing decision support tools, is found to be the most effective. This could help policymakers not only to get much more insight from policy assessments, but also to help avoid the limitations of one certain type of data. Finally, current research projects are rather data supply-driven. Future research should engage with policy practitioners to reveal best practices, constraints, and potential of more demand-driven data use in mobility policy assessments in practice.

Policy Significance Statement

This research helps urban mobility policymakers to get more insights in how to better use data in the policymaking process, which provides new opportunities for policymakers toward evidence-based policymaking. In this context, it answers the questions “Is big data increasingly employed and found more useful and powerful than traditional survey data?” “How are the new types of data applied in mobility policy practices” that policymakers usually have when they apply data in urban mobility policy evaluations. Based on the analysis of 74 cases, we summarize the strengths and limitations of different types of data and provide policymakers recommendations for how the potential use of data in sustainability assessment of urban mobility policy and decision-making can be better understood and tapped.

1. Introduction

Cities around the globe struggle to create better and more equitable access to important destinations and services, all the while reducing the energy consumption and environmental impacts of mobility (Schiller and Kenworthy, 2017). Urban mobility issues are crucial problems in many regions because of rapid urbanization in the last several decades, which puts significant pressure on environmental quality, economic structure, and public health in urban areas, and challenges mobility policies (Fedra, 2004). It is now almost three decades since the concept of “sustainable mobility” first appeared in the 1992 EU Green Paper on the Impact of Transport on the Environment. Nevertheless, the transport sector still consumes approximately one-third of our final energy and probably causes more environmental and social problems than any other sector (Holden et al., 2019). Although much progress in understanding its “unsustainabilities” has been made (Gwilliam et al., 2004; Cepeda et al., 2017; Forehead and Huynh, 2018), this has not yet led to the implementation of corresponding policies in practice, leaving urban mobility systems still far from sustainable (European Commission, 2021).

In order to improve the effectiveness of policies, there is need for more evidence-based policymaking (Howlett and Giest, 2012). Evidence-based policymaking requires ex ante assessment of policies, based on data and sound methods. Typical challenges for the effective monitoring and evaluation in urban policymaking practice are: limited financial and staff resources; gaps in technical knowledge; and experience with regard to defining performance indicators, the retrieval, collection, preparation, and interpretation of data (Gühnemann, 2016). One of the challenges the EU’s regulatory scrutiny board has highlighted in the 5th international conference Data for Policy 2020 is the problem of a lack of data: the necessary data in order to evaluate the impact of the policy. Moreover, earlier studies also found that a lack of data and a poor culture of conducting monitoring and evaluation activities in urban governments are limitations in policymaking practice (Chinellato et al., 2017; Awasthi et al., 2018). From interviews with cities that are relatively advanced with sustainable mobility planning, it emerged that for many relevant indicators, data availability and use are restricted—data are either not available at all, its use is restricted, or there is a fee for doing so (Chinellato et al., 2017). Additionally, many cities do not have experience with conceptualizing and conducting evaluations and selecting the most appropriate indicators (Chinellato et al., 2017).

At the same time, developments in the last decade regarding the availability and openness of (big) data seem to provide new opportunities for evidence-based policymaking. Open data are touted as having the potential to transform science and fast-track the development of new knowledge (Dietric et al., 2009). Urban data centers are emerging (Statistics Netherlands (CBS), 2019), while the UN has organized the first UN World Data Forum. The improved access to both traditional and new types of data have the potential to improve evidence-based evaluations of policies regarding sustainability. But how this new potential can be tapped in policy practice is an emerging problem faced by the urban mobility policymakers (OECD, 2016).

Although data may not necessarily be a blessing for policy evaluations, big data is increasingly employed and found more useful and powerful than traditional survey data. Still, it is yet unclear how the new type of data can be applied best in mobility policy practices, for instance, in which part of practical policy cycles. This paper seeks to answer such questions, which helps urban mobility policymakers to get more insights on how to better use data in the policymaking process and also provides new opportunities for policymakers toward evidence-based policymaking. It reviews the state of the art of data use in sustainability assessment (SA) of urban mobility policy in academic literature. Based on the review, this research gives insights on how different types of data are used in urban mobility policy assessment and provides recommendations about how to tap potential for evidence-based policymaking.

The paper is structured as follows. After describing the policy domain of study, urban mobility policymaking and SA are explained in more detail in Section 2. After that, we describe our research method in Section 3. Then, we classify the various types of data used in SAs of urban mobility policies and transportation management (Section 4). In Section 5, we describe a review of 74 case studies to show how these types of data are employed in different (academic) urban mobility evaluations and discuss the advantages and disadvantages of them. Based on an analysis of these cases, we discuss how to improve data use in SA of urban mobility policies. Finally, Section 7 concludes.

2. Urban Mobility and Sustainable Assessment

2.1. Urban mobility

Urban mobility refers to the “way people move in urban areas,” considering all transportation modes (De Oliveira Cavalcanti et al., 2017). As noted, urban planners are challenged to keep urban areas accessible in an equitable and resource efficient way amidst the challenges regarding rapid urbanization, climate change, and others. Under such pressure, traditional urban mobility planning is struggling to give weight to sustainability in policymaking and project implementing, and to adapt to the continually changing social circumstances. Most of the traditional transportation modes consume considerable amounts of energy and resources, which mainly focus on efficiency and convenience for travelers but are highly depended on unrecycled materials and cause serious environmental pollution with negative effects on human health (Schiller and Kenworthy, 2017).

It is now almost three decades since the concept of “sustainable mobility” first appeared in the 1992 EU Green Paper on the Impact of Transport on the Environment (Com, 1992). In 1990, the belief that urban mobility was not sustainable as it was developing became more mainstream among local governments in Europe (Com, 1992). The need for a different approach was seen that included much more priority on public transportation. Nevertheless, cars were still given great freedom, although somewhat restrained by parking limitations and charges, sometimes justified by environmental reasons (Holden et al., 2019). This approach was seen to be more “balanced” as the case was made that the car had to adapt to the city and that the city could no longer cope with the congestion that resulted from the continued growth in car use (Holden et al., 2019). Next to the promotion of public transport, technology was introduced to manage demand to use existing infrastructure most optimally (e.g., traffic control systems, parking indicator systems, and traffic free central areas).

At the beginning of the century, urban mobility had still not become more sustainable (Holden et al., 2019). Although much had been learned about the nature of the problem in a technical sense including possible solutions, the barriers to implement changes in practice had not been overcome (Costa et al., 2017; Ellis and Glover 2019). This sheds light on the societal complexity of the problem: the idea that solutions can be implemented top-down is incorrect, but solutions need to be co-created with multiple actors, transport and parking operators, citizens, businesses, NGOs, along with the municipality.

The 2011 White Paper acknowledged that “still, the transport system is not sustainable” (EU, 2011 p. 4), and stated that “curbing mobility is not an option” (EU, 2011 p. 5). Instead, the White Paper called for a common strategy of de-carbonization.

Since the adoption of the European Commission’s Urban Mobility Package in 2013, the Sustainable Urban Mobility Plan (SUMP) concept has been promoted as a strategic planning instrument for local authorities. It has been proposed as a framework to foster the balanced development and integration of all transport modes and create a harmonized transport offer, while also encouraging a shift toward more sustainable modes and improving transport accessibility for all.

An “Urban Agenda” for the EU was launched in May 2016. It represents a new multilevel working method promoting cooperation between Member States, cities, the European Commission and other stakeholders in order to stimulate growth, livability, and innovation in the cities of Europe and to identify and successfully tackle social challenges. It includes a section on urban mobility, Partnership for Urban Mobility (PUM, EU, 2018), which proposes solutions to improve the framework conditions for urban mobility for cities across the EU. This covers issues relevant to technological advancements, encouraging the use of active modes of transport, improving public transport, and promoting multilevel governance measures.

Based on a survey across 328 cities in Europe in 2017, 44% said they are already conducting integrated sustainability transport planning, including 37% which said they have a plan that qualifies as a SUMP (as defined above). In addition, 16% of cities surveyed declared they were currently developing a SUMP, while 19% were eager to do so. There is a clear growth of cities with well-established SUMP’s from 7 in 2011 to 19 in 2017 (Chinellato et al., 2017). The study also states that simply making SUMPs obligatory in itself does not guarantee the adoption of good quality SUMPs (Chinellato et al., 2017, p. 17). Hence, it is “the way in which” the SUMP is developed and implemented that makes it effective or not. If the local political will and majority for transformation toward low-carbon mobilities is not present, a SUMP plan is unlikely to have much effect.

In summary, after 2010 attention for more structural changes in urban mobility is growing (e.g., modal shift from car mobility to other modes, associated to more attention for public health and livability), and the Paris Agreement has given further thrust to this trend. The question is to which extent these more structural changes are occurring. At first glance, it seems that despite much attention for “sustainable mobility,” both at EU, national and local level, the modal share of car mobility in urban areas is not decreasing significantly. In various urban areas the concept of “sustainable mobility” is reduced to promoting electric mobility and cleaner fuel, but not car alternatives (Bi et al., 2016; Calise et al., 2019).

2.2. Sustainability assessment

According to the research of Intelligent Energy Europe, an EU Program, there are four key policy (making) challenges for sustainable urban mobility: participation, cooperation, measure selection, and monitoring and evaluation (European Commission, 2016). Monitoring and evaluation is a key process to decide if the policies and plans could be implemented in further steps and which measures or approaches should be improved according to the results of evaluations (Chandrakumar and McLaren, 2018). Measure selection should be based on ex ante assessment of options.

SA is an important tool to do such ex ante policy evaluations in an integrated way. SA has been regarded as a “marriage” between environmental assessment and sustainable development (Dijk et al., 2017). It refers to the systematic and integrated frameworks to assess and identify the effects of alternative undertakings and find the best way for progress toward sustainability (Pope et al., 2004; Gibson et al., 2013). It has been widely used in the sustainability evaluation of urban mobility policies (Lima et al., 2014; De Oliveira Cavalcanti et al., 2017).

SA, like policy assessment and formulation, generally consists of four steps (De Ridder et al., 2007): (a) problem analysis, (b) finding options, (c) assessment of options, and (d) follow-up. Ideally, the problem analysis involves data-based evaluation, as Jordan and Turnpenny (2015) note:

“Having established the existence of a policy problem (or problems) through some form of data collection, the various policy-relevant dimensions of the problem are then evaluated to determine their causes and extent, chiefly as a basis for identifying potential policy solutions. (…) While the point is often made that causation tends to be difficult to precisely establish, Wolman observes that “the better the understanding is of the causal process … the more likely … we will be able to devise public policy to deal with it successfully” (Wolman 1981, p. 437). Understanding causation, as Wolman puts it, is also reliant on the generation of adequate theoretical propositions in addition to relevant data on which to support them.”

Clearly, data are vital element of both ex ante assessment of measures and also of monitoring and evaluation in order to understand the current urban mobility status, including the role of implemented policy (Keseru et al., 2019). In practice in Europe however, as noted in Section 1, policy evaluation is generally rather limited and a lack of data and a poor culture of conducting monitoring and evaluation activities exists in urban governments (Chinellato et al., 2017). The rest of this paper seeks to review academic literature to sketch the state-of-the-art on the role of data in SA of urban mobility policy.

3. Materials and Methods

In order to understand the current state-of-the-art data use in urban mobility policy assessments and further to explore the potentials of different types of data applied in this process, we used systematic and critical review as a method to search the relevant published academic books, journal papers, and governmental documents that reported on them, available in academic databases. The whole process is depicted in Figure 1, followed the guidelines of Liberati et al. (2009).

Figure 1. Information flow of literature search and review.

The search term “(urban AND [mobility OR transport OR travel] AND policy AND data)” existed in title–abstract–keywords fields was used in Scopus and Web of Science. The date parameters of publication were limited to 2000–2020 and the search inspected all records published until 1st July 2020. The search in Web of Science led to 2266 records and the search in Scopus led to 2,882 records, of which 647 books, chapters, articles not in English, or not peer-reviewed were removed. Then 4,501 articles were screened by title and abstract. After, these literatures were eye-balled to remove duplicates and the articles are not consistent with the search keywords. The number of literatures then cut to 2,544.

In terms of selecting the studies that are relevant for this research in the full-text read process, the criteria for the selection is presented in Table 1. By reviewing and understanding the data use in these literatures, we identify four different types of data that are frequently used in urban mobility studies and policymaking processes: survey data, statistical data, Geographic Information System (GIS) data, and big data, which has been illustrated more in details in part 4. Hundred and sixty-six papers met the criteria were selected for a second-round of full-text screening for eligibility. Finally, 74 of them were reviewed and analyzed in the case studies (see Table 2 and more details about these cases are list in Appendix A). These final selected cases give most extensive insights about data use in their studies as well as show the state of art of how the data promote or impede policy evaluations. For the discussion part of this review, we also refer to other papers that are not included in the systematic review to discuss findings and for critical review.

Table 1. Literature selection criteria

Table 2. Policy-associated process in the literatures

4. Classification of Data Use in Urban Mobility Studies

Data have played a role in mobility policy planning for decades (Meyer, 2016) and forecasted travel demand, often based on extrapolation from historic traffic intensities, has been important. Also, household travel surveys have been a typical way to understand travelers’ behavior and to evaluate specific mobility policy (Chen et al., 2016). Analyzing previous governmental statistical yearbooks and relevant policies, building spatial transportation models, as well as collecting commuters’ daily travel data, are the key approaches to study the most important mobility issues including travel safety, transport system design, and sustainable mobility development (Hall, 2012). More recently, big data has emerged in mobility studies, which has been largely used in road user’s behavior detections and travel modal shift operations (Welch and Widita, 2019). Across these data applications, we can identify four different types of data that are frequently used in urban mobility studies and policymaking processes: survey data, statistical data, GIS data, and big data.

4.1. Survey data

Survey data have been widely used in various research domains (i.e., social science, economics, policy assessments, and risk management). In urban mobility research, they have largely applied through travel surveys to analyze the motivations and reasons of traveler behaviors in order to stimulate more sustainable mobility behaviors (Bamberg et al., 2003; Cao et al., 2008; Long and Thill, 2015). It is also a new trend to combine Global Positioning System (GPS), smart cards, and such kind of sensing data with survey data together to comprehensively understand urban mobility issues in different angles. These combinations give researchers more opportunities to upscale their studies (Long and Thill, 2015; Gong et al., 2016).

However, some critics argue that survey data are often constrained by unrepresentative sample sizes. For example, the household surveys by the Federal Highway Administration in the United States had a relatively small sample size compared to the size of the project (TMIP, 2013). Furthermore, it may also lose the representativeness if investigators choose unsuitable survey targets. Thus, it is critical to give a certain range of which sort of projects or researches are fitted to use survey data for their studies.

4.2. Statistical data

Statistical data in this study are defined as the statistics compiled from statistical yearbooks and various related documents, which are normally sourced from data collection by official departments and published in governmental reports. It is significant to use statistical information for understanding and quantifying impacts of political decisions in a specific area, which also plays an important role in different research domains, especially in projects, policies, and social development evaluations (Huo et al., 2018; Liu et al., 2018; Yang, 2018).

In urban mobility studies, statistical data have been widely used by policymakers to assess mobility status and to evaluate the implemented planning and policies, also to indicate problems of the current policymaking and implementing process (Cervero, 2013; Annema et al., 2017). The limitations of using these data are that it is hard to monitor the changes caused by one particular indicator and it usually does not contain all the indicators needed by assessments, which means that it can only support basic information to data analysts (Mozos-Blanco et al., 2018).

4.3. GIS data

GIS is a broadly used information technology that has transformed the ways investigators conduct research and has had tremendous effects on research techniques (Foote and Lynch, 1996). ArcGIS is one of the GIS applications using technologies that could help geographers to gain multiple categories of spatial data by working with maps and geographic information, which can also be used to compare the data in different timeframes, and to analyze mapped information applied in a wide range of research domains (Johnston et al., 2001). The main feature of GIS data is that it can provide visual displays for data analysts, especially for policymakers as it helps to transfer complicated data in a straightforward and understandable way (Scott and Janikas, 2010).

According to these characteristics of GIS data, it has been used in urban planning (Maantay and Ziegler, 2006), resource management (Pettit et al., 2008), public health (Hirschi et al., 2002), transportations (Thill, 2000), and also many other different fields. Researchers mainly applied it to acquire the information of landscapes, streets, public transport lines, and roads lines, which all of them are very useful for mobility studies (Greene and Pick, 2012). Increasingly urban mobility researchers combine GIS data together with the other types of data (i.e., GPS data, mobile phone data, and social media data) to detect urban travel modes and behaviors, which could compensate the limited information provided by GIS data (Gong et al., 2012; Khan et al., 2016).

4.4. Big data

Big data refers to data in large volumes, is heterogeneous, and has autonomous sources in decentralized control according to the techniques used to explore the complex relationships among the data (Wu et al., 2014). It has the potential to depict overall macrotrends with huge amounts of available, and with a high level of detail, information, which also helps to change traditional ways of collecting and analyzing data in practice and research (Pucci and Vecchio, 2019). Global Navigation Satellite Systems, location-based services, public transportation cards, and so on all generate numerous data as a by-product in these operations (Semanjski et al., 2016). Big data has been first used in business-oriented domains as the data could measure customers’ performance in which providing rich information and knowledge about consumers’ behaviors and preferences for companies to help them making commercial strategies (Linden et al., 2003; Hasan et al., 2009). From there, it has gradually spread into other fields. It is a new opportunity for experts to exhaustively grasp people’s mobility behavior in order to implement corresponding policies by analyzing these data from multiple sources (Milne and Watling, 2019). Furthermore, it has already made big contributions to solve urban mobility-related issues, such as real-time traffic monitoring, traffic congestion regulation, and traffic accident management (Abdulazim et al., 2013; Calabrese et al., 2013; Tamblay et al., 2016). The typical application of big data in urban mobility studies employ GPS, smart cards, mobile phones, and social media, which we discuss in a bit more detail.

Since the 2000s, it has been prevalent to collect GPS data from GPS loggers, GPS-phones, and GPS-enabled PDAs. With the size and weight of GPS devices becoming smaller and lighter, new potential for gathering people’s mobility information arose (Stopher et al., 2008; Zheng-chang 2008). GPS data include locations, time, speed, and moving tracks (Stopher et al., 2008). Therefore, more and more projects intend to detect people’s travel behavior by analyzing individual movement from GPS data, especially when the cost of these devices has gradually decreased (Liao et al. 2006, Liao et al. 2007). In urban mobility studies, understanding transportation modes, improving traffic regulation, and evaluating management strategies of road networks are the most commonly applied GPS data fields (Mintsis et al., 2004; Bastani et al., 2011). However, the raw GPS data are usually analyzed directly, without understanding trip purposes or other related contexts (Gong et al., 2014).

Smart card data have been predominantly used by public transport systems around the world since the automated data collecting system emerged in the last few decades, which offers sufficient data to investigate travelers’ mobility behaviors for transport planning, traffic management, and mobility policymaking (Pelletier et al., 2011). Two main characteristics of smart cards are that it is quite convenient to take and durable to use (Lu, 2007), which makes it easier to acquire data from smart cards, while it also improves the quality of gathered data compared with magnetic cards. Privacy issue is the biggest concern for card users who do not want to share all of their personal data for analysis (Bagchi and White, 2005).

Mobile phones are becoming an important medium for data analysts to acquire large-scale sensing data used in various domains.Urban spatial planning and management (Louail et al., 2014; Pei et al., 2014) as well as social networks (Phithakkitnukoon et al., 2012; Jiang et al., 2017) are two of the most common areas of study applying mobile phone data, which give fundamental knowledge and experience to other research fields. In terms of urban mobility studies, it not only serves new opportunities and perspectives for investigators to understand people’s mobility behavior by a lower cost approach with large sample size and frequently update datasets, but also supports policymakers to monitor the emerging mobility issues and respond correspondingly through measurements promptly (Calabrese et al., 2013). Meanwhile, analyzing raw mobile phone data is complex work that needs sufficient knowledge of modeling and computer science that are the basic requirement for data analysts to process the huge amount of data and to detect valuable information (Rojas et al., 2016).

Facebook, Twitter, Instagram, Weibo, and so on are the most popular social media platforms for everyone to create their own accounts and share their personal data to others. This type of data has been predominantly used in business analytics in the last decade, for example, companies analyzing social media data to explore what are the most trends, and so on for their business (Kaplan and Haenlein, 2010). In the urban mobility domain, social media data help policymakers detect driving forces of people’s movement behavior, which could be regarded as convincing evidence to make some changes of the current implemented policies according to travelers’ real needs (Hasan et al., 2013). Although it can provide more in-depth data for experts compared with the other types of big data that we mentioned before, there is not a uniform format for social media data analysis, which means more attention is needed for classification of it (Grant-Muller et al., 2014). Moreover, the privacy issue should always be taken into consideration when such data are collected and used.

5. Case Studies Analyses

Figure 2 shows the main data type used in each study. There are 32 cases for survey data, 10 for statistical data, 9 for GIS data, and 23 for big data. Twenty of these 74 cases combined at least two different types of data in their studies. Survey data are the most popular data type for combined data use, which has been applied in 14 of these 20 cases. GIS data and big data are also very commonly applied with other types of data in urban mobility policy-related research, 10 and 9 out of these 20 cases, respectively.

Figure 2. The number of cases for each type of data.

Regarding the publication date of these articles, it could be seen from Figure 3 that the research about data use in urban mobility policymaking becomes more popular after 2011, especially for survey and big data. Although big data use in mobility policymaking studies shows a rapid increase after 2015, survey data still play the main role in this research domain.

Figure 3. Publication year of the final reviewed literatures.

5.1. Cases analyses

Each case’s main characteristics and core information, including data types, sources, subjects, regions, policy associated process (according to the policy cycle explained by Howlett et al., 2009, see Figure 4), and how data used in the cases are summarized in detail in Appendix A. Specifically, what types of data used in each case associated with different processes in a policy cycle are illustrated in Table 2 (please check the serial number of the articles in Appendix A). Furthermore, different types of these studies, including pure academic research and policy practice, are distinguished and shown in the same table as well, which shows that little research has been applied in a real policymaking process.

Figure 4. Policymaking cycle (Howlett et al., 2009).

These cases found and analyzed are mostly academic studies published in scholar journals studying advancements in assessing policies. Some of them did policy assessments first and then took sustainability into account in the discussion, some of them focused on data use techniques for policy assessments but hardly in practice. There was only one SA cases (De Oliveira Cavalcanti et al., 2017) from actual policy practice among all of these 74 cases, which evaluated the sustainability of five urban mobility projects in the Curitiba metropolitan region.

As Table 2 shows, big data is becoming an important resource for urban mobility policy-related studies. Comparably, survey data still play an essential role in the same domain. By analyzing these case studies, we found the strength and limitations of different types of data used in the urban mobility policy-related studies, as follows.

Valuable information and deep insights from different perspectives could be provided by survey data, especially if the respondents are experts in the urban mobility fields. For instance, Mansourianfar and Haghshenas (2018) analyzed interview transcriptions with local mobility policymakers and combined this with analysis of governmental documents in an ex ante assessment of policy measures at the neighborhood level, which provided targeted and insightful recommendations for urban mobility policymaking. On the other hand, small sample size and information being out-of-date are two common limitations of survey data applied in these studies, which had been shown evidently in Hirschi et al. (2002)’s and McGuckin et al. (2005)’s studies.

All of the cases applying statistical data in their studies reflect that it is the most convenient way to collect historical mobility data through various document sources. Moreover, it also plays a vital role in comparing the same mobility policy measure implemented in different cities, such as Mozos-Blanco’s (Mozos-Blanco et al., 2018), which compares the sustainable urban mobility plans of 38 Spanish cities by analyzing the relevant documents. The same historical recorded data could be easily acquired through statistical year books and governmental documents among cities, which provides a stable source of data for policy assessments. One common limitation showed in these cases is that the resolution of statistical data is relatively low, which may cause information loss for the assessments. Wiersma et al. (2016), for instance, note that limitations in the statistical data available prevented them from taking social factors well into consideration in their study.

The biggest strength of GIS data is that it can provide adequate geographical transport information, including traffic lines, locations of transport infrastructures, and urban road networks both in national and regional scales. All the cases which applied GIS data as the main data source in their studies mention that various online GIS databases could be found to support their studies, whereas sufficient experience of relevant software use is required to process data and build models.

The prominent advantage of big data application in urban mobility policy studies is that it provides massive information that can give a comprehensive assessment of urban mobility policy measures based on traveler behavior analysis. For example, massive traffic data were applied in Paffumi et al. (2015) and Zeitler et al. (2012) for the ex ante assessments of policy options in a decision-making process. Moreover, big data also shows strong adaptability of use in different urban mobility policy domains combined with other types of data, especially with survey data in developing decision-making support tools. This can be seen in Jiang et al.’s (2017) and Wismans et al.’ (2018) studies. On the other hand, a limitation of employing big data in urban mobility policymaking is that it is difficult to structure the input data sourced or constructing models, which means it costs much more time to process and analyze these data. Jiang et al. (2017) specifically mentions that, in practice, it will be challenging for urban mobility policymakers to have enough capacity to process and analyze big data.

5.2. Policy-related analysis

Regarding the data use in the policymaking cycle, Figure 5 shows that according to these cases, data use in urban mobility policymaking cycle mainly focused on policy evaluation, decision-making, and policy formulation phases. Big data is the only type of data that has been applied in all processes of the policy cycle and it has been mostly employed in decision-making processes, focusing on decision-making tools development. For instance, Jiang et al.’s examined how to analyze raw mobile phone data combined with census data and geographical datasets in different models in order to see which model is more effective in translating and gaining information for sustainable urban mobility planning. In Andrenacci and Genovese’s (2019) research, floating car data contains information on the travel speed, time, and routes which were continuously detected by devices on board the cars, which helped to obtain information on the journey to be examined in models determining the best policy option. Additionally, big data has also been widely used in policy evaluation, mainly focus on applying real-time traffic data to assess the impacts caused by implemented mobility policy measures in order to make prompt regulations. There are only two cases where data were employed in the implementation process, one of which is from big data. Maranzano et al. (2020) applied traffic data combined with GIS data to assess the early-stage impact of an extended limited traffic zone based on a developed traffic model, which provides in time insights for policymakers to adjust the policy according to the evaluation.

Figure 5. Different types of data use in policymaking cycle.

GIS data are another type of data which have been applied in implementation step to explore the optimized regulation methods for efficient mobility regulation improvement (Wang et al., 2014). Notably, it has been relatively equally used in policy evaluation, decision-making, and policy formulation phases as well, especially by combined with other types of data. It provides the basic information of road networks, regional maps, and other relevant traffic information for traffic model and decision-making tool development.

Statistical data are a valuable resource for both ex ante policy assessment in policy formulation and ex post assessment in policy evaluation, which could be easily acquired in different cities and regions and also easily be compared based on the same dataset. There is only one policy practice case among the 74 cases, which exacted the information from various governmental and academic documents to evaluate the sustainability of the Curitiba metropolitan region mobility projects (De Oliveira Cavalcanti et al., 2017). It provided the policymakers clear sustainability goals to achieve the evaluated mobility projects.

Survey data are the dominant data type used in almost all steps of the policy cycle except implementation and it is one of only two data types that has been embedded in agenda setting processes. Large-scale commuter travel surveys had been used in this step to detect mobility problems and then to set up corresponding policy measures, for instance, McGuckin et al. (2005) did a national survey to investigate participants’ daily travel information so as to define mobility problems for policy measure design. Survey data are also the main resources for ex post assessments of urban mobility policies—nearly 53% of all the cases in policy evaluation process employed survey data as the main database, which shows that big data has not replaced this traditional data type in urban mobility policy assessments.

6. Discussion

6.1. Comparisons of strengths and weaknesses of four different data types

Comparing the 74 cases, we can see that the most detailed information obtained for transport policy planning and assessment is from survey data. It not only contains each respondents’ personal information, but also tells of the motivations behind their travel behaviors directly, which is rather useful for developing sustainable policies. However, limited available data, related to the rather time-consuming nature of organizing surveys, is the main weakness of its application. One suitable way to solve this problem is to set a certain target group of responders, for example, 168 respondents in Soria’s (Soria-Lara et al., 2015) research are EIA developers, transport planners, and some other professionals with transport planning or evaluating experience, providing sufficient valuable information to evaluate the EIA process for urban transportation planning in Spain. Besides this, web-based survey approaches can help to improve efficiency of the data collection by sending easy links to questionnaires to targeted groups.

The application of statistical data in mobility assessment studies is widely practiced as well, especially on a national scale. The main purposes of four (De Grange et al., 2012; Haghshenas et al., 2015; De Oliveira Cavalcanti et al., 2017; Mozos-Blanco et al., 2018) of the 10 cases which applied statistical data were to establish assessment criteria and to make comparisons among different projects and policies. Another case (Wiersma et al., 2016) sheds light on combining statistical data and GIS data together to examine the driving force of car dependency in the Netherlands. The mixed use of data in this research has been analyzed in a spatial context, which provides sufficient knowledge for policymakers to study car dependency caused by different related factors as well as make it easier to see the variety of results among the cities in the Netherlands. Although a large amount of data used in this research aims to solve the research question—how does the spatial context shape conditions for car dependency, social factors, for instance education level, may cause people choose different ways to various destinations, which has not been taken into consideration because of a lack of data.

According to the case analyses, GIS data have an outstanding capacity to do ex ante assessments for mobility policy decision-making compared with other data types, because a variety of policy alternatives can be tested in models to see which one will have the best performance according to different key factors. Financial issues regarding urban transportation planning can be examined by together analyzing GIS and statistical data which was detected through constructing a new methodological approach to measure the spillover effects of transport infrastructure investments in a spatial context dealt with in ArcGIS software (Gutiérrez et al., 2010).It shows that GIS data have somewhat different function in urban mobility research, mostly related to the opportunity to explain the results with maps to policymakers.

Big data has been widely applied together with survey data, road networks data, and GIS data in ex ante assessment and decision-making tool development for urban mobility policy studies. The combination of survey data and GPS data in Zeitler et al.’s project (Zeitler et al., 2012) for identifying suburban environmental impacts and evaluating mobility policy options is instrumental to get insights into both travelers’ basic needs and motivations, as well as their actual travel behaviors, which help policymakers see the real requirements of commuters. One prominent characteristic of big data is that it can provide massive individual traveling information offered by tracking devices. These data can be used to depict selected groups of travelers’ activities and to assess efficiency of the relevant decision-making. Nevertheless, only relying on big data, especially GPS and traffic data, will cause data sparsity problems, as noted in Zhan et al.’s (2016) research. Not only are mobility researchers trying to explore the potential use of big data in sustainable transportation policies and governance development, but also data mining and analyzing scientists have begun to detect the valuable messages from it, extending the implementation fields of big data. A study (De Gennaro et al., 2016) published on “Big Data Research” has developed five models based on the information provided by GPS and GIS data, aiming to better use data in urban mobility policy evaluation and governance. One issue observed through reflections on the big data cases is that big data use in the urban mobility policymaking process is still mainly supply-driven and hardly demand-driven.

Currently, there are some new opportunities for researchers and policymakers to develop better mobility policies since a new data type, social media data, has been used in the mobility policy assessment process. For instance, 1.5 million social media data elements from Weibo (the biggest Chinese microblogging platform) and 8 million smart card data units have been analyzed in Yang et al.’s (2019) study to explore connections between social activities and mobility behaviors. This created insight in various spatial and temporal trends of urban transport. The study suggests that social media data can also reflect travel motivations from those data sharers, while taking less time to collect (than surveys), because it can be collected online. However, one common challenge for big data analysis in urban mobility studies is data processing. It is difficult to structure and format input data that are from various sources. Nevertheless, modeling and programming are both necessary skills that are required for analysts to deal with big data. Moreover, according to the features of transportation policymaking, real-time data monitoring and analyzing are both significant factors to have an effective assessment for urban mobility policies.

6.2. Potential better use of data in SA of urban mobility policies

Sustainable assessment of mobility policies should give insight into the impact of policies in terms of accessibility, environmental, and social indicators (Black et al., 2002; Costa, 2008). Ideally, SAs show possibilities to stimulate transport modal shifts, to reduce private car use, to cultivate people’s green traveling consciousness, and to improve efficiency in urban transport systems (Banister, 2008). Data are an essential ingredient in these assessments. The case studies reviewed helped to learn the current use of different types of data in urban mobility policy assessments, of which most of them are academic studies. Furthermore, it also helps to explore the potential of available data innovations of applications for policy practice.

Da Silva et al. (2015) emphasize that data availability and quality are the most important elements to run an assessment, which also depends on whether policymakers and researchers are involved in formulating assessment criteria based on their problem perceptions. Data reliability should be a concern when analysts are going to deal with collected data as it determines problem solving and corresponding measures designing directions (Witlox, 2007). It is also necessary to weight the representativeness of data before using analyzed results into policy assessment, which has been highlighted for transport studies since 1993 (Schoonees and Theron, 1993). Moreover, privacy is the most common issue that we must care about when we use individuals’ information for policymaking (Hwang et al., 2009; Kifer and Machanavajjhala, 2011; Callegati et al., 2015). A thoughtful way to deal with it is giving announcements to respondents who share their private data for a certain use as well as informing them on the final research results after assessments. Lastly, a practical issue has been mentioned by an EU mobility policymaker in 2019 EU Conference on Modelling for Policy support which was that most of the data currently available for urban mobility policymaking is supply-driven but not demand-driven, which causes policymakers to have limited space when choosing the data they really need. This requires more cooperation among different parties working together to give more opportunities for mobility policymakers gathering the data they need for policymaking.

In order to advance mobility policy assessments in terms of data use, exploring the role of data in various phases of the policymaking cycle and detecting what kinds of skills and expertise are needed for policymakers could be helpful. Firstly, in the agenda setting phase, historical data collection and processing can be used to define and frame problems (Doern and Phidd, 1983). However, if the statistical data, such as the number of electric vehicles, charging stations, and PM 2.5 emissions, could not be measured periodically in this stage, it would be demanding work for policymakers to define actual mobility problems. This step also requires policymakers to select basic indicators that are easy to collect periodically which is essential for urban mobility issue defining. In the second phase, policy formulation, policy options should be developed and preliminarily ranked. Traditional survey data, GIS data, and big data all show their usefulness for policy option formulation, especially the combined use of survey data and big data has been found to have a big potential to help understand traveler behaviors and corresponding motivations. This will help policymakers design more humane and sustainable transport policy measures. Besides this, ex ante assessment can also be very useful in both this stage and next in the decision-making process, which helps policymakers select the most suitable solutions.

In the third phase, the final policy measure for implementation needs to be decided. Gathering GIS data and traffic data processed in ArcGIS software is an effective way to evaluate and compare different policy options. This gives governments more chances to see different forecast results based on varied scenarios and further to draw a bigger picture of their transportation planning. Analyzing the data in this step requires professional employees such as modelers and data analysts since sufficient data processing and modeling knowledge are needed to dig information from raw GIS and big data. Otherwise, it will cause a common problem facing policymakers where they have a lot of data but they do not know how to select and use it.

In the fourth step, selected policies should be implemented, and in the last phase, monitoring and evaluation of the policies should be conducted. In practice, these two steps are often not sequential but iterative (Hessing and Summerville, 2014). Big data, such as real-time traffic data, GPS data, mobile phone data, and social media data, can give more in-time reflections of implemented policies in this period, which could let policymakers make prompt adjustments responding to the problems showed in the policy implementation phase. Ex post evaluation can also employ statistical data and survey data to compare outcomes of current policy with those of previous policies as well as to analyze feedback from travelers after policy trails, which is an important step to respond to potential problems. However, this also requires sufficient work capacity from the urban mobility departments to conduct monitoring and evaluations.

7. Conclusion

In this paper, we review recent (2000–2020) academic literature on urban mobility policy assessments to understand the current state-of-the-art of data use in these activities and further explore the potential of available data innovation in more evidence-based policymaking. The 74 case studies reveal a surge of attention and availability of open, big data, although, it cannot replace traditional data usage (surveys, statistics). We do find that the new types of data provide new opportunities for evidence-based policymaking.

Overall, the data use innovations in SA for urban mobility policy can be concluded as follows: (a) big data shows the most potential for use in decision-making support tools development, especially combined with survey data which shows even higher effectiveness; (b) Specifically, big data (most of the available big data are location-based data) used in traffic models can more easily provide detailed information about travel patterns, but reveals less about motivation while traditional surveys remain more useful for this; and (c) The use of new types of data in urban mobility policymaking requires policymakers and related working staff to have certain knowledge and skills for data analysis, modelling and extra working capacities.

7.1. Limitations and future research

In the literature search and selection process, it was a criterion that studies shed light on the use of data in urban mobility policies, especially for policy assessments toward sustainability. Because of this real-place particular focus, a broad range of mobility policy assessments in the literature are left out. Additionally, only one of the case studies is based on policy practice, while the others are all academic research, so we can hardly conclude with extremely certain suggestions for urban mobility policymakers in practices.

Moreover, because the 74 cases are mostly academic studies, not from actual policy practice, it is a gap that should be addressed by future research since innovation in policy assessment likely takes place in practice as well. This can lead to better understanding of the use of state-of-art of data in practice and recommend the most optimal use of new data types used in urban mobility policymaking. The studies we reviewed did not reveal how policymakers appreciate the various data types and how they are involved in shaping data analysis. It seems like there is a tendency for supply-driven data in practice as well. Studies of innovation in policy assessments in practice can reveal the best applications, constraints, and potential of more demand-driven data use in mobility policy assessments.