Identifying Spillovers of Trade Agreements through Impact Assessments: A New Database

Abstract

In the past decades, a backlash against globalization has been brewing, especially in advanced economies. Despite this backlash being only partly determined by trade, we observe an increasing demand for transparency on procedures, methodologies, and results. Impact assessments (IAs) aim at identifying expected effects of trade agreements and at highlighting policymakers' concerns, thus representing an important tool to foster public acceptance. To help us identify spillovers of trade liberalization, we construct a country and sector-specific database of impact assessments. This database provides an overview of the evolution of the coverage and methodological approaches taken by the EU and US for their IAs. We rely on official EU and US sources over the period 1990–2023. We first observe differences in terms of methodology and institutional framework within and between the two regions. Secondly, the coverage of non-trade outcomes has evolved over time both for the EU and the US, with the inclusion of more labour, environmental, and human rights indicators as well as cross-cutting issues. We observe that the depth of the evaluation is correlated with the partner country's social protection and environmental performance. Lastly, we find that the inclusion of a sector in the analysis is driven by economic reasons in the EU but by political reasons in the US.

1. Introduction

In the past decades, we have observed a rising backlash against globalization, particularly in advanced economies and partly determined by trade. We have also witnessed a shift towards protectionist policies following the 2008 financial crisis (Colantone et al., 2022). Despite this growing backlash from society, trade agreements have deepened over the past 20 years (Mattoo et al., 2020). Several databases on the content of preferential trade agreements (PTAs) have been constructed to capture this growing depth of trade agreements. For instance, Horn et al. (2010) compare the content of EU and US PTAs and whether they include a significant number of obligations in areas not currently covered by the WTO Agreement (‘WTO-X provisions’) while Dür et al. (2014) and Mattoo et al. (2020) have expanded the country coverage. The expanding mistrust towards trade liberalization can be partly explained by the potential negative spillovers an agreement might have on society. Impact assessments (IAs) carried out by countries before the ratification of an agreement aim at identifying the expected effects of trade agreements (Baker and Thi Hong Le, 2022). They could, thus, be used as a tool to highlight policymakers' concerns but also be important for public acceptance. However, little information is known about the EU and US impact assessment process. This is surprising given the growing demand for transparency from society. This paper sheds light on the methodology used and how it differs across both economies.

This paper first presents the institutional framework regulating the ex-ante PTA evaluations in the EU and the US. It highlights the differences over time and across countries. The main difference between both regions is the institution in charge of running the evaluations: while the US conducts the evaluations internally, the EU contracts independent consultants to carry out these analyses. To reflect the potential spillovers resulting from the PTAs and to highlight society's concerns, we construct a database on the content of impact assessments.¹ We code the different spillovers into four categories: economic, social, environmental, and human rights. In addition, we also code ‘cross-cutting’ issues, issues that affect multiple sectors and are important for the approval of the trade agreement by civil society and policymakers. This database first allows for a comparison between the US and the EU's evaluations. By coding the economic, environmental, and employment US ex-ante evaluations and EU sustainability impact assessments (SIAs), we emphasize the heterogeneity within a country across time and within time across countries.

We also include in the database evaluations of different trade agreements regardless of their signature status. We combine impact assessments of signed PTAs, PTAs in negotiation, and PTAs the EU/US withdrew from. We are then able to compare these PTAs using information contained in the database and find that the number of social and human rights indicators is higher in not-signed/in negotiation PTAs. The number of cross-cutting issues investigated in the ex-ante evaluations is also higher in not-signed/in negotiation PTAs. Another main feature of the database is to include a sectoral dimension in the analysis. This feature points out the differences across PTAs and reflects the importance of the partner country in the evaluation. Political motives appear as a main driver in the choice of sectors in the US; its probability of carrying out a sector analysis decreases when it has a comparative advantage in this sector. However, economic reasons seem to drive the choice in the EU; its probability of running a sectoral analysis increases when the EU has a revealed comparative advantage (RCA) in this sector and decreases when the partner country has one RCA in this sector. We observe that IAs' social analysis is deeper when partner countries have lower labour market protection. A similar pattern is found for partner countries with lower environmental performance. However, using a policy perspective proxy, we observe a positive correlation with the number of environmental indicators, indicating either a growing interest in environmental issues or a protectionist stance. Finally, we do not find any correlation between the number of human rights indicators in IAs and the performance of partner countries in this regard.

Our work builds on and contributes to several strands of the literature. It first relates to the literature studying the role of transparency in regulatory governance. Dudley and Wegrich (2016) examine the transparency of procedures in the US for general impact analysis and public comment and compare it with what is in place in the EU. The authors conclude that each jurisdiction emphasizes transparency, analysis, and consultation to different degrees and at different stages in the regulatory process. They find that both the EU and the American procedures have been rightfully criticized for lack of transparency regarding the inputs underlying regulatory decisions. Piermartini and Teh (2005) highlight a ‘black-box feel’ to Computable General Equilibrium (CGE) models that could be reduced with further information on the process used. They also provide a comprehensive picture of the most relevant strengths and weaknesses of CGE and gravity models. Compared to the above works, this paper documents how the US trade impact assessment process is structured and provides a comparison with the European impact assessment procedure. More closely related, Kirkpatrick and George (2006) examine the challenges of consultation and stakeholder involvement in multi-country studies and the technical aspects of the European impact assessment process. Hoekman and Rojas-Romagosa (2022) also analyse the EU SIA consultation process and conclude that to become more effective and representative the SIA should devote attention to a more in-depth and evidence-based evaluation of a limited set of priority non-trade issue areas. Moïsé and Rubínová (2021) carry out a comprehensive and critical review of existing methods for performing sustainability impact assessments of FTAs by discussing the strengths and weaknesses of various approaches employed by different countries, such as the EU, Canada, Switzerland, and the UK. Rojas-Romagosa (2018) summarizes the evolution of the coverage and methodological approaches of non-trade policy objectives (NTPOs) in EU SIAs and finds that the analysis of the impact of NTPOs has substantially increased over time, with labour-related issues and human rights topics gradually gaining more prominence.

With respect to these works, we include a larger sample of trade agreements, including all trade agreements signed by the EU and the US, regardless of their signature status, which allows for comparison between the countries. Our work also evaluates how different spillovers are investigated across PTAs, over time, and at the sector level.

Finally, the literature assessing PTAs' spillovers has been increasingly focusing on nontrade outcomes (Fernandes et al., 2023). Lechner (2016) shows that social and environmental clauses in trade agreements are shaped by domestic protection levels. By contrast, Francois et al. (2023) do not find any evidence that provisions related to labour or civil rights improve the associated outcome indicators, while results are mixed for environmental outcomes. To our knowledge, we are the first to use information on impact assessments to highlight which non-trade outcomes are evaluated during the negotiations and are important to policymakers.

The remainder of the paper proceeds as follows. Section 2 presents the institutional framework used for the impact assessments in the US and in the EU as well as the methodologies used to carry out these ex-ante evaluations. Section 3 describes the construction of the database and highlights some of its characteristics and new stylized facts. Section 4 concludes.

2. Institutional Frameworks and Methodologies

A relevant part of the analysis relies on understanding how the institutional framework governing the impact assessment process in the US and the EU is structured. The laws regulating this process reflect policymakers' concerns while also providing a homogeneous starting point for the creation of a database of official impact assessments. More specifically, it is important to understand who the formal actors involved in these assessment procedures are, the exact timeline and methods (both quantitative and qualitative) employed, and how they differ or potentially overlap over time and across the two economies.

2.1 United States

The impact assessment process in the US has been legally regulated since the Tariff Act of 1930. More specifically, Section 1332(g) of the Act gives the Office of the United States Trade Representative (USTR) the authority to request a report from the US International Trade Commission (USITC) to assess the potential economic impacts of newly signed trade agreements. Over time, new legislation has established the system that is now in place.

In particular, the Trade Act of 1974 establishes an Advisory Committee for Trade Policy and Negotiations (ACTPN) to provide overall policy advice while granting the President the possibility to establish individual Policy Advisory Committees (PACs) for industry, labour, agriculture, services, investment, defence, and other interests as deemed appropriate given the complexity of the agreement under negotiation. These committees have varied in size and scope depending on the specificities of the related trade agreement.²

The system is arranged into three tiers: the ACTPN is in the highest tier, while the Agricultural Policy Advisory Committee (APAC), the Intergovernmental Policy Advisory Committee (IGPAC), the Labor Policy Advisory Committee (LAC), and the Trade and Environment Policy Advisory Committee (TEPAC) are in the second tier. In the third tier, there are 27 sectoral, functional, and technical advisory committees, organized in two areas – agriculture and industry – and appointed by USTR and the Secretary of Agriculture or Commerce, respectively. These committees include in their reports a general view of relevant stakeholders on the potential impacts of the trade agreements for their respective sectors. This constitutes a fully fledged consultation process that informs the qualitative analysis of the USITC economy-wide impact assessment.

The US Trade Promotion Authority Act of 2002 (Section 2104(f)-(g), 19 U.S.C. § 3804(f)-(g)) puts the USITC in charge of official ex-ante impact assessments following a strict schedule, specifically no later than 90 calendar days after the President enters into the agreement. In addition, the Act introduces two additional assessments related to the potential impact of trade agreements on domestic labour and the environment, carried out by the Department of Labor (USDOL) and USTR, respectively. These assessments draw upon their own calculations, the respective second-tier PACs' reports, and the relevant expertise of federal agencies.

Currently, the economy-wide impact assessment is regulated by Section 105 of the Bipartisan Congressional Trade Priorities and Accountability Act of 2015 (19 U.S.C. § 4204), which keeps a similar structure and timeline as defined in 2002. The USITC is required to conduct and publish an assessment if and only if the agreement is signed. For instance, no official US impact assessment has been published in the case of the Transatlantic Trade and Investment Partnership (TTIP) due to the negotiations failing in 2015.

2.2 European Union

Contrary to the United States, there is no legal framework regulating EU impact assessments, but only a methodological process since 1999 and an evaluation framework entitled ‘Better Regulation’.³ The European Commission (EC) conducts four types of evaluations. It first carries out an ex-ante impact assessment on initiatives expected to have significant economic, social, or environmental effects. The impact assessment includes a description of the potential impacts. If any of these effects are considered significant, it highlights who might be affected by the initiative and how the consultation process and the results obtained from it support the EC and the EU member states' governments when deciding whether to approve the launch of new trade deal negotiations.

Since 1999, the EC conducts a second type of ex-ante evaluation: the SIAs.⁴ The EC contracts independent consultants to conduct SIAs, which estimate the potential economic, social, human rights, and environmental impacts. The SIA methodology was first described in an official handbook in 2006 and later refined in the second handbook in 2016. It consists of three phases: (i) The inception report describes the methodological approach proposed by the external consultants, the sectors to be analysed, the stakeholders to be consulted, and a preliminary screening and scoping process to identify key sustainability issues. (ii) The interim report presents the initial expected impacts on the general economy as well as potential sectoral analyses. Moreover, the consultation process is intensified using workshops, questionnaires, and interviews. (iii) The final report must include a list of recommendations and accompanying policies to be implemented to mitigate potential negative spillovers. One issue that was raised due to this change in methodology is the absence of a commonly agreed definition of sustainable outcomes. The methodology moved from an indicator-based analysis to a risk-based one (even if indicators are still employed), aiming at identifying social groups at risk and specific environmental issues. Contrary to the US, the EU releases the SIAs of trade agreements even if the negotiations end up failing or if the agreement is still in negotiation.

Thirdly, the EC conducts an economic analysis of the negotiated outcome, using tariff dismantling schedules and information on non-tariff barrier reductions to evaluate the economic impact of the agreement. Finally, once the trade deal has been signed, the EC conducts an ex-post evaluation of the effects of the agreement. This evaluation allows the Commission to assess whether the agreement has achieved its objectives.

2.3 Methodology

Both countries use a similar mix of qualitative and quantitative methods for their ex-ante evaluations. However, their methodology has changed over the years.

In the early years of conducting impact assessments, the US initially relied on various papers to estimate the potential spillovers of agreements, primarily using Computable General Equilibrium (CGE) modelling techniques. At the conclusion of each paper, one can find critical comments from experts. These comments mostly focus on the methodological shortcomings of the paper, such as the choice of parameters, level of aggregation, and tariff levels. In general, the policy instruments investigated include the removal of tariffs and non-tariff barriers, as well as their effects on trade, employment, and GDP.

For example, the USITC relied on a symposium of 12 papers to evaluate the impact of NAFTA, focusing largely on the relationship between the US and Mexico. Interestingly, while the forum is US-based, the papers presented are the result of collaborations between economists and research institutions of the three countries.

Following the Trade Act of 2002, the USITC now provides a more comprehensive evaluation of assessments, utilizing both quantitative and qualitative methods. The quantitative assessment is focused on the liberalization of tariffs and tariff-rate quotas. It employs a multi-country model with economy-wide coverage of merchandise and service sectors (a global CGE model). Goods and service sectors identified for qualitative analysis are selected based on a comprehensive examination of the trade liberalization schedules of the Free Trade Agreement (FTA). Information to assess the liberalization of barriers in non-quantifiable areas, due to lack of data or their intangible nature, is obtained from industry and public sources, testimonies presented at public hearings at the Commission, and written submissions in response to Federal Register notices. Government sources may also be used to gather information for the report.

The EU uses a similar methodology. However, the evaluation is conducted by a consortium of independent consultants led by one institute. This consortium of consultants and the consultation process often allows the SIA to incorporate the perspective of the partner country.

The primary quantitative analytical component of SIAs is a Computable General Equilibrium (CGE) model, which generates the main economic effects of the trade agreement. A specific number of sectors are selected for more detailed analysis, usually involving partial equilibrium methods. The results of this ‘primary’ coverage are then supplemented with qualitative techniques, including Causal Chain Analysis, to assess the potential social and environmental impacts of the trade agreements. In some cases, particularly for environmental analysis, a more integrated quantitative approach is employed by linking a CGE model with an energy model (E3MG), for instance. From a labour perspective, household and individual-level micro-data are used to quantitatively investigate the impact of trade liberalization on inequality. However, these more complex endeavours are often limited by data availability and are therefore not consistently applied across all studies. Table 1 presents a summary of both frameworks and methodologies.

Table 1. Comparison of the US and EU impact assessment framework

3. A New Impact Assessment Database

This section first explains how the database is constructed and the information it contains.

We then present some descriptive statistics and new stylized facts derived from it.

3.1 Database Construction

To enhance transparency in the impact assessment procedure, we construct a database coding official impact assessments conducted by the EU and the US over the past 30 years.⁵ The choice of countries is motivated by their importance in trade negotiations and the availability of data in English. To build this database, we first rely on the list of all trade agreements signed by the EU and the US since 1989, provided by the Desta Database developed by Dür et al. (2014). This list allows us to include trade agreements regardless of whether they are notified to the WTO. To cover the entire sample of trade agreements, we also include agreements that are still in negotiation and those for which negotiations started but were not concluded.⁶ For EU agreements, we use the list provided by the European Commission, while for the US, we rely on the list developed by Konken (2021).⁷

The first part of the database includes, for each treaty, the year of signature and entry into force when applicable. It also indicates whether an impact assessment, a labour, environmental, human rights evaluation; SIA; and ex-post evaluation have been conducted, and in which year. We also include the main institution in charge of the impact assessment and the document used.⁸

In the second part of the database, we code the different spillovers assessed. To address not only the economic dimension but also the growing demand to assess the social and environmental impacts of trade agreements, we code the economic, environmental, and employment impact assessments conducted by US institutions, USITC, USTR, and USDoL, respectively.⁹ For comparison purposes, in terms of content and timeline, we code the SIAs commissioned by the European Commission. To cover the latest version of the agreement, we focus on final impact assessment reports, often published near the end of negotiations when the trade agreement is nearing finalization. Indeed, an earlier assessment might not cover all the areas present in the agreement. For example, the Australian Department of Foreign Affairs and Trade commissioned a study to investigate the potential economic effects of AUSFTA in 2001, although the trade agreement was only signed in 2004. The impact assessment assumed liberalization across several major sectors. However, the sugar sector was not liberalized in the final version of the trade agreement, leading to an overestimation of trade gains.¹⁰

We could not use final reports in two cases. First, for trade agreements still in negotiations, we consider the latest available report.¹¹ Second, in the case of negotiations for multi-regional trade agreements, multiple SIAs were conducted at different stages, focusing on different groups of countries or specific sectors. For example, in the EU-ACP Economic Partnership Agreement negotiations, Phase 1 reports focused on regional trade in West Africa and the Caribbean; Phase 2 reports focused on Agribusiness in West Africa, Tourism in the Caribbean ACP region, and Fisheries in the Pacific ACP region; and Phase 3 reports focused on Horticulture in Eastern and Southern Africa. Thus, we combine two SIAs (Phase 1 and Phase 2) to evaluate the agreement between the EU and Cote d'Ivoire.¹²

To cover various spillovers of interest to policymakers and civil society, we selected a list of economic, social, environmental, and human rights indicators. Initially, we created a selection of indicators based on a smaller sample of impact assessments over different periods to account for changes in methodology across both economies. We then expanded this list to include other relevant indicators commonly used in trade agreement evaluations. The database includes 17 economic indicators (Table A.1), 19 social indicators (Table A.2), 19 environmental indicators (Table A.3), 9 human rights indicators (Table A.4), and 20 cross-cutting issues (Table A.5).¹³ The cross-cutting issues identify specific components of the trade agreement that affect multiple sectors, such as intellectual property rights, investment, competition policy, and sanitary and phytosanitry measures (SPS). The significance of these horizontal issues for agreement acceptance was underscored by the contentious debate surrounding the investment chapter in CETA during its negotiation and ratification process.

To assess the range of indicators estimated in evaluations, we systematically read all the impact evaluations carried out by each country and coded a dummy variable equal to 1 if the evaluation mentions the indicator and 0 if it does not. For instance, if the impact assessment estimates the effect on ‘Wildlife trafficking’, the dummy is 1. It is worth noting that even if the evaluation predicts no effect on ‘air pollution’, the dummy is still 1 as the impact is assessed. The database does not report the magnitude or sign of estimated effects as we aim to reflect policymakers’ concerns rather than the magnitude of potential trade agreement spillovers.

A novel feature of our database is the inclusion of a sectoral dimension to enhance analysis precision. We select specific sub-sectors investigated in case studies, grouped within five main sectors: agriculture, mining and energy, manufacturing, services, and environmental goods and services.¹⁴ When an indicator is examined only at the sub-sector level, we also include it at the economy level.

Economic effects often stem from CGE analyses, while other indicators typically rely on qualitative assessments. When the impact on an indicator for a sector studied in a specific case study is not explicitly mentioned but can be inferred from CGE analysis results, we consider it covered by the impact assessment, assigning it a value of 1.

3.2 Descriptive Statistics

Table 2 presents the number of impact assessments and trade agreements included in our database. Building upon DESTA as an initial foundation to identify PTAs signed by the EU and the US, we also incorporated agreements under negotiation and those that were not concluded. Currently negotiated agreements by the EU have impact assessments available, whereas for the US official assessments are only conducted upon agreement signing.

Table 2. Number of PTAs and impact assessments per country

Our sample encompasses agreements that entered into force starting from 1990. Of the 74 agreements that the EU has signed since 1989, all have entered into force. The US has signed 18 agreements out of 37, and with the exception of the TransPacific Partnership (TPP), they all entered into force.¹⁵ Simultaneously, multiple agreements are in the negotiation phase for both the EU and the US, with modernization of agreements considered part of the negotiation process.

We identify agreements that have not been signed through the list provided by the EC, while for the US we rely on the list developed by Konken (2021). Nearly all US-signed agreements have a corresponding impact assessment (excluding the agreement with Vietnam concluded before the Trade Act of 2002, and the Trade and Investment Framework Agreement with Laos), but the same is not true for the EU. When comparing the percentage of agreements covered by an impact assessment for the 1989–2023 period, the figures are quite similar for both countries: 53% for the EU and 43% for the US. However, when comparing the percentage of signed agreements covered by IAs, 89% of US-signed agreements are covered but only 40% of EU-signed PTAs are covered. This difference is partly due to the accession treaties for which an official SIA was not carried out. Regarding the difference in terms of the number of impact assessments carried out, the EU has performed 13 evaluations for agreements in negotiation. Another notable difference is the relatively high number of negotiations the EU is involved in compared to the US. The US, however, has withdrawn from substantively more agreements than the EU has.

Figure 1 displays the number of indicators in each category (economic, social, environmental, and human rights) investigated in EU and US IAs. The number of economic indicators investigated is relatively stable over time (around 10) for both the EU and the US. In the left panel, it is observed that the number of social and environmental indicators investigated by the EU has slightly increased over time, while the investigation of human rights indicators has grown in the latest years of the sample. In the right panel, the number of social indicators investigated by the US has increased over the period, while the investigation of human rights indicators has remained quite low and stable.

Figure 1. Number of indicators per PTA and category

Notes: PTAs marked with the letter N are PTAs in negotiation, the ones marked with the letter A are accession, the ones marked with the letter M are modernization, and the ones marked with the letter W are the ones the EU/US withdrew from.

To exploit the richness of our data, we then compare the number of indicators investigated in signed EU PTAs and those not signed or in negotiation (Figure 2). The trends for economic and environmental indicators do not appear to differ across the two samples.

Figure 2. Number of indicators in signed versus not signed/in negotiation EU PTAs

Notes: The left panel shows the number of indicators for PTAs the EU signed, while the right panel shows the number of indicators for PTAs the EU did not sign or is negotiating.

However, the number of social and human rights indicators is higher in the non-signed/in negotiation PTAs on average.

Figure 3 shows the number of cross-cutting issues investigated in each IA. Looking at the left panel, one can observe that the number of cross-cutting issues investigated by the EU grows on average over time. In constrast, looking at the right panel, the number of cross-cutting issues explored by the US increases in the early years of the sample and then remains relatively stable.

Figure 3. Number of crosscutting issues by PTA

Figure 4 shows the number of cross-cutting issues examined by the EU in signed PTAs and in non-signed/in negotiation PTAs. While a growing trend is observed in both samples, the number is larger in the group of PTAs not signed or in negotiation.

Figure 4. Number of crosscutting issues in signed versus not signed/in negotiation EU PTAs

Notes: The left panel shows the number of indicators for PTAs the EU signed, while the right panel shows the number of indicators for PTAs the EU did not sign or is negotiating.

To further investigate the sustainability content of the ex-ante evaluations, Table 3 displays the number of indicators investigated in each of the main sectors. On average, the EU considers 7.4 economic indicators in both the agriculture and manufacturing sectors. Agriculture also has the highest number of social and environmental indicators investigated, while human rights are primarily examined in the mining sector. However, more than 50% of IAs do not explore human rights in sectoral analysis.

Table 3. Number of indicators at the sector level

The US explores, on average, 6.6 economic indicators in the agricultural sector, closely followed by the manufacturing sector with 6.4 economic indicators on average. This detailed economic analysis may be due to the significant share these sectors represent in terms of employment and GDP. Social indicators are equally assessed across all sectors on average, while the services sector has the highest number of environmental indicators. Human rights indicators are predominantly explored in the manufacturing sector. Nonetheless, more than 50% of IAs do not explore human rights in sectoral analysis.

Digging deeper into the sectoral analysis, Figure 5 displays the average number of sub-sectors investigated by the EU and the US in their ex-ante evaluations. As the number of sub-sectors investigated might increase along with the depth of the PTA, we also include the number of deep PTAs signed per year, as well as the number of PTAs in negotiation and the number of PTAs from which the EU/US withdrew. However, the number of signed deep PTAs does not seem to follow the same growth pattern as the number of sub-sectors analysed.¹⁶ On the other hand, the number of PTAs from which the US has withdrawn has been following a similar trend to the number of sub-sectors analysed, while the number of PTAs the EU is currently negotiating has increased alongside the number of sub-sectors in the last 10 years.

Figure 5. Number of PTAs and of sub-sectors investigated

Notes: The left panel shows the number of deep PTAs signed by the EU, the number of PTAs the EU has withdrawn from, the number of EU PTAs in negotiations per year, and the number of sub-sectors investigated in SIAs. The right panel shows the number of deep PTAs signed by the US, the number of PTAs the US has withdrawn from, the number of US PTAs in negotiations per year, and the number of sub-sectors investigated in ex-ante evaluations.

In Table 4, the frequency measures the number of impact assessments investigating each of the five main sectors. The EU carries out the impact assessment only at the economy level (i.e., without any sectoral analysis) in five IAs, while the US always investigates the effect on the manufacturing sector in addition to the economy level. The effect on the agricultural sector is evaluated in 87% of ex-ante evaluations conducted by both the EU and the US. The impact on the manufacturing sector is investigated in all US evaluations and in 79% of EU assessments. Services sectors are considered in more than 70% and 86% of EU and US evaluations, respectively.

Table 4. Frequency of (sub-)sectors investigated in IAs

The second and fourth columns of Table 4 show the average number of sub-sectors investigated in each sector. Agriculture is the sector with the most sub-sectors investigated by the EU, with an average of 4.4, while around 3 sub-sectors in the manufacturing and services sectors are investigated. In US evaluations, manufacturing is the sector with the most sub-sectors investigated, with an average of 7.2, followed by agriculture and services, with 6 and 5.9 sub-sectors evaluated on average, respectively. The agricultural sector seems relatively more important for the EU, while the manufacturing sector seems relatively more important for the US, reflecting differences in trade patterns. Indeed, having a comparative advantage in a sector is correlated with the choice of sub-sectors included in the analysis for both the EU and the US (Figure 6).

Figure 6. RCA and sectoral analysis

Note: Time fixed effects are included in the econometric specification and standard errors are clustered at the PTA level.

3.3 Stylized Facts

We investigate the possible drivers behind the choice of sector selection during the evaluation. We proxy the importance of the sector by calculating whether the country has a Revealed Comparative Advantage (RCA) in the sector analysed.¹⁷ We use the trade database developed by Borchert et al. (2021) to calculate RCA.

We observe that the EU is more likely to carry out a sectoral analysis when it has a comparative advantage in this sector but is less likely to conduct a sectoral analysis when its partner has an RCA in a given sector (Figure 6). Its probability of conducting a sectoral analysis increases by 2% when the EU has an RCA, while it decreases by 3.6% when at least one of its partner countries has an RCA. However, the US is less likely to conduct a sectoral analysis: its probability decreases by 3.9%, and the choice of sectors is not affected by the RCA of its partner countries.¹⁸

This differing result might be explained by different political considerations driving the choice of sectors. Political factors such as the lobbying power of a sector may play a more significant role than its economic strength when the US selects sectors for analysis.

Secondly, we explore whether partner countries’ performance in terms of social, environmental, and human rights influences the depth of the respective impact assessment.

To this end, we estimate the correlation between the aggregate number of indicators in each category investigated in the impact assessment (IA) and social, environmental, and human rights indicators at the partner-country level in the year the IA was published.¹⁹

We rely on different sources for these indicators. For social indicators, we use the labour market index from the Non-Trade Policy Outcomes (NTPO) database developed by Manchin (2021).²⁰ For environmental indicators, we consider total CO2 emissions from the EDGAR database by Crippa et al. (2020) as a proxy for the partner country's environmental performance. Additionally, we use the total number of climate change and mitigation laws and policies from the Climate Change Laws of the World (CCLW), a database developed by the Grantham Research Institute on Climate Change and the Environment (2021). These environmental indicators are sourced from the University of Gothenburg Environmental Indicators dataset developed by Povitkina, et al. (2021). Lastly, we utilize the human rights index from the Our World in Data database created by Herre et al. (2016).

We observe a negative and statistically significant correlation between the depth of the social analysis in impact assessments (IAs) and the labour market performance in partner countries (Table 5). This suggests that particular attention is given to potential societal backlash against the agreement when the partner country's level of social security is low. We document a positive and statistically significant correlation between the level of CO2 emissions in partner countries and the depth of the environmental impact assessments. Similarly, it appears that the presence of more laws and policies on climate change and mitigation in partner countries is associated with deeper environmental assessments. This could reflect an increasing interest in potential environmental spillovers of trade liberalization. However, this finding could also be attributed to the fact that import tariffs and non-tariff barriers are generally lower for industries with higher emissions than for cleaner industries (Shapiro, 2021). From this perspective, a higher probability for the EU to conduct an IA when partners have lower environmental standards could potentially reflect a protectionist attitude rather than a genuine interest in environmental impacts. Finally, we do not observe a meaningful correlation with the quality of human rights indicators. This might be due to the relatively low inclusion of human rights indicators in IAs.²¹

Table 5. Correlation between the number of indicators in IA and partner countries’ performance

Notes: Number of observations in parenthesis, varies due to data availability.

*, **, ***indicates significance at the 10, 5, and 1% level, respectively.

4. Conclusion

This paper aims to increase the transparency of the IA process by providing a comparison of the methodologies used by the EU and the US in their ex-ante evaluations. We construct a database on the content of IAs to emphasize the heterogeneity of evaluations across time and across countries. The inclusion of economic, social, environmental, and human rights indicators allows us to compare and investigate the content of the assessments. We also include cross-cutting issues to cover all possible spillovers that might be of interest to policymakers and/or civil society. We compare the content of IAs based on the partner country's social, environmental, and human rights performance. We find that the lower the country's social protection and environmental performance, the deeper the evaluation is, which suggests that the IA tries to integrate potential backlash from society.

A new feature of the database is the inclusion of PTAs regardless of their signature status. When comparing the content of IAs, we observe that evaluations of unsigned or in-negotiation EU PTAs incorporate more social and human rights indicators, on average. A second important feature of the database is the inclusion of a sectoral dimension. This allows us to compare the choice of sectors in both the EU and the US. The choice seems to be driven by political reasons in the US, while economic reasons appear relatively more important for the EU, as its probability of investigating a sector increases if the EU has an RCA in that sector.

This database can also be used in different disciplines. In law, it might be useful for identifying relevant flanking policies depending on the spillovers identified in the impact assessment. International relations scholars might be interested in the timeline of IAs and whether they help with public acceptance of PTAs. Scholars might also further investigate whether the content of PTAs evaluated in the IA might explain the probability of their being ratified.

To further increase trust in ex-ante evaluations, one could include ex-post evaluations to show whether the trade agreements' spillovers were correctly estimated in the ex-ante analysis. We leave to further research to include unofficial estimations and to expand the scope of countries' perspectives.