Are active labour market policies effective for the older unemployed? A meta-evaluation

Abstract

We present a meta-evaluation of the literature on the impacts of active labour market policies for unemployed people over 50, extracting 82 impacts for analysis. The meta-evaluation includes only impact evaluations that examine both a group of beneficiaries and a control group of comparable non-beneficiaries. On average, we find that active policies have a slightly negative effect (−0.8 percentage points) on the probability of unemployed people over 50 finding a job and that this negative effect disappears 24 months after policy implementation. However, this effect is very different when disaggregated by policy type. Direct job creation policies have a clear negative effect (−3.9 percentage points), and training policies have a positive average effect, either in isolation (2.4 percentage points) or when combined with search assistance or counselling (1.7 percentage points). We also find slight differences by gender, with the effect of active policies being greater for women than for men. These results have important implications, given that the European Commission and the Organisation for Economic Co-operation and Development have made active policies the cornerstone of their efforts to improve the re-employment of older people. Our results support training policies, either in isolation or in combination with search assistance and counselling. The greatest impacts are obtained after 12 months of policy implementation.

Introduction

In this article, we analyse the effectiveness of active labour market policies for unemployed people over 50 and seek to establish how the effects differ according to policy type. Active policies for the older unemployed have become very important, as such individuals are at a distinct disadvantage when seeking a job (Börsch-Supan et al., 2021). As Wuebekke (2011) points out, the older unemployed rarely leave the labour market due to a low propensity to work; rather, they are forced out due to the impossibility of meeting the demands of available jobs, a lack of employment opportunities or the absence of specific support from public employment offices. Therefore, it is extremely important to understand whether active policies intended to address these problems are actually effective.

Although many studies evaluate active policies aimed at the older unemployed, we still lack an integrated vision of the results generated by this literature. Studies that consider all unemployed people – regardless of age – show that active policies have a positive effect, with the effect of training courses being greater than that of direct job creation (Card et al., 2010, 2017; Vooren et al., 2018). These studies provide evidence that active policies have a greater effect two or three years after their implementation. There are several studies on the older unemployed that generally analyse a single active policy (Caliendo et al., 2008; Arni, 2010, 2015; De Groot and Van der Klaauw, 2016). This literature finds heterogeneous effects of active labour market policies for unemployed people over 50 (Caliendo et al., 2008; Bollens, 2011). In some cases, the effect is not statistically significant (Cavaco et al., 2013). However, to the best of our knowledge, there is no systematic review of the literature analysing the weighted average effect of these policies or verifying the effectiveness of the various policies considered.

We fill this gap by conducting a meta-evaluation of studies on the effectiveness of active labour market policies for unemployed people over 50. This methodology allows us to estimate the weighted average effect of active policies, which is a novel contribution to the literature on this subset of the unemployed. The database we use was compiled from a systematic review of all available impact evaluations on active policies for unemployed individuals over 50. For the systematic review, impact evaluations meeting certain methodological criteria were selected. Specifically, we considered studies that compared beneficiaries with a control group based on an experimental or quasi-experimental approach. Subsequently, the effects were selected and classified according to the methodology, policy type, time elapsed since implementation and gender. Policy implications of the findings are discussed.

Literature review

Unemployment affects each subset of the population in different ways. For people over 50, unemployment presents special challenges due to the many limitations on labour market re-entry possibilities (Heyma et al., 2014). We must take into account the fact that finding a job becomes less likely the older the unemployed individual is (Heyma et al., 2014). It is little wonder, then, that age is considered a risk factor for entering long-term unemployment (Boockmann and Brandle, 2015). For this reason, countries in Europe have given more priority to the problems faced by unemployed individuals over the age of 50 (Organisation for Economic Co-operation and Development (OECD), 2017).

Among the measures implemented in developed economies, active policies have gained prominence in recent decades as a result of the development of so-called ‘activation strategies’ (Malo, 2018). These strategies, which have been strongly promoted by the European Commission and the OECD, are mainly aimed at accelerating the transition from unemployment to employment (Martin, 2015). Active labour market policies are the cornerstone of these strategies; they are defined as policies to facilitate the re-entry of the unemployed into the labour market and/or improve such individuals’ career outlooks. Following the OECD criteria (Malo, 2018), the following are considered active policies: labour market services, labour market training, employment incentives, sheltered and supported employment and rehabilitation, direct job creation and start-up incentives. For people over 50, the most important policies relate to job search assistance, training courses, subsidies on private-sector employment and direct job creation in the public sector.

Many studies have examined the impact of active policies on employment outcomes, such as the probability of employment in a given time horizon and the quality of job matches (Malo, 2018). A meta-evaluation is a powerful tool that allows us to draw general conclusions about the impact of active policies from many studies at the same time (Kluve, 2010). As a study design, meta-analysis was originally used in the health sciences to obtain empirical evidence on the effect of certain medical interventions studied by different research teams. Since the 1970s, the use of meta-analysis in the health field has grown exponentially (Haidich, 2010) due to the advantages of this methodology. This methodology has been extended to other areas of knowledge, allowing researchers to synthesise impacts estimated in different studies. In the field of active labour market policies, we highlight the pioneering meta-evaluations of Kluve (2010), Card et al. (2010, 2017) and Vooren et al. (2018).

These meta-evaluations find that active labour market policies have a positive effect on unemployed people's probability of finding a job (Kluve, 2010; Card et al., 2017). In general, these effects become stronger in the longer term, i.e. a time horizon of two to three years (Lechner et al., 2011; Card et al., 2017). At the same time, the effects of active policies display great heterogeneity depending on the population under study (Hujer et al., 2004; Card et al., 2017). Regarding gender, there is evidence that active policies have a greater effect on women's probability of finding a job relative to men (Card et al., 2017).

There are reasons to think that the impact of these policies will also display unique features when focusing on the older unemployed population. The older unemployed – i.e. those over 50 years of age – are relatively close to the retirement age. In many countries, such individuals even have access to subsidies that continue until the designated retirement date. Despite that, these individuals show a strong propensity to seek re-employment, especially if they anticipate low pensions (Wuebbeke, 2011).

Several studies separately analyse incentives, training and direct job creation policies for the entire unemployed population. There is evidence that these policies have a positive effect in terms of reducing unemployment (Breunig et al., 2003; Stenberg and Westerlund, 2008; Cockx et al., 2013). Nevertheless, some studies show a negative effect on the probability of finding a job, especially in the case of direct job creation policies (Hujer et al., 2004).

As mentioned earlier, there has been no meta-evaluation focusing exclusively on unemployed people over 50. Some of the results of existing meta-evaluations can provide hints about policy choices, but a focused meta-evaluation on this sub-population is crucial. Choosing the most effective policy solutions is particularly important for unemployed people over 50, because ineffective policies may definitely expulse these workers from the labour market in the critical years leading up to retirement age.

Methodology

Main concepts

A meta-evaluation is an evaluation of evaluations – impact evaluations in this case. Impact evaluations compare the situation of individuals exposed to a policy intervention with the situation of those not exposed to it, ceteris paribus.

In impact evaluations, different techniques are used to measure the net effect of active policies on a treatment group relative to a control group with statistically similar characteristics. Such techniques include randomised controlled trials, matching, difference in differences, discontinuous regression and instrumental variables. Under certain assumptions, these techniques ensure that control and treatment groups (non-beneficiaries and beneficiaries, respectively) have similar characteristics, making it possible to measure the net effect or average treatment effect of a policy.

Our meta-evaluation allows us to aggregate and compare the findings of a series of impact evaluations on unemployed people over 50. To be included in the meta-evaluation, studies must meet two fundamental conditions. First, they must be conceptually comparable; that is, they must study the same policy for the same population – in this case, active policies for unemployed people over 50. Second, they must use similar statistical methods and present their results in a similar way (Lipsey and Wilson, 2000). In particular, they must be impact evaluations in the sense that active policy beneficiaries are compared with a group of non-beneficiaries with the same average characteristics.

In our meta-evaluation, the measure used to analyse the average effect of active policies is the probability of finding a job. The average effect size allows us to measure the aggregate effect of different active policies aimed at unemployed people over 50 according to different variables: type of policy, methodology, time elapsed since implementation and gender. The average effect size is calculated by computing the weight of each effect size by the inverse of the variance for each of the studies. Therefore, the average size of the weighted effect can be calculated using the following formula:

$$\overline {ES} = \displaystyle{{\mathop \sum \nolimits ( {\omega_iES_i} ) } \over {\mathop \sum \nolimits^ \omega _i}}, \, \rm where \,\omega = \displaystyle{1 \over {SE^2}}.$$

ES_i represents the average effect of each evaluation included in the analysis, and ω_i represents the inverse of the variance of each evaluation. Thus, the sample size is present for each of the studies, with SE being the standard error for each evaluation. The value of i ranges from 1 to k, where k represents the total number of impacts or effects obtained from all the evaluations included in the analysis. For interpretation purposes, we present our results using so-called ‘forest plots’, which are simply error-bar charts with effect sizes and 95 per cent confidence intervals from the different studies (their name comes from their vague resemblance to trees).

We also run a meta-regression to analyse the relationship between study characteristics and the effect sizes of active policies. This is done to check whether and how heterogeneous study characteristics affect the size of the effects estimated in the evaluations (Thompson and Higgins, 2002).

Systematic selection of studies

We conducted our selection of studies in accordance with widely adopted research guidelines, especially in the analysis of active labour market policies (Card et al., 2010, 2017; Kluve, 2010). We started by selecting all impact evaluations aimed at the older unemployed and then reviewed them to obtain only comparable evaluations for analysis.

For the selection of impact evaluations, we consulted up to six data sources. The first source was the Centre for Research on Impact Evaluation (CRIE), a repository created by the European Commission that seeks to improve the evaluation of policies by promoting the use of counterfactuals in analysis. This repository stores all impact evaluations that meet a set of standards and requirements to ensure the quality of the studies. A search was carried out in this database using certain key words (e.g. labour market and older unemployed) to narrow the focus to impact evaluations aimed at unemployed people over 50. We also reviewed impact evaluations targeting the entire unemployed population to check for any subgroup-level analyses focusing on the older unemployed. This first review yielded four studies on unemployed people over 50 years of age.

Our second source was the work published by Boockmann and Brandle (2015). This article reviews a series of evaluations focused exclusively on the older unemployed. Our selection criterion was the same: studies using impact evaluation techniques to examine unemployment for people over 50. Our review of the bibliography of Boockmann and Brandle (2015) yielded a total of ten possible impact evaluations to include in our analysis.

Our third source was Google Scholar. The search criteria remained the same: the keywords were older unemployed, active labour market policies and impact evaluations. This search yielded two additional impact evaluations for potential inclusion in our analysis.

Our fourth and fifth sources were the Dialnet and Scopus platforms. However, our search of these two sources yielded no additional impact evaluations concerning active policies for the older unemployed beyond those already found.

Finally, as our sixth source of impact evaluations, we exhaustively checked the bibliographies of the studies already selected in the five previous sources. The search criteria remained the same in that we focused only on impact evaluations of active policies aimed at unemployed people over 50.

After completing our search of the above sources, we had a total of 20 studies available to include in our analysis. However, after a detailed examination, we determined that some of the studies did not meet certain requirements for inclusion in the meta-evaluation (see Table A1 in the Appendix).

First, we excluded Baumgartner and Caliendo (2008), Caliendo and Kunn (2011) and Cahuc et al. (2019). Despite being impact evaluations with rigorous methodologies, these evaluations do not study the net effect of policies on unemployed people over 50 but rather focus on the entire unemployed population.

Second, we excluded Heyma et al. (2014), Koning and Raterink (2013) and Hullegie and Van Ours (2014), which had been extracted from the bibliography of Boockmann and Brandle (2015). These three studies were excluded for methodological reasons: although they use rigorous analysis techniques, they are not impact evaluations.

Third, Bernhard et al. (2008) was ruled out because this study examines the effect of a subsidy programme on unemployed individuals between the ages of 35 and 49, which is not our group of interest. Boockmann et al. (2012), Centeno et al. (2009) and Heinrich et al. (2013) were also excluded from the analysis because they estimate the effect of active policies in terms of the level of income or time spent in unemployment and not in terms of the transition into employment.

Fourth, we discarded two studies due to a lack of information necessary for our meta-evaluation. Boockmann and Brandle (2018), despite fulfilling all desired characteristics, does not provide information on standard errors. Romeu-Gordo and Wolff (2011) was excluded for the same reason.

Furthermore, Huttunen et al. (2013) was excluded because the individuals in the control group are of a different age than the individuals in the treatment group. Lastly, Lammers et al. (2013) was discarded because the results are only available disaggregated by gender.

Following the review, the database for this investigation consisted of 82 evaluated impacts extracted from six studies that fulfilled the requirements for inclusion. The review was minutely carried out, extracting information on the average estimated effects of active policies, the standard errors, the sample sizes, the methodologies used, the country and the time elapsed since policy implementation. In short, we included impact evaluations that have control groups, focus only on unemployed people over 50, provide all information detailed above and measure impacts in terms of the probability of accessing a job. There is not a standard minimum of studies that should be considered in meta-analysis, but the small number of studies available about older workers leads to not much heterogeneity in the methodology within each country considered. The number of evaluated impacts is large enough to carry out a meta-evaluation but results on country and methodology should be taken carefully and as provisional.

As shown in Table 1, our database covers a wide variety of methodological techniques. The studies we included use difference in differences, matching, randomised controlled trial and instrumental variable approaches. In addition, they provide data on the effects of active policies until 36 months after implementation. Regarding countries, bear in mind that there are different estimations by region in the case of Germany (East versus West), following the pattern of some previous studies (Caliendo et al., 2008). Although in almost all the studies included in the meta-analysis both institutional context (country) and methodology differ, most countries analysed (Austria, Belgium, Germany, Switzerland) share many important institutional features (Esping-Andersen 1990; Sapir, 2006; Card et al., 2010). As Card et al. (2010) point out, there might be differences in the design of active labour market policies between countries. These differences cannot be analysed herein because of the small sample of studies considered.

Table 1. Characteristics of the evaluations included in the database

Source: Authors’ calculations.

Analysis

Descriptive analysis

Table 2 presents the raw information on the average treatment effects collected from the selected studies. There are some striking differences when analysing the obtained effects according to policy type. Training and training jointly provided with counselling and job search assistance show a positive average effect of 2.6 and 2 percentage points on the probability of finding a job for unemployed people over 50, respectively. However, for direct job creation policies, the average effect obtained is negative, reaching −3.2 percentage points, and for training jointly with only counselling the average effect is extremely negative, −5.9 percentage points, although this average effect is the result of only four impacts.

Table 2. Average effects on the probability of finding a job for the older unemployed according to impact measurement variables

Notes: Total number of observations = 82. ATE: average treatment effect.

Source: Authors’ calculations.

Table 2 also reveals interesting patterns when analysing the effects of active policies according to the time horizon. The effect is practically null up to the first 12 months and then turns negative as the time horizon increases to 24 months. However, between 24 and 36 months after policy implementation, we observe a positive impact of 2.3 percentage points on the probability of finding a job for unemployed people over 50. This evidence seems to be consistent with the results of other studies, which find a greater effect of active policies in the long term (Card et al., 2017).

In terms of the methodological approach, Table 2 shows important differences between matching and difference in differences estimations. The evaluations using matching to calculate the impact of active policies show a negative average effect of 1.5 percentage points on the probability of finding a job. In contrast, the difference in differences and instrumental variable methods show positive average effects of 1.9 and 2.7 percentage points, respectively.

Table 2 also reveals slight differences in the estimated impact of active policies for unemployed people over 50 according to gender. Specifically, the estimated average effect is lower for men than for women. This result is consistent with the literature on the entire unemployed population, regardless of age (Card et al., 2017).

Meta-evaluation

Now, we present the weighted average effect obtained for the whole set of active policies, which is −0.8 percentage points (see Table 3, last row). This effect is between −1 and −0.5 percentage points with a 95 per cent confidence interval. Therefore, we find a negative average effect on the probability of finding a job for unemployed people over 50. A hasty interpretation that may be drawn from this result is that active policies for the older unemployed are ineffective and even detrimental. However, we observe an important heterogeneity across different active policies. Whereas two versions of training policies show a positive impact on the probability of finding a job (2.4 and 1.7 percentage points), direct job creation policies show a negative impact (−3.9 percentage points). For training policies combined exclusively with counselling, we found an effect of 0.1 percentage points but this effect is not statistically significant different from zero.

Table 3. Weighted average effect (WAE) of active policies on the probability of finding a job for the older unemployed according to methodology, type of policy and time

Note: Calculations for the lower and upper limits correspond to the 95 per cent confidence interval.

Source: Authors’ calculations.

Table 3 also presents data obtained for the set of active policies according to the methodology used. Impact evaluations using the difference in differences method show a positive effect of 1.6 percentage points on the probability of finding a job for the older unemployed. In contrast, evaluations using matching show a negative effect of 3.6 percentage points on this probability.

As can be seen in Figure 1, for impacts obtained using the difference in differences method, the deviation is very small. Therefore, we can affirm that impact evaluations using this methodology show a positive impact of active policies.

Figure 1. Forest plot of the estimated impacts and average treatment effect of the difference in differences method.

Notes: ES: effect size. CI: confidence interval.

Source: Authors’ calculations.

The weighted average effect obtained for the matching method is −3.6 percentage points, as can be observed in Figure 2. Again, the effect deviation is very small. Thus, we can affirm that impact evaluations using this method show a negative estimated effect of active policies. Therefore, we find important differences between the difference in differences and matching methods.

Figure 2. Forest plot of the estimated impacts and average treatment effect of the matching method.

Notes: ES: effect size. CI: confidence interval.

Source: Authors’ calculations.

The results for training policies are very encouraging (Figure 3), but those for direct job creation policies are in the opposite direction. As shown in Figure 4, the weighted average effect of direct job creation policies is −3.9 percentage points. Taking into account the fact that this type of policy is mainly aimed at unemployed people with unique problems re-entering the labour market, the results obtained regarding the effectiveness of these policies for people over 50 are not very encouraging. However, some studies show large differences in the effects of these policies according to the characteristics of unemployed individuals (Caliendo et al., 2008).

Figure 3. Forest plot of the estimated impacts and average treatment effect of training, counselling and search assistance policies.

Notes: ES: effect size. CI: confidence interval.

Source: Authors’ calculations.

Figure 4. Forest plot of the estimated impacts and average treatment effect for direct job creation policies.

Notes: ES: effect size. CI: confidence interval.

Source: Authors’ calculations.

Finally, there are some differences when analysing the effect of active policies according to the time horizon. As shown in Table 3, the results are slightly different from those obtained in the descriptive analysis. Although a negative effect is still observed in the ranges of 7–12 months (−2.3 percentage points) and 13–24 months (−1.4 percentage points), there is no longer a positive impact in the ranges of 0–6 and 25–36 months. For these two ranges, we cannot distinguish the average effect from zero; although the value of the weighted average effect is negative, the range obtained by the effect deviation exceeds zero.

The above evidence indicates that the effect of active policies on unemployed people over 50 displays a changing trend as the time horizon lengthens. The effect first decreases from null to negative until the lowest estimation for 7–12 months and then increases becoming null for the range of 25–36 months. Despite this, several positive effects are observed, which suggests that effects become greater as time increases. This is particularly true four or five years after policy implementation.

Next, we turn our attention to differences by gender. The results of the meta-evaluation show that active policies have a negative impact on women's probability of finding a job, with an effect size of −1.8 percentage points. For men, the negative impact is greater, at −3.2 percentage points.

Apart from the evaluations included in the database, we also consider other evaluations that, despite being excluded from the main analysis due to a lack of data, provide interesting information to interpret the above results. For training policies, impact evaluations that use both the difference in differences and matching methods (Romeu-Gordo and Wolff, 2011; Boockmann and Brandle, 2018) show similar effects to those obtained in our analysis, i.e. positive effects. Regarding wage subsidy policies, some impact evaluations show evidence of positive impacts (Huttunen et al., 2013), but there is also conflicting evidence showing a negative effect (Boockmann et al., 2012).

It is also interesting to compare our meta-evaluation of studies on the older unemployed with other meta-evaluations that consider the entire unemployed population. Among these meta-evaluations, the one conducted by Card et al. (2017) stands out as a benchmark work in the field of active labour market policies. Furthermore, given that we followed the same analysis guidelines for our meta-evaluation, we can compare results between our study (unemployed people over 50) and theirs (all unemployed).

First, we focus on differences in the effects of active policies according to the methodology used in impact evaluations. For two of the most commonly used methods, i.e. matching and difference in differences, we find a difference of 5 percentage points. A potential problem when using the matching method is that non-observable variables that can have a considerable influence on the population under study are not considered. In some areas of knowledge, such as economics, matching is done in a rigorous step-by-step fashion, especially propensity score matching, due to the importance of non-observable variables when making inferences (Cunningham, 2021: 561–572).

Although any technique that falls under the matching method faces non-observable variable issues, we must remember that this method allows evaluation in cases where other methods are not possible. In addition, certain techniques, such as exact matching, reduce the dependency and bias of the model while providing for simpler analysis (Iacus et al., 2012).

Given the potential impact of non-observable variables, the difference in differences method is often chosen because it reduces selection bias. However, this method also has certain limitations. For example, one of the assumptions on which it is based, namely the parallel trends assumption, requires that policy beneficiaries and non-beneficiaries display similar trends in the period before the policy intervention; this assumption has been criticised because it is not always true (Kahn Lang et al., 2018). However, in most cases, problems arise not from the technique chosen but from data quality or research design issues. These differences by methodology might also be affected by differences in the policy design of each country (Card et al., 2010). Although policy design may differ leading to differences in the results of the evaluation, in our case we do not have enough heterogeneity to obtain conclusions about the effect of policy design.

Second, the results obtained according to policy type show that direct job creation policies are less effective than training policies in terms of increasing the probability of finding a job for unemployed people over 50. Our results are in line with those of other meta-evaluations considering the entire unemployed population, where policies emphasising the accumulation of human capital are found to have a greater effect than direct job creation policies (Heckman et al., 1999; Card et al., 2010, 2017).

However, although we observe a smaller or negative effect of direct job creation policies, the positive aspects of these policies should be kept in mind. They are designed to help individuals who have been unable to enter the labour market to obtain a job, and in some cases they spring from a social equity motivation to help the disadvantaged (Auer et al., 2008). In addition, some studies show evidence of a positive impact of direct job creation policies on other labour market outcomes, such as earnings (Jespersen et al., 2008).

For unemployed people over 50 exposed to training policies, our meta-evaluation results show a positive impact of 1.7 percentage points on the probability of finding a job. Although this value may seem small, it actually represents an important effect for this subset of the unemployed, who face great disadvantages in the labour market. Furthermore, the analysis shows that the training policies with the greatest effect are comprehensive ones that also incorporate search assistance and counselling.

Third, regarding the time horizon, the effects of active policies show an interesting pattern: the effects are negative in the short term but then increase 25–36 months after policy implementation. The literature reveals a similar trend for the entire unemployed population (Card et al., 2010, 2017; Vooren et al., 2018). As discussed earlier, the timing of effects is crucial for the older unemployed, who are near the end of their working lives. Therefore, the fact that positive impacts are observed only after 25–36 months raises the question of how profitable these policies are for them. However, considering that, behind the non-significant effects we have not null impacts but a mix of negative and positive impacts. This lack of precision may be related to the crucial role of policy implementation to obtain a positive impact (Malo and Cueto, 2017; Malo, 2018).

Finally, with regard to gender, results obtained for unemployed people over 50 show a greater negative effect of active policies for men than for women. Card et al. (2017) similarly observe a greater impact of active policies on women. Furthermore, in some cases, we find a strong dispersion of effects. Together, these results suggest that it is essential to maximise the effects of active policies by focusing adequately on a target population, as recommended by Malo and Cueto (2017).

Meta-regression

The meta-regression is an ordinary least squares regression that sheds light on the differences and relationships among the variables included in the analysis and the estimated effects of the active policies.

The dependent variable is the estimated effect of the active policies, and the independent variables (all categorical) are policy type, methodology, time horizon, gender and country. However, the country variable is not included in the model due to collinearity problems. These problems may arise due to the small sample of studies and because there is not much heterogeneity in terms of country and methodology. The following categories are taken as a reference for each variable: direct job creation policies for policy type, difference in differences for methodology, men for gender and 0–6 months for time horizon. The methodology was defined as a dummy variable that takes a value of 1 for the instrumental variable method and 0 for the rest. Because of collinearity problems in estimations, we cannot introduce a variable on methodology with more categories. Again, these collinearity problems may arise due to the small number of studies that could be considered in the analysis.¹

As shown in Table 4, we obtain significant differences for policy type, gender and time horizon. The estimated effects are greater for training policies, with particularly large effects for more comprehensive training policies that incorporate search assistance and counselling. This is in line with Malo (2018), who states that international evidence shows that active policy packages are more effective than isolated policies.

Table 4. Meta-regression on the size of the policy impacts

Notes: Total number of observations = 82. Adjusted R ² = 87.54 per cent. Ref.: reference category.

Source: Authors’ calculations.

The estimated effect becomes greater as the time horizon lengthens to the range of 25–36 months after policy implementation, and the estimated effect is greater for women than for men. The first result is in line with evidence shown in the meta-evaluation of Vooren et al. (2018), and the worse result for men is also in line with other meta-evaluations that consider all unemployed people (Card et al., 2017). All of this evidence is in line with the data obtained in the meta-evaluation, although differences observed according to the methodology used were not significant. This is important, as we are unable to affirm that the method chosen has any influence on the estimated effect.

As a robustness check, we also run meta-regressions by the type of active policy. The results obtained are consistent with those in the previous model and are available upon request.

Conclusions

The objective of this article was to analyse the effectiveness of active labour market policies for the older unemployed. To that end, a meta-evaluation of 82 impacts extracted from a set of comparable impact evaluations was carried out. The analysis took into account the type of policy, the methodology, the time elapsed since policy implementation and the gender of beneficiaries. Based on the analysis, we present the following conclusions regarding the effectiveness of active labour market policies for unemployed people over 50:

1. (1) Active policies have, on average, a negative effect of −0.8 percentage points on the probability of such individuals finding a job. The negative effect is observed from the moment of policy implementation until 24 months later. However, the weighted average effect of such policies is null 25–36 months after implementation, suggesting that the passage of time mitigates the negative effect.

2. (2) The above negative effect displays significant heterogeneity according to the type of active policy. For unemployed people over 50, training policies have a positive effect of 1.7 or 2.4 percentage points on the probability of finding a job, depending on joint implementation with search assistance and counselling or not, respectively. In contrast, direct job creation policies have a relatively large negative effect on their probability of finding a job.

3. (3) The meta-evaluation results reveal that the estimated effect on the probability of finding a job is greater when impact evaluations use the difference in differences method than when they use the matching method. In contrast, the meta-regression results reveal no statistically significant differences regarding the influence of the impact evaluations’ chosen methodologies. Therefore, we cannot affirm that methodology is a variable that influences the effects of the active policies analysed.

4. (4) There are slight gender differences with respect to active policies aimed at the older unemployed, with the weighted average effect being greater for women. Although there is evidence of positive impacts for both men and women, the average effect is negative for both groups.

These conclusions are in line with evidence from other meta-evaluations that consider the total unemployed, regardless of age (Card et al., 2010, 2017; Vooren et al., 2018). However, when analysing the effects of active policies, there are slight differences between the older unemployed and the total unemployed.

Our findings have several important policy implications with regard to the use of active policies aimed at unemployed people over 50. Although we cannot ignore the positive aspects of direct job creation policies, such as the help they provide for certain disadvantaged groups, it is clear that training policies are preferable if the objective of public institutions is to reduce unemployment rates for older people. Training policies show a positive effect, either in isolation or when combined with counselling and job search assistance. Furthermore, when implementing these policies and evaluating their cost-effectiveness, it is important to remember that the observed effects tend to be greater in the long term. Therefore, we can affirm that the position adopted by European countries in favour of these policies (Martin, 2015) is positive in terms of reducing unemployment rates. Due to limitations in comparability, we could not include evaluations of wage subsidy policies in our systematic review. We note, however, that other evaluations show a clear positive impact of this type of policy, especially when combined with counselling and/or the interaction with case workers (Bernhard et al., 2008; Huttunen et al., 2013).

Although our conclusions are based on a relatively small number of impact evaluations, our systematic review provides useful evidence about the effectiveness of active policies aimed at the older unemployed, something that has not been done before. The main limitation here is the low number of studies that could be considered, which is reflected, for example, in not much heterogeneity in the methodology within each country. However, considering the rising number of impact evaluations on active policies for the older unemployed, our research opens the door to future meta-evaluations that may include a larger number of observations and/or compare other policy types and impact variables that could not be analysed in this research due to a lack of systematic information or comparable data. Likewise, as future steps, it would be interesting to analyse the effect of the institutional context between countries.