Measuring Peace from the Bottom Up with the Pasto Indigenous Group in Nariño, Colombia

In the early 1990s, a revolutionary movement emerged in the international development research community using collaborative methods and bottom-up approaches to not only create knowledge through qualitative inquiry but also to define criteria and indicators to quantitatively measure key concepts, program results, and impact (Holland 2013, 1). Proponents argued that collaborative methods often provide a more accurate and complete picture of the reality on the ground. In addition, including local people in the process of designing measurement tools allows their perceptions and priorities to be communicated to policy makers even in a quantitative format (Chambers 2010; Firchow 2018; Holland 2013). Thus, international development researchers have used quantitative collaborative methods to develop participatory statistics—sometimes called participatory numbers—to design, monitor, and assess the impact of projects, programs, and policies (Gaillard et al. 2016). This article explores the use of one collaborative methodology applicable to the field of political science, with particular emphasis on scaling it for use at higher levels of analysis.

The Everyday Peace Indicators (EPI) methodology forms part of the participatory statistics toolbox. It was developed in 2012 by Pamina Firchow and Roger Mac Ginty as a result of dissatisfaction with traditional approaches to measuring peace-related phenomena. The EPI project pioneered a methodology to source indicators of peace at the community level using focus-group discussions and indicator-verification community meetings to develop quantitative data, or participatory statistics, using collaborative methods (Firchow 2018; Firchow and Mac Ginty 2017; 2020). By generating bottom-up indicators of peace-related concepts, the EPI project illustrated that localized perceptions of peace are not only articulated in different ways from top-down narratives and conceptualizations but also raise different issues. In contrast to top-down indicators that measure peace on a country level, the EPI methodology allows for an analysis of perceptions of peace in different villages and neighborhoods in conflict-affected contexts. Moreover, EPI indicators also can be used to collect longitudinal, individual-level quantitative data that can be used to track changes in perceptions of peace at the local level. Our research team has had tremendous success utilizing these locally sourced indicators to measure peace and related concepts in a wide variety of contexts. For example, our work in Afghanistan shed light on potential openings for women’s-rights issues in rural areas of Eastern Afghanistan (Firchow and Urwin 2020). Our research also has demonstrated how communities of reintegrated fighters in Colombia can contribute to transitional justice by bringing together human rights and peacebuilding initiatives (Dixon and Firchow 2020). In addition, international organizations including the US Institute for Peace, the US Agency for International Development, and the Inter-American Foundation are using EPI in practical applications to guide and evaluate their local-level programming in war-affected contexts such as Colombia and Sri Lanka.

However, the very nature of EPI, which allows for such a detailed localized picture, prevents this methodology from illuminating the wider context. When researchers are focused on community-level peace, this is not problematic. Indeed, EPI’s context specificity is a demonstrated strength of this methodology. Yet, many research questions in political science entail regional or national comparisons and therefore require a higher level of analysis. In these circumstances, EPI’s strength becomes a limitation. Indeed, EPI is so context specific that we are careful to use indicators in the community only in which they were sourced. In this project, however, we seek to build on our prior successes by determining whether the EPI methodology can be scaled up such that a regional measure of peace might be built from bottom-up indicators (Duursma, Firchow, and Levy 2020). If successful, this project will render data gathered using bottom-up indicators more comparable to existing top-down indices and barometers and also will create bottom-up measures that can be compared across contexts and groups.

In this project, however, we seek to build on our prior successes by determining whether the EPI methodology can be scaled up such that a regional measure of peace might be built from bottom-up indicators.

This article describes a pilot project that uses this collaborative methodology in an effort to produce a bottom-up barometer of peace for the Pasto Indigenous group in Nariño and Putumayo, Colombia. We initiated this project as a collaboration with representatives from the Coordinación Nacional de Mujeres Indígenas de Colombia (CONAMIC), an indigenous women’s activist group in Colombia. At the same time as we were interested in exploring how our community-focused methodology might be scaled up, members of CONAMIC were dissatisfied with what they perceived as top-down processes of measuring the impact of the 2016 Colombian peace accord in Indigenous communities, and they were looking for alternative measurement approaches that would be more inclusive. ¹ In 2018, we were able to come together to begin piloting a way to scale up EPI with the Pasto group, which represents one of the member groups of the 10 different indigenous groups that form part of CONAMIC. ²

The project is currently ongoing but the third phase of the work has been delayed because of COVID-19 restrictions; therefore, we are not yet able to reach firm conclusions about the scalability of EPI. However, we already have learned much about the scaling process and can assess what it will take to successfully scale up locally sourced, everyday indicators. Here, we discuss the complexity involved in scaling up locally sourced indicators from five different Pasto resguardos (i.e., indigenous reservations) for use in measuring peace in all 32 of the Pasto resguardos in Colombia. We explain our community selection and provide an overview of the EPI methodology. We then describe our use of two criteria to narrow the 965 indicators we sourced to an initial selection of 40 representative indicators. We also discuss how we balanced the criteria in our selection process. We conclude with next steps and the larger implications of this project for participatory statistics and scaling everyday indicators.

METHODOLOGY AND RESEARCH DESIGN

With the goal of sourcing indicators that are applicable to the widest range of communities, we began by selecting five resguardos in the Putumayo and Nariño provinces according to a “most different” design. Our selection was designed to maximize variation on a set of key contextual variables, such as levels of violence during the conflict and ethnic diversity. When possible, we used a stratified random-sampling scheme to select the resguardos, but our community selection was also partially purposive as well as dictated by issues of accessibility. Among other considerations, ongoing armed conflict made some of the resguardos too dangerous for conducting research. In addition, our implementing partners requested to begin in the resguardo in which they had the deepest ties to the community leaders. To the purposive selection of the requested resguardo, we added four more resguardos. We stratified the list by the Pastos’ own geographical understanding, which divides the Amazonian landscape in which they live into five different areas, including four groups of resguardos in the Nariño Department and another in Putumayo. These zones correspond to differences in economic, political, social, and conflict-history characteristics. Finally, we used a combination of random and purposive sampling within the geographic zones to ensure that we had variation on our key community-level variables. Table 1 shows the variation on key contextual variables across our five selected resguardos, and figure 1 displays their geographic location.

Table 1 Community-Level Characteristics of Selected Resguardos

Notes: Information in this table was sourced primarily from Pasto Indigenous leaders in Nariño. Demographic data for all resguardos other than Gran Tescual are from the 2018 Censo Nacional de Población y Vivienda (CNPV) (Government of Colombia 2018). Demographic data for Gran Tescual were provided by Pasto Indigenous leaders.

* This resguardo is not officially listed in the Colombian Census; it also is referred to as “Puerres.”

** The CNPV lists Orito Liberia as “Alto Orito.”

*** We report municipality-level data here because indicators were gathered with Pasto participants living in resguardos as well as the surrounding municipality.

**** Levels of violence related to human rights, crime, and armed conflict: low=minimal current or historical violence; moderate=some current or historical violence; high=sustained levels of violence over time.

Figure 1 Map of Selected Resguardos

We conducted a standard EPI process to collect indicators in the five selected resguardos. In the first stage of data collection, this consisted of community members generating indicators through focus-group discussions. Our research team led three indicator-generating focus groups in each resguardo (i.e., one each with men, women, and youth). For each focus group, the researchers followed a strict set of criteria to select a diverse and representative set of 10 to 15 residents of the resguardo who self-identified as Pasto. In these focus groups, community members discussed the signs they use and look for in their communities to judge whether they are more or less at peace, with questions including “What signs indicate that your community is more or less at peace?” and “What signs indicate there is a lack of peace?” Essentially, we tried to glean specific, locally contextual indicators that people already are using in their daily lives to measure their own everyday peace rather than gather indicators related to a particular activity or project. In this project, we worked with researchers from a Colombian non-governmental organization, Corporación de Investigación y Acción Social y Ecónomica, as lead facilitators and partnered with a local Pasto Indigenous facilitator. Our lead facilitators worked with our local facilitators, who are members of the Pasto Indigenous group, and they helped to guide us with contextual knowledge to extract a long list of indicators from the transcripts and notes of the focus-group discussions.

In the second stage, we engaged in a two-step indicator-verification process wherein community members vetted and culled the long list of indicators. The field research team presented the list to representatives of the original focus groups during a verification group. In this group, the participants reassessed their lists and eliminated or added indicators. Next, the facilitators invited the community at large to join the original focus-group participants in a larger indicator-verification meeting in which each participant cast 15 votes for their top EPIs in their community. The lists presented to the wider community in the five resguardos ranged in length from 147 to 210 indicators. This final exercise was a rigorous vetting that resulted in a list of indicators along with the number of votes that each indicator received. We then transformed these votes into importance scores, which adjust for the total votes cast in the resguardo and the total number of indicators in the resguardo’s long list.

In the standard EPI process, a short list of indicators that are most representative of a community then is used to conduct longitudinal surveys of peace in the community. ³ Building the short list is considerably more complex in this scaling-up project because we are aiming to build a barometer of peace that can be used to conduct longitudinal surveys of peace throughout all of the Pasto resguardos. The following section discusses our approach to scaling up to this higher level of analysis.

SCALING UP

The first two stages of the EPI process produced a total of 965 indicators from the five selected resguardos. It would be impossible to collect longitudinal data on all 965 indicators; therefore, to build a barometer of peace, we need to select and then test the applicability of a subset of indicators throughout the 32 Pasto resguardos. To date, we have completed the first step in this process, which involved carefully selecting for further evaluation a subset of potential indicators from those sourced in the five selected resguardos. The following discussion focuses on how we balanced two criteria for indicator selection: importance and commonality. When we are able to return to the field, we will survey individuals throughout all of the resguardos to test the applicability of our selected indicators and to further narrow our selection of indicators.

Because EPI invites those we study into the conceptualization process, it allows us to learn much about what peace means to the very people who live it. As is generally the case with EPI, we found tremendous variation in the types of indicators that emerged. For example, some participants specifically referenced violence (e.g., “There aren’t people killed in the resguardo”) whereas others referenced indirect signs of community harmony (e.g., “There is no garbage in the street of the town”). Faced with hundreds of bottom-up indicators and the task of narrowing them down to form a barometer of peace, we needed to evaluate how well each indicator captures these highly varied local understandings of peace. When we use EPI to construct a survey to gauge peace over time in a single community, our primary criterion for selecting indicators is the importance that the community places on each indicator. Yet, the indicators chosen by the importance score alone often are highly context specific. For example, in one community, the single highest vote-getter was “The pipeline is not bombed.” This indicator might be an excellent measure of peace in those communities with nearby pipelines; however, in communities located far from pipelines, this indicator would not be relevant. In building our bottom-up barometer of peace, therefore, we considered a second criterion: commonality across disparate communities. To assess commonality, we carefully analyzed the indicators in each of the five communities to determine which themes they have in common.

Because EPI invites those we study into the conceptualization process, it allows us to learn much about what peace means to the very people who live it.

To identify the common themes across resguardos, we undertook a two-stage coding process. First, we grouped the indicators into coded categories. Working from the indicator list from each resguardo, a team of four coders classified the indicators that received at least one vote using a preestablished codebook of indicator categories. The categories ranged from “everyday conflicts,” which captured indicators related to conflicts and problems at the smallest scales of interaction (e.g., school, family, and neighborhood) that are not necessarily tied to armed conflict; to “tranquility and security,” which captured indicators that refer to a general sense that the local context is positive, calm, or safe; to “local power,” which captured indicators that refer to the practices and institutions related to the exercise of power at the local level. Ultimately, each indicator was assigned a maximum of two categories. ⁴

We then calculated the importance scores for each community, omitting categories that had low levels of importance across the five communities. We identified 16 groups of 30 to 100 similarly categorized indicators for thematic analysis. Because many indicators were initially assigned two categories, most were assessed twice in this thematic analysis. At this stage of the process, we coded indicators from all five resguardos using an open-coding scheme, looking for themes that were present in at least three of them. Ultimately, we identified 58 common themes and selected a single indicator that best represented each.

Importance and commonality are not mutually exclusive. Therefore, we strove to select themes and representative indicators that met both criteria. Our final narrowing process from the 58 representative indicators to the 40 indicators we ultimately selected considered the importance that the five communities placed on each category. Table 2 lists the importance scores and balance of selected indicators across the original coded categories. The community figures are presented as percentages because they reflect the proportion of the total importance that each community places on the given category. ⁵ The final column displays the proportion of the selected indicators that fall into each category.

Table 2 Relative Importance of EPI Categories by Community and in Selected Indicators

Note: Some categories are omitted due to space constraints. Columns do not total 100% due to rounding.

As shown in table 2, there are community-level differences in the importance of particular indicator categories; however, our selected indicators reflect the overall level of importance of categories across the five resguardos. Although this shows that community priorities and indicators of peace are different even within the same indigenous group, we were able to identify themes that were present in multiple resguardos and to select representative indicators for those common themes. Our intuition is that if similar indicators were generated spontaneously in multiple resguardos with different characteristics, then they might be widely applicable throughout the 32 Pasto resguardos. Once research can resume, we will test this intuition in the next stage of the work by surveying all accessible resguardos throughout the Pasto territories. Our survey will allow us to test the applicability of the selected indicators and to select a final group of 20 indicators that are common across the various contexts throughout the Pasto people. This final selection of the most widely applicable indicators will form our bottom-up barometer of peace for the Pasto indigenous group, which will be used to collect longitudinal quantitative data that will allow inter-community and over-time comparisons.

Although this shows that community priorities and indicators of peace are different even within the same indigenous group, we were able to identify themes that were present in multiple resguardos and to select representative indicators for those common themes.

CONCLUSION

We recognize that there are limitations to our efforts to scale up. For example, the indicators that we eventually include in the barometer necessarily will be limited by the commonalities across the communities. For each community, we necessarily will be omitting highly context-specific indicators that might be of great importance in assessing everyday peace for that community. Even with all of these challenges, however, we believe that it is valuable to use this bottom-up process to create a barometer of peace for the Pasto. Doing so allows us to build a barometer that is based on localized conceptualizations of peace and can be used longitudinally in multiple Pasto resguardos. This will be a major improvement over a measurement approach imposed from the outside. Nevertheless, our initial analysis reveals that by selecting those aspects of peace that all of the communities share, it is necessary to omit specific contextual exceptions for each resguardo. This demonstrates the impossibility of producing a truly localized and highly contextualized measure of peace at a higher level of analysis. By producing a universally appropriate measure, we necessarily lose detail. Moreover, if our approach were to be extended further to the macro level, this loss of detail would be exacerbated. Ultimately, the more we scale up, the more detail we lose—even when we generate indicators from the bottom up. For these reasons, we urge researchers to consider carefully the level of analysis at which they work. We recommend that researchers who are interested in measuring change at the community level and who are able to work collaboratively with their research subjects use indicators generated in the community to prioritize the local view. However, if researchers are asking questions at a meso or macro level, we hope that our work can guide the development of bottom-up measurement that is appropriate for higher levels of analysis.