The Accuracy of Identifying Constituencies with Geographic Assignment Within State Legislative Districts

Abstract

Identifying the geographic constituencies of representatives is among the most crucial, yet challenging, aspects of state and local politics research. Regularly changing district lines, incomplete data, and computational obstacles can present barriers to matching individuals to their respective districts. Geocoding residential addresses is the ideal method for matching purposes. However, cost constraints can limit its applicability for many researchers, leading to geographic assignment methods that use polygonal units, such as ZIP codes, to estimate constituency membership. In this study, we quantify the trade-offs between three geographic assignment matching methods – centroid, geographic overlap, and population overlap matching – on the assignment of individual voters to state legislative districts. We confirm that population overlap matching produces the highest accuracy in assigning voters to their state legislative districts when polygonal location data are all that is available. We validate this finding by improving model estimates of lobbying influence through a replication analysis of Bishop and Dudley (2017), “The Role of Constituency, Party, and Industry in Pennsylvania’s Act 13,” State Politics and Policy Quarterly 17 (2): 154–79. Our replication suggests that distinguishing between out-of-district and in-district donations reveals a greater impact for in-district lobbying efforts. We make evident that population overlap assignment can confidently be used to identify constituencies when precise location data is not available.

Introduction

The institutional design of representation in the US necessitates matching individuals to their respective legislative districts amidst an array of geographic boundaries that vary in both size and shape. Understanding a representative’s geographic constituency is central to American state and local politics research (Fenno 1978), from progressive ambition (Rohde 1979) to policy responsiveness and lobbying influence (Caughey and Warshaw 2018; Lewis 2013). However, the process of matching individuals to their representatives is a major constraint within the study of state and local politics; one that is further complicated by regularly changing district boundaries and the lack of perfectly nested sub-geographies inside politically relevant boundaries like legislative districts. The challenges arising from the need to match individual data points (like voters) to geographic units (like legislative districts) are experienced by numerous groups including both election administrators¹ and researchers.²

Although there have been significant advances in geocoding and database management to improve data sources like voter registration files (Amos and McDonald 2020; Ghitza and Gelman 2020), modern techniques can still be out of reach. If a researcher has access to precise location data like the full address of individuals – which is often not the case – the costs of locating those addresses inside a geographic unit can be prohibitively expensive in regard to time and money. Even with appropriately powerful software and hardware, large-scale geocoding of voter files, as employed by Amos and McDonald (2020), can take dozens of hours to complete.³ Pay-as-you-go geocoding tools like Google API can be financially demanding when the $5.00/1,000 addresses rate is applied to state voter files with tens of millions of registrants.⁴ Although universities can help bridge the resource gap needed to conduct large-scale spatial audits, less-resourced individuals may find the steps outlined by researchers like Amos and McDonald (2020) inaccessible. Fortunately, these costs can be overcome through geographic assignment matching – the process of assigning individuals to a higher level geography based on their inclusion in a nested lower level geography. Crucially for state politics scholars making use of legislative districts, there is no single geography that can be used to match individuals to districts. During the 2011 redistricting cycle, state lower and upper chambers split approximately 48% and 32% of the smallest unit of publicly known geographic units within the US – ZIP codes – respectively.⁵ Failing to account for these geographic nuances can lead to error, yet the costs involved in addressing these sources of error can be intimidating. To date, the trade-offs to geographic assignment matching over more computationally intensive methods are presently unclear.

In this study, we first quantify these trade-offs by testing three geographic assignment matching methods – centroid, geographic overlap, and population overlap matching – on the assignment of individual voters to their state legislative districts. In doing so, we confirm that population overlap matching produces the most accuracy in assigning voters to their legal state legislative districts when polygonal location data (e.g., ZIP codes) are all that is available.⁶ Additionally, so long as the effective number of districts within a lower level polygonal unit is under 1.3, population overlap matching can be used to locate individuals with confidence. We illustrate the applicability of geographic assignment in improving research by replicating and extending the effect of lobbying on legislator behavior by Bishop and Dudley (2017). We show that the burden associated with geocoding individual cases can be significantly reduced by first identifying which lower level geographies are not split between multiple higher levels (like state legislative districts). By better distinguishing between these areas, we discover that an average of 3% of the data is in question and requires use of a geographic assignment method for allocation. Due to the nature of the data, all three methods of controlling for district residency improve model estimates. Following these results, we conclude this study with a set of suggestions by which users can determine whether and how to implement geographic assignment methods within the US context.

The problem

Identifying the geographic constituency of an individual is a multi-stage problem. First, is there an existing data source that matches individuals to their legal representatives? Second, when such data is not present, do the necessary district boundaries in the form of geographic shapefiles or individual coordinates for constituents exist? Third, if coordinate data for individuals is not available, are the addresses in a format conducive to geocoding? And, finally, does the researcher have the means to pay the cost – in both time and money – to perform these computations?

Regarding these problems, the first issue tends to afflict any data that is not a state voter file or a proprietary equivalent. While there is much that can be done with voter files, as demonstrated by Ghitza and Gelman (2020) in improving upon multilevel regression with post-stratification, most research cannot use voter files – even when they are publicly available.⁷ Furthermore, the second issue in identifying the geographic constituency of an individual lies in the quality (and existence) of residential address data or legislative district shapefiles for the chamber(s) being studied.⁸

Scholars addressing issue three – matching residential addresses to coordinates – have made many advances. Amos and McDonald (2020) demonstrate the process by which to employ ESRI and Google geocoders, which engage in fuzzy string matching, to geocode millions of addresses from state voter files. The process of hierarchical geocoding devised by Amos and McDonald (2020) improves upon the strengths of each geocoder while mitigating their shortcomings (Swift, Goldberg, and Wilson 2008) to the point of even identifying thousands of errors in misassigned voters within official Colorado and Florida state voter files.⁹ Therefore, their process demonstrates that it is possible to locate constituents when address and boundary data are present.

The final issue is related to cost. Even when precise coordinate data is available, the issue of cost – in both time and money – often prohibits research from approaching the standards set by researchers like Amos and McDonald (2020). As Goplerud (2015) notes, addressing these issues related to geographic assignment and matching individuals to different levels of geography is consistently expensive in regard to programming skills or the purchase of proprietary software. For example, the hierarchical geocoding process employed by Amos and McDonald (2020) requires access to ESRI proprietary software, ArcGIS, and required 5.5 hours for the Florida voter file alone. If accuracy and non-missingness are a concern, the set of backup checks necessary with a suite of several geolocators can significantly increase time costs.¹⁰ Furthermore, those interested in relying upon proprietary software, such as the Google API, will spend upward of tens of thousands of dollars to geocode a state as populous as Florida. Shepherd et al. (2021), in their recent work analyzing polling place access in North Carolina, relied on a service that provides unlimited geocoding, but at the cost of a $1,000 per month subscription.¹¹ Absent the resources of a larger research university – or one of the 175 universities with the infrastructure necessary to run graduate programs in geographic information systems (GIS)¹² – these costs can be prohibitively expensive. Therefore, the question naturally arises, are there any methodological shortcuts that might decrease the burden of relying upon hierarchical geocoding that does not sacrifice the quality of the research?

Within the US context, it is possible to avoid both geocoding and more advanced geographic assignment methods in cases where a small enough unit of geography can be identified that is fully nested within a larger geography. ZIP codes provide the smallest unit of publicly known geography and, in many cases, can fulfill this purpose. Figure 1 graphically shows the percentage of a state’s population that lives in effectively wholly nested ZIP codes within in relation to state legislative districts. There are apparent differences between chambers, with the results ranging from a low of 5.6% in Rhode Island’s state house to a high of 92% in Vermont’s state senate. We additionally see that heavily populated states, such as California, Florida, and Michigan, have populations where well over 50% of the state’s residents live in ZIP codes nested fully within both the lower and upper state legislative districts. Nationwide, 43% of the population lives in ZIP codes fully nested within state house districts and 61% in state senate districts. It is therefore possible to reduce the need and burden associated with geocoding and more complex methods of geographic assignment, though some of either method will still be necessary when locating individuals that live in non-nested ZIP codes.

Figure 1. Percentage of state’s population living within fully nested ZIP codes, by state legislative chamber.

Assignment of lower levels of geography to higher levels when the two are not perfectly nested has historically used one of three geographic assignment methods: centroid matching, geographic overlap, and population overlap. The centroid matching technique assigns an observation from a lower-level geography when its geographic center, or centroid, falls within the boundaries of a higher-level geographic unit. Geographic overlap matching assigns or weights a lower-level geography according to the shared area between it and its higher-level overlapping units. Population overlap matching assigns or weights a lower-level geography by using a third level of atomic tabulation units to estimate the population distribution for the overlap between the lower-level and higher-level geographic units.¹³ While each of these methods is prone to some assignment error, Amos, McDonald, and Watkins (2017) determine the assignment accuracy of these three techniques is highest for population overlap and lowest for centroid matching. However, their analysis was conducted in aggregate. It did not distinguish between where these methods were most useful and when researchers could expect their results to be biased depending on the geographies employed. Some lower-level geographic units are heavily split between several higher-level geographic units, other lower-level units are wholly nested inside a higher-level unit. Knowing where and when these matching methods can accurately assign individuals without geocoding is necessary for its application to be used with confidence.

Figure 2 illustrates how these geographic matching methods are computed and the challenges that arise from their use. Pictured is the 27713 ZIP code in Durham, North Carolina. For individual data points originating from this ZIP code, there are three overlapping legislative districts.¹⁴ For a ZIP code like 27713, assigning a voter to a legislative district using only this identifier is challenging. Using the centroid method, a researcher would place all individual data points from this ZIP code in the fourth legislative district (as denoted by the star in the center of the figure). Researchers using geographic overlap would also assign this ZIP code’s data points to the fourth legislative district given the approximately 40% of geographic space that is shared between the ZIP code and the legislative district. Researchers using population overlap, though, would assign this ZIP code to the first legislative district because the majority of the ZIP code’s population resides to the north. This divergence plagues research attempting to allocate individual data points to one geography based on the point’s membership in a smaller level of geography. This study quantifies the trade-offs when using centroid matching, geographic overlap matching, or population overlap matching in these situations and confirms that population overlap matching is consistently more accurate than alternative geographic matching methods.

Figure 2. Example of difficulties matching ZIP codes to legislative districts, NC 27713.

Validation

We probe the accuracy of geographic matching techniques against the validated and audited geocoded voter file used by Amos (2019). Their data provide the correct and known district residency for each voter within their voter file data (Amos and McDonald 2020). Using their data on correct legislative districts, we predict the correct assignment of voters to their (upper and lower) state legislative districts using only their ZIP codes in Colorado, Florida, Louisiana, New York, North Carolina, and Ohio. ZIP codes are the smallest publicly known geographies within state voter files. Therefore, their use does not require geocoding for matching purposes and can significantly reduce the costs associated with geographic matching.¹⁵

The dichotomous dependent variable for this validation captures whether an individual within a voter file is assigned to the correct district (1) or not (0) for each matching method. To match each voter to their legal state legislative districts using centroid matching, we employed the ArcGIS feature-to-point tool to calculate the geographic center of each ZIP code (constrained to fit within the boundary of the ZIP code) and then overlaid these onto state legislative maps.¹⁶ Geographic overlap matching was accomplished using Missouri Census Data Center (2018), which produced dyads of each ZIP code/legislative district pairing as well as the degree of geographic overlap ranging from 0 to 1.¹⁷ For the dichotomous assignment, we assign each ZIP code to the legislative district that it shares the most geographic overlap with.¹⁸ Likewise, we make use of Missouri Census Data Center (2018) to produce dyads of each ZIP code/legislative district pairing as well as the degree of population overlap ranging from 0 to 1, with Census blocks as the atomic tabulation unit. We then follow the same dichotomous assignment of ZIP codes to state legislative districts as used in geographic overlap matching. This resulted in three models, one for each matching method.

We identify the context of where a matching method is most appropriate through a continuous measure of the degree of nestedness between ZIP codes and legislative districts. To do so, we employ the recommended measure as posited by Curiel and Steelman (2020) – the Herfindahl index. It is calculated on a 0 to 1 scale by taking the sum of squared proportions for all of the ZIP code-district dyadic population overlap scores to the ZIP code level. When a ZIP code is fully nested inside a legislative district, its Herfindahl index score is 1. As the effective number of districts inside a single ZIP code reaches infinity, the score approaches 0.¹⁹ The scores are calculated from the GeoCorr output and represent every ZIP code’s overlap for the state house and state senate district maps.²⁰

Of the three methods, population overlap performs best in aggregate, ranging from 80% to 90% accuracy in predicted legislative district membership for the six states – in line with expectations from Amos, McDonald, and Watkins (2017). As evident in Table 1, geographic overlap performs on par or slightly better than centroid matching – both of which are less accurate than population overlap in assigning voters based solely on a ZIP code. Increased accuracy when using population overlap matching varies across states from a minimum of a single percentage point in Ohio to eight percentage points in Colorado. These results fall short of what is necessary for a full spatial audit consistent with the recommendations from Amos and McDonald (2020) for election administration. However, these findings support the notion that geographic assignment methods are generally helpful for research applications.

Table 1. Accuracy of geographic matching methods in six US states by matching method

In order to determine where these methods differ in accuracy, we predict the probability of correct assignment given the degree of nestedness between a ZIP code and its overlapping legislative districts. We conducted the analysis stratified by state, with the results not substantively different by state. As an example, we present the predicted probability plot for North Carolina in Figure 2. The x-axis presents the Herfindahl index to measure the degree of nestedness, with (1) equating to a ZIP code wholly within a legislative district and (0) representing a ZIP code that is infinitely split.

Looking at the left panel of Figure 3, we see that with a Herfindahl score of 0.90, the probability of correct matching exceeds 95% for all three methods. A score of 0.95 on the Herfindahl index corresponds to effectively 100% accuracy in assignment. Insofar as the accuracy starts to dip below 90% accuracy, it will occur for Herfindahl scores around the 0.75 to 0.79 range. Such a score is equivalent to an effective number of districts within a ZIP code being approximately 1.3. It is around this range that we also start to see the differences in the accuracy of each matching method diverge.

Figure 3. Predicted probability of correct matching.

The right panel of Figure 3 plots the difference between the accuracy of population overlap matching compared to the centroid and geographic overlap methods. We see that population overlap reaches its maximum advantage over geographic overlap at a Herfindahl score of 0.41 by approximately 10% points. However, at such a score, the population overlap method is only accurate in 46% of cases. Compared to the centroid approach, population overlap performs its best at a Herfindahl score of 0.47, increasing 8.5% points in accuracy. At such a score, population overlap is estimated to have approximately 56% accuracy in assignment. It is also important to note that Herfindahl index values in this range represent a large portion of all ZIP code-legislative district pairs – as made evident from the density plot at the bottom of the panel. Therefore, while substantive disparities in matching methods arise, researchers should hierarchically geocode their data if possible and utilize population overlap matching when hierarchical geocoding is not possible.

The light gray shaded area reflects the distribution of ZIP code nestedness within the North Carolina data. The right panel limits the analysis to ZIP codes with a Herfindahl index under 0.95 for the purpose of focusing on the changes in accuracy across methods. ZIP codes hovering above the solid lines represent a sampling of ZIP codes representative of ZIP code-district nestedness.

Application

By utilizing geographic matching methods to distinguish between constituent and non-constituent influence in lobbying, we can apply the various techniques using a real-world situation. Bishop and Dudley (2017) research the influence of lobbying relative to constituency interests among Pennsylvania state legislators voting for pro-fracking legislation. Their case study selection allows for a critical test of matching techniques that minimizes the impact of endogeneity that typically accompanies research on lobbying and policy outcomes. Bishop and Dudley (2017) tackle the challenges posed by Ansolabehere, de Figueiredo, and Snyder Jr. (2003) in identifying a causal relationship between lobbying and policy outcomes head on, choosing a case where a lobby formed within a few years, effectively precluding a mistake in the causal direction. Additionally, the authors employ geocoded gas-well data to measure constituency reliance upon the natural gas industry.

In a roll call model of voting, negative values of the dependent variable reflect a more conservative voting record in favor of the natural gas industry. The explanatory variables of interest are, first, natural gas production within a legislator’s district, and, second, donations from the natural gas industry’s political action committee and associated individuals. In their original analysis, the authors find lobbying, in general, exerts a significant, albeit modest, impact on legislator voting.

The only measurement shortcoming of Bishop and Dudley (2017) can be captured in not having distinguished between donations that arise from in- or out-of-district sources. As theorized by Kingdon (1977), representatives attempt to minimize the tension between their stakeholders, ideally never choosing between influential lobbyists and their constituents, hence their preference for committees relevant to their district. For example, Kalla and Broockman (2016) find that the combination of being both a constituent and donor leads one to be more likely to secure meetings with representatives in their randomized field experiments. If one could separately estimate lobbying by in-district versus out-of-district sources, it would be possible to ascertain how much power and influence lobbyists had relative to a legislator’s own voters. If all lobbying arose from in-district, it would suggest that Pennsylvania representatives were acting within their constituency’s interests. If out-of-district funds still retain an effect, that would suggest a degree of power more associated with fears of corruption and responsiveness to corporations raised by critics of the expansion of natural gas industry goals.

The benefit of population overlap to this analysis is the ability to better distinguish constituency versus non-constituency interests measured using the ZIP codes reported within the contribution data. Pennsylvania is not the ideal state for wholly assigning individuals dichotomously to a district using only a ZIP code. Fortunately, population overlap analysis allows us to weight donations as in-district based upon the proportion of a ZIP code’s population that is located within a legislative district to calculate the respective logged donations. Such coding allows us to estimate the impact of out-of-district lobbying. With these new data, we re-estimate the first model presented by Bishop and Dudley (2017) in their Table 3 (169), predicting legislator voting scores as measured by the Pennsylvania League of Conservation Voters (PLCV).

When analyzing the donation data, we find that approximately 97.2% of the donors to members of the state house lived in ZIP codes completely outside the state house member’s district. The figure is 90.6% when looking at donors to members of the state senate. Approximately 0.4% of donors to members of the state house were completely nested within one district, and 7.5% of the state senate. This results in 97.6% of the donations made to members of the state house and 98.1% of the donations made to members of the state senate bypass the need for geocoding. In fact, only 4.7% of the donations made to members of the state house and 2.0% of the donations made to members of the state senate require the use of a geographic matching method to allocate them as in- or out-of-district. As a result, this suggests that model differences comparing geographic matching methods will be minimal.

Table 2 estimates separate models for the effect of in-district and out-of-district donations using each matching method for comparison. Furthermore, for population overlap and geographic overlap matching, we include separate models where donations are assigned wholesale to the legislative district with the greatest overlap with the donation’s ZIP code and where assignment is weighted on the shared proportion of overlap between a ZIP code and its greatest overlapping legislative district.

Table 2. Comparisons in predicting PLCV scores

Note: *p < 0.1; **p < 0.05; ***p < 0.1.

Perhaps most telling about the usefulness of matching methods is the difference between coefficients for the effect of donations in the original model (column 1) to the coefficients of in-district and out-of-district contributions in each of the subsequent models. It is clear that the original impact of industry donations are somewhat muted relative to estimates obtained when the source of donations are distinguished from one another. For example, when examining the impact of in- and out-of-district donations using population overlap matching, it is clear that in-district donations are having an outsized impact relative to donations coming from outside the legislator’s district. The coefficients for in-district donations tend to be around double that of out-of-district donations. This pattern is found regardless of matching method and without respect to how donations are aggregated. The lack of significant differences between estimations is not surprising, given the aforementioned lack of donor ZIP codes split between districts. Therefore, these results suggest that while the nature of the data makes it less meaningful how the user assigns data, any type of control for donation source can add more nuance as to the differential impact of lobbying by source. Regardless, their original conclusion not only holds up, but is strengthened by utilizing geographic matching to distinguish the source of contributions.

Discussion

As demonstrated, there are situations where geocoding might be necessary to ascertain who represents an individual. However, in the context of American state and local politics, the use of geographic assignment matching methods can reduce the geocoding burden by at least a quarter in every state. Even when an area is split between multiple districts it is possible to confidently assign individuals to legislative districts by gauging the degree of lower-level geography nestedness to make appropriate decisions about the use of geographic assignment methods. These tools can be of use to scholars – and reviewers – in making the call of whether more rigorous methodologies must be employed when assigning voters to legislative districts. Furthermore, it is possible to review previous research, such as Gimpel, Lee, and Pearson-Merkowitz (2008), and determine methodological soundness given matching method and geographic context. In the case of Gimpel, Lee, and Pearson-Merkowitz (2008), we estimate approximately 75% of the nation’s population to reside within ZIP codes fully nested within congressional districts during the 2000s, with the greatest inaccuracies arising within Maryland, Nevada, Florida, New York, and North Carolina.²¹

Following our results, we present a cost/benefit analysis of each geographic matching method in Table 3. We organize the results by method, accuracy, allocation method, impact on replication, and the program used to conduct each.

Table 3. Cost/benefit analysis of geographic matching methods

We ultimately find that in regard to accuracy, population overlap ranks highest, followed by geographic overlap and finally centroid. The impact of accuracy becomes most clear when a lower level geographic unit is split between two effective higher level units. Allocating individual data points using geographic matching proves to be most flexible when using population or geographic overlap, since each provides a continuous 0–1 score to allocate fractions of a value or assign an entire unit of geography based on the greatest degree of overlap. Costs were highest in terms of compromised accuracy and financial burden for the centroid method as it required access to the centroid geographic bounding tool through ArcGIS. This is compared to a free download of the program GeoCorr which can be used to facilitate geographic and population overlap.²²

From Table 3, it is apparent that population overlap weakly dominates the other two methods, and centroid assignment is weakly dominated by both population and geographic overlap. In order to aid in future research, we suggest the following rules by which to implement geographic matching assignment.

First, are both levels of geography available from the US Census? If so, then it is possible to employ GeoCorr. If either level of data is not from the US Census, employ a package that can read in raw shapefiles and use geographic operations to find the population or geographic overlap. These might consist of an R CRAN available package by Goplerud (2015) or the recently developed arealOverlapr package (Curiel 2022).

Second, to what extent does the higher level of geography split the lower level? This should be determined by finding the Herfindahl index/effective number of higher geographic units nested within the lower level. To avoid the need of discarding data with accuracy under 90% as practiced by Enos (2015), it is recommended to weight data by dyadic overlap should the Herfindahl index fall below 0.75.

Finally, should the researcher feel uncomfortable with partial weighted geographic assignment and they prefer to geocode individual observations, it is recommended that the researcher lessen the burden of geocoding. By identifying those lower level geographic units completely nested within the higher level, researchers can subset the data they must geocode to only those observations that are not fully nested within the higher level geography being employed. This procedure can save researchers hours of time and potentially thousands of dollars. As illustrated in Figure 1 and the replication of Bishop and Dudley (2017), it might be the case that only few observations even need to be partially weighted or geocoded after properly identifying the data to be geocoded.

Although there will always be uncertainty in geographic assignment where different geographies do not nest within each other, we have improved the confidence that one can have when researching such matters and utilizing such matching techniques. The improvements in population overlap analysis highlight the usefulness of more recent advances in GIS capability and ease of access to individuals. We assert that population overlap analysis offers a valuable tool to anyone pursuing research questions involving the geographic assignment of inconsistently nested geographies.

We conclude by noting that even though the field of state and local politics is highly variable in regard to the quality of data available, it is possible to overcome the challenges of identifying geographic constituencies scientifically. Moreover, more nuanced identification of these constituencies via geographic assignment and weighting can in turn improve and expand our understanding of state and local politics.