Measuring income of the aged in household surveys: evidence from linked administrative records

Abstract

Household survey estimates of retirement income suffer from substantial underreporting which biases downward measures of elderly financial well-being. Using data from both the 2016 Current Population Survey Annual Social and Economic Supplement (CPS ASEC) and the Health and Retirement Study (HRS), matched with administrative records, we examine to what extent underreporting of retirement income affects key statistics: elderly reliance on social security benefits and poverty. We find that retirement income is underreported in both the CPS ASEC and the HRS. Consequently, the relative importance of social security income remains overstated – 53 percent of elderly beneficiaries in the CPS ASEC and 49 percent in the HRS rely on social security for the majority of their incomes compared to 42 percent in the administrative data. The elderly poverty rate is also overstated – 8.8 percent in the CPS ASEC and 7.4 percent in the HRS compared to 6.4 percent in the administrative data.

1. Introduction

As people transition from working life to retirement, their income sources change. They become less reliant on earnings and more reliant on income from social security, pensions, and personal savings, although the precise mix of these income sources varies across the income distribution (Mitchell et al., 2022). This transition raises questions of retirement income adequacy, which is of primary importance in U.S. policy discussions. Most older Americans are beneficiaries of federal income support programs,¹ and the majority of federal budget expenditures will soon be devoted to supporting Americans aged 65 and over.

To assess the performance of the U.S. retirement system, including the relative importance of each of its components, analysts have long relied on public-use microdata from household surveys. For example, the Social Security Administration (SSA) Office of Research, Evaluation, and Statistics has published biennial statistical series on the income sources of the aged population since 1976.² Two of its most well-known publications are Income of the Population 55 or Older and the Income of the Aged Chartbook. These publications provide financial well-being estimates for the older population and for various demographic subgroups. Such estimates include total money income, the prevalence and amount of income received from different sources (such as earnings, social security, pensions, and assets), and poverty status. Both publications are based on public data from the Current Population Survey Annual Social and Economic Supplement (CPS ASEC), also known as the ‘March CPS’,³ which is the nation's official source of income and poverty statistics.

However, an accumulating body of research has cast doubt on the reliability of income measures from household surveys in general (Meyer et al., 2015), and especially retirement income data in the CPS ASEC (Schieber, 1995; Woods, 1996; Iams and Purcell, 2013; Miller and Schieber, 2013; Bee and Mitchell, 2017; Dushi et al., 2017; Brady and Bass, 2021).

Bee and Mitchell, linking the 2013 CPS ASEC microdata at the individual level to Internal Revenue Service (IRS) and SSA administrative records, established that reporting errors were widespread throughout the income distribution. About half of surveyed Americans aged 65 and over who received income from retirement plans (as indicated by IRS administrative data) reported no retirement income at all in the survey, such that the median household income was understated by approximately 30 percent and the poverty rate was overstated by approximately a quarter (7 versus 9%). While reporting rates of distributions from defined contribution (DC) plans were lower than those from defined benefit (DB) plans, the relative prevalence and size of DB plans among current retirees meant that much of the underreporting could be attributed to DB pensions. In contrast, social security income was reported relatively well in the CPS; however, due to underreporting of retirement and asset income, estimates of beneficiaries' relative reliance on social security had been overstated.⁴

Some analysts subsequently suggested that the measurement concerns were limited to the CPS and that other household surveys were more reliable. Chen et al. (2018) identified the University of Michigan's Health and Retirement Study (HRS) as a promising candidate, as it focuses solely on households with a head aged 51 years or older, follows a panel of households over several years, and records data on sporadic withdrawals as well as regular payments. Thus, if any other household survey could be expected to better reflect the financial well-being of the elderly, it seems likely it would be the HRS.⁵

This paper's main contribution is to examine whether previously documented income measurement issues are limited to the CPS or are also found in other household surveys like the HRS. Our findings will inform policymakers in general and SSA's decision makers in particular: (a) whether the post-redesign CPS is a reliable and accurate data source for continued publication of income statistics for the aged population, (b) whether and to what extent administrative data can enhance the quality of those estimates, and (c) whether the HRS does a better job in measuring income of the aged. More specifically, we address the following questions. Are incomes of the elderly population as underreported in the HRS as they are in the CPS? If so, how does underreporting of retirement income affect key measures of the retirement system's performance, such as reliance on social security benefits and the poverty rate of older Americans?

To address these questions, we analyze the 2016 wave of the HRS matched with SSA administrative records.⁶ We compare our findings to results derived from the 2016 CPS ASEC linked to administrative records from SSA and IRS. We show that even after the survey redesign, pension and asset income are still underreported in the CPS. We also find that once we account for underreporting of retirement income by using IRS administrative data, social security is no longer the main source of income for elderly beneficiaries – 53 percent in the survey versus 42 percent in the linked data receive at least half their family income from social security. Furthermore, we find that the poverty rate for the population aged 65 and over is substantially lower in the linked data than in the survey (6.4 versus 8.8%, respectively). Finally, we find that while the HRS does a better job than the CPS ASEC in terms of capturing retirement income, it still falls considerably short of the CPS ASEC when linked to SSA and IRS administrative records. Hence, key statistics such as the relative importance of social security income (49% in the HRS versus 42% in the linked CPS) and the poverty rate (7.4% in the HRS versus 6.4% in the linked CPS) remain overstated. Based on these findings, we conclude that income underreporting is not confined to the CPS ASEC, and surveys linked to a broad range of administrative records remain the most reliable tools to assess the performance of the U.S. retirement system. Therefore, it is our hope that decision makers at both the SSA and the U.S. Census Bureau will facilitate the regular usage of the CPS ASEC linked to administrative data for the continued production of the Income of the Aged publications.

The remainder of the paper proceeds as follows. Section 2 provides a brief background of the relevant literature. Section 3 describes the data and methods used for this analysis. Section 4 presents statistics from the different data files. Section 5 provides concluding remarks.

2. Literature review

Extensive literature has shown that household surveys are subject to several types of errors which can lead to inaccurate estimates of income and its distribution, poverty rates, and program participation (Bollinger and David, 2001; Johnson and Moore, 2008; Meyer and Wu, 2018; Meyer and Mittag, 2019; Meyer and Mittag, 2021; Corinth et al., 2022; Celhay et al., 2024). While total survey error comprises many forms of non-sampling error such as coverage error, item non-response, and misreporting, in the last two decades researchers have focused more on the issue of measurement error by taking advantage of survey data linked to administrative data (Meyer and Mittag, 2019).

Historically, the CPS ASEC has been widely used because of its large sample size, public availability, broad array of demographic, geographic and economic data collected, and annual periodicity. However, prior research has shown that different types of income in the CPS ASEC, relative to alternative data,⁷ are subject to reporting errors. Schieber (1995) was the first to show that, even as early as 1990, income from pensions and annuities was underreported in CPS ASEC by as much as one-third when compared to aggregate estimates from the National Income and Product Accounts and the IRS Statistics of Income division (SOI). Since then, as DC plans have become more prevalent among workers, the underreporting of income from pensions and retirement accounts continued to be identified by researchers as a major concern (e.g., Woods, 1996, Rector et al., 1999, Roemer, 2000, Brady and Pierce, 2012, Iams and Purcell, 2013, Miller and Schieber, 2013, Brady and Bass, 2021),⁸ while other analysts identified lower aggregates in interest and dividends relative to benchmarks (e.g., U.S. Bureau of the Census, 1993; Ruser et al., 2004).

In the absence of validation studies based on individually linked data, retirement researchers were forced to speculate as to the causes of the large and growing aggregate discrepancies. Many pointed to the wording of the CPS retirement income question, which only asked about ‘regular’ distributions. As DC plans became more common among workers and younger cohorts of retirees, evidence that many plan participants withdrew money from these plans in irregular, ad hoc payments implied that CPS would miss these payments (Sabelhaus and Schrass, 2009). It also remained unclear to what extent aggregate discrepancies reflected underreporting only among the affluent or broadly across the distribution.

While the prior studies largely focused on aggregate income comparisons, Bee and Mitchell (2017) took advantage of microdata linkages between the 2013 CPS ASEC and administrative records from SSA and IRS to measure misreporting at the individual level.⁹ They compared amounts reported in the CPS with those reported in the matched administrative records for five types of income: earnings, social security benefits, supplemental security income (SSI) payments, interest and dividends, and ‘retirement income’ (comprised of DB pension benefits and DC account withdrawals apart from rollovers and conversions). With respect to income sources that are available in the SSA administrative data – earnings, social security benefits, and SSI – the authors reported that among the population aged 65 and over, survey reported earned income¹⁰ and social security benefits track the respective measures in the administrative records very closely (98 and 96%, respectively), whereas survey reported SSI amounts comprise only 73 percent of the amount in the administrative records.¹¹ The authors also found that, among all households headed by an individual aged 65 or older, median household income was approximately 30 percent higher in the administrative records than in the CPS and that the survey-based poverty rate for persons aged 65 or older was overstated by approximately one-quarter.

Bee and Mitchell also showed that while earned income and social security benefits were reported accurately, the difference in estimated overall income was mainly due to underreporting of retirement income (from both DB pensions and DC withdrawals) and, to a lesser extent, of interest and dividends. The discrepancy in median income between survey and administrative data increased from 20 percent in 1990 to 30 percent in 2012. As a result of differential underreporting among income sources, Bee and Mitchell found that the CPS ASEC overstates the relative importance of social security income. Specifically, the proportion of persons aged 65 or older who rely on social security for at least 50 percent of their family income is overstated (55% in the survey versus 42% in the linked data) as is the proportion of those relying on social security for at least 90 percent of their family income (26% for the survey versus 12% for the linked data). These findings clearly quantify the magnitude of misreporting of retirement income and support the Census Bureau's decision to redesign the income questions.¹² However, in preliminary work, Bee and Mitchell (2021) find that the effect of the redesign on income estimates was limited.

More recently Chen et al. (2018) compared reported income from the CPS ASEC and four other national household surveys with aggregates from SOI and SSA's Annual Statistical Supplement to the Social Security Bulletin. They concluded, as prior researchers have, that the CPS ASEC misses large portions of retirement income. Interestingly, they found that the other four national surveys were much better than the CPS in terms of capturing retirement income.¹³ Among the other surveys, the HRS emerges as a particularly attractive option for retirement researchers, as it is specifically designed to measure the well-being of older adults and produces publicly available microdata every 2 years.

Income misreporting in the HRS has also been studied. Retirement income, however, has not typically been the focus of prior work perhaps due to the lack of linkable IRS administrative records. Hurd et al. (2003) report that changing the HRS questionnaire to combine asset ownership questions with asset income questions appeared to reduce underreporting of asset income. Alwin et al. (2014) exploit the HRS's panel design to assess the reliability of its income data. While they were not able to assess retirement income items due to changes in the questionnaire, they find that measures of veterans' payments and social security are reported more reliably than SSI. Hyde and Harrati (2023) find net underreporting of SSI and disability insurance receipt, relative to linked SSA records.

Some HRS validations have shown that older workers do not accurately report their retirement program participation. Gustman et al. (2010), using information from employer plans, find workers are often confused about key plan characteristics. Dushi and Honig (2015), using information from SSA earnings records, find that workers often do not accurately recall whether they contribute to their retirement plans and misreport the amount of their contributions. Gustman et al. (2014) find that pairing a cohort of HRS workers in their early 50s to CPS ASEC responses in their late 60s reveals ‘disappearing pensions’, as over a third of implied annuitized pension wealth seemed to vanish in the CPS ASEC relative to the HRS.

3. Data and methods

The data we use in this paper come from two major household surveys – the 2016 CPS ASEC and the 2016 wave of the HRS – and from two sources of administrative data – SSA data covering earnings, social security benefits, and SSI payments, and IRS data covering interest income (both taxable and tax-exempt), dividends, and retirement income (both DB pensions and DC withdrawals). We compare several income and poverty estimates for the aged population using five alternative data files that we derive consisting of:

1. (1) HRS

2. (2) CPS ASEC

3. (3) HRS data linked with administrative data from SSA (the ‘HRS + SSA’ data file)

4. (4) CPS ASEC data linked with administrative data from SSA (the ‘CPS + SSA’ data file)

5. (5) CPS ASEC data linked with administrative data from both SSA and IRS (the ‘CPS + SSA + IRS’ data file).

Our focus is on pre-tax family income of persons aged 65 or older.¹⁴ Following SSA's Income of the Aged publications, we provide estimates of total family income¹⁵ as well as separate estimates for each of five income categories: social security benefits, income from pensions and retirement accounts, earnings, asset income (including interests, dividends, and rental income), and other income (including income from veterans' benefits, unemployment and workers' compensation, government transfer programs, and other income).¹⁶ For this analysis, we follow the recommendations of Anguelov et al. (2012) and count distributions from retirement accounts as income, including infrequent and periodic withdrawals. This includes payments from both DB and DC plans, as well as traditional and Roth Individual Retirement Account (IRA) withdrawals, but excludes transfers between tax-preferred accounts such as rollovers and conversions. This method was also employed by Bee and Mitchell (2017), and as they pointed out the nature of the administrative data lends itself to counting all withdrawals that permanently leave tax-preferred accounts as income. In both the CPS ASEC and HRS the interviews took place in 2016, but the reference year for all income measures is the previous calendar year, 2015.

3.1 Current population survey annual social and economic supplement

The CPS ASEC surveys a nationally representative sample of the U.S. non-institutionalized population. The survey collects detailed demographic information, including but not limited to age, gender, race/ethnicity, marital status, and household composition. In addition, the CPS ASEC collects information on income from different sources – such as earnings, social security, pensions, assets, and government transfer programs – that each household member aged 15 and over received during the previous calendar year. As such, the CPS ASEC has long been the official source of estimates of household income and poverty.

From the beginning, the main goal of the CPS ASEC was to capture regular cash income received that people could spend (Roemer, 2000). The Census Bureau's money income concept is defined as all regular income received before payments of personal income taxes, social security payroll tax, union dues, Medicare deductions, etc.¹⁷ Note that prior to the redesign of the CPS questionnaire, the survey did not ask respondents about certain income sources, and the Census definition of money income excluded irregular income such as any lump sum withdrawals from pensions or retirement accounts. As noted above, several studies have been critical of the CPS and have emphasized that income from assets and tax-advantaged retirement accounts (such as 401(k) plans and IRAs) are inadequately measured in the CPS, resulting in estimates that understate the importance of such accounts and consequently overstate the importance of social security benefits (Schieber, 1995; Woods, 1996; Fisher, 2008; Davies and Fisher, 2009; Iams and Purcell, 2013; Miller and Schieber, 2013; Munnell and Chen, 2014).¹⁸

The Census Bureau has been receptive to these studies and fielded a redesigned survey instrument aimed at improving the collection of income from different sources – particularly to capture irregular distributions from retirement accounts and for specific types of households (older and low-income) – by implementing several changes. These included eliminating redundant questions to reduce query fatigue and revising the order of the income questions to target the most likely sources of income. In addition, a ‘dual-pass’ approach was implemented that first asks about sources of income and then about the amounts from each source. Lastly, respondents are now asked about any distributions from DC accounts, which are now included in the Census definition of money income.

As noted above, of central interest is whether the underreporting of retirement income remains as prevalent as it was prior to the CPS redesign. Hence, we use data from the 2016 CPS ASEC collected during the second year of full implementation of the redesigned questionnaire. This allows us to compare our findings with Bee and Mitchell (2017), who assessed income reporting in the 2013 CPS ASEC prior to the redesign. We follow their work and link the CPS ASEC with restricted-access administrative records from SSA and IRS using an anonymized unique identifier on a secure Census Bureau server.^{19, 20} This linkage allows us to validate data on several income sources by comparing respondents' survey reports to respective amounts in the administrative records for the same year, 2015.

For respondents that are linkable (approximately 90% of respondents aged 65 or older), we replaced the values of income reported in the CPS ASEC with values from the administrative records. For this group of respondents, administrative records from SSA allow us to validate social security benefits (retirement and disability),²¹ SSI payments, and earnings from employment, resulting in what we call the ‘CPS + SSA’ data file. Adding the IRS administrative records allows us also to validate income from pensions and retirement accounts and income from interests and dividends, resulting in what we call the ‘CPS + SSA + IRS’ file. For the remaining respondents who are not linkable, we used their survey-reported values. It is worth noting here that Bee and Mitchell (2017) limited their analysis sample to linkable respondents and then reweighted the sample using inverse probability weights. Here, we decided not to exclude respondents who cannot be linked and thus we do not reweight the sample. The main reason is we want to maintain a consistent methodology with our analysis of the HRS which, as described below, has a much lower linkage rate to the SSA data and thus would be more sensitive to any reweighting procedure. However, it is worth noting that the 90 percent match rate in the CPS ASEC is high enough that any differences in the main results shown here versus restricting to linkable respondents and reweighting are found to be negligible.^{22, 23}

3.2 Health and retirement study

The HRS is a comprehensive national longitudinal survey of Americans aged 51 or older. The first HRS interviews took place in 1992, with follow-up interviews conducted every other year since then. The main goal of the HRS is to provide data that allow researchers to examine interactions between social, economic, health, and psychological factors and the retirement decisions of older adults during pre- and post-retirement years. By conducting in-depth interviews, it also provides a broad array of information on topics such as employment, income, wealth, and other characteristics of the population aged 51 or older.²⁴ Another advantage of the HRS is that at the end of the interview it asks respondents for their consent to link their survey information with earnings and benefits information from social security administrative records.²⁵ Furthermore, from its inception the HRS has been more systematic than the CPS in collecting information on pensions, retirement plan account balances, and their distributions. The HRS respondents are first asked whether they have income from different income sources, pensions, and retirement accounts, and then about the amount of income received and frequency (with ‘other’ being one of the frequency options, in addition to the regular ones). If respondents, when asked, do not report the amount of income or wealth, then they are asked follow-up questions about the dollar amount using an ‘unfolding brackets’ approach to identify the range limits of the missing data item.²⁶ For respondents with unfolding brackets, their income is imputed and made available in the public use data file.

Czajka and Denmead (2008) showed that HRS-reported household income amounts in 2002, among people aged 51 or older, were substantially higher (by 20–30%) than amounts reported in the CPS ASEC; and while both samples had similar demographic characteristics, the HRS respondents were less likely to live alone than were their CPS ASEC counterparts. The authors conclude that ‘HRS incomes are higher than those of the Census Bureau surveys but resolving whether this is due to better measurement or over-representation of higher-income families must be left to future research’.

In this article, we use income information from the RAND-HRS user-friendly data file, which includes information from all interviews conducted from 1992 to 2016,²⁷ standardized across survey years. Specifically, we focus on the sample of people who were aged 65 or older (if married the spouse can be of any age) in the 2016 wave and use only the income variables collected in that wave. For each respondent in the HRS, income measures include earnings, private pensions, social security benefits, income from government welfare programs, capital income, and other sources.²⁸ For married respondents, information about the spouse's income from those same sources is also collected and available as separate variables. Thus, the total household income (H13ITOT is the variable we use) is simply the sum of income received from all sources by the respondent and the spouse (if married or partnered). It is worth noting that while we use the ‘household’ designation to be consistent with the terminology used in the RAND-HRS file, we believe that it is in fact closer to the Census family income definition, particularly for respondents who live alone or with only a spouse/partner. Given that, as noted above, our measure of total household income does not include income of other residing family members, it is likely that our measure of total family income will be downward biased among respondents with three or more resident family members, and consequently reliance on social security will be biased.²⁹ For the purpose of estimating poverty status, RAND-HRS has created another total household income variable (named HwPOVHHI), which is derived by adjusting the abovementioned variable (HwITOT) in order to conform to the Census definition of family income by adding back the amount of Medicare Part B and/or D premiums (not included in the social security payments) and income of other resident family members, and subtracting food stamps and income of family members in nursing homes. We use this adjusted family income variable for estimating poverty status. However, given this adjustment includes several items, we cannot disentangle the amount of income of other resident members or the source of income (whether earnings, pensions, social security, etc.) and thus we cannot include it in our measures of family income; we acknowledge that our measure of HRS family income is likely understated.³⁰

To complement the HRS and to provide a comparison for the CPS + SSA data file, we also created an HRS + SSA data file. For the HRS survey respondents who consented to have their administrative records linked with HRS data, we matched their survey reports with information from SSA earnings and benefits records using anonymized unique household and person identifier on a secure virtual enclave at the University of Michigan.³¹

For respondents with a match, we replaced survey earnings and social security benefits with their respective values from the administrative records. Out of 42,053 observations that ever entered the HRS sample, 20,912 observations were alive respondents in the 2016 wave, of which 9,986 observations were alive respondents aged 65 and over. From this sample we exclude 407 respondents residing in institutions to be consistent with the CPS ASEC universe, and the remaining 9,579 observations constitute the overall sample for this analysis.^{32, 33} Table A2 shows the match rate overall and separately for each administrative data file for our sample of elderly respondents who had a valid consent for the income year 2015.³⁴

3.3 SSA administrative data

Previous research has shown that survey respondents may misreport their earnings, social security benefits, or transfer program participation (Rodgers et al., 1993; Bollinger, 1998; Pedace and Bates, 2000; Bound et al., 2001; Bricker and Engelhardt, 2007; Iams and Purcell, 2013; Meyer et al., 2015; Dushi et al., 2017). To account for potential reporting error, we use social security administrative records, which maintain information on annual earnings, social security (OASDI) benefits, and SSI payments. The earnings information comes from the Detailed Earnings Records file (DER),³⁵ which contains wages and salary payments for each job reported on Form W-2 (defined as the sum of wages, tips and other compensation, and employee tax-deferred contributions to retirement accounts) as well as Medicare-covered self-employment earnings reported on Form 1040 Schedule SE. Information about social security benefit amounts comes from the Payment History Update System (PHUS) file, which contains information separately for each month of the net amounts of benefits paid to a beneficiary as well as the amounts of the Medicare premium paid on the beneficiary's behalf to the Centers for Medicare and Medicaid Services. Hence, the true (or gross) monthly amount of social security benefits that a beneficiary is entitled to is the sum of the net benefit and the Medicare premium.³⁶ Finally, we use the information from the Supplemental Security Records (SSR) file to obtain information about SSI receipt and payment amounts for respondents with a match.³⁷ In all instances, for respondents who are not linkable to administrative data, we use their survey-reported values.

3.4 IRS administrative data

Available IRS administrative data include information on retirement income from sources other than social security and income generated from asset holdings. We match information from two IRS administrative data files to the 2016 CPS ASEC. Unfortunately, the IRS data are not linkable to the HRS.

The first of these files is composed of data taken from the IRS information return Form 1099-R. The 1099-R data allow us to validate retirement income received from both DB pensions as well as withdrawals from DC employer-sponsored plans and from IRAs. As Bee and Mitchell (2017) note, this file excludes data on direct rollovers, Section 1035 exchanges, and Roth IRA conversions. This is to our advantage, as we want to count only income that permanently leaves tax-preferred accounts.³⁸

The second IRS administrative data file we use is IRS Form 1040. From this file, we validate interest and dividend income for CPS ASEC respondents who filed a return for income year 2015. For CPS respondents who are linkable but did not file a 1040 we use the interest and dividend amounts reported in the survey.

3.5 A brief word on income not captured by administrative data

Administrative data, while generally reported with greater accuracy than survey data, are still not free of errors. Moreover, many types of income are simply not available in the set of administrative data we have access to. For example, income data collected in the CPS ASEC or the HRS that are not available in our administrative data files include unemployment insurance; workers' compensation; public assistance (other than SSI); rents, royalties, estates, and trusts; educational assistance; alimony; child support; and in-kind support from outside the household. For these income sources, we use the values reported in the survey. Additionally, the SSA DER includes only the taxable portion of self-employment earnings and likely misses earnings from the informal labor market. As a result, it is possible that among the self-employed, earnings in administrative records are under-reported. However, given that our focus is people aged 65 and over who are less likely to work, it is plausible that such underreporting is only a modest problem. Furthermore, while SSR contain information about federal SSI benefits, the portion of benefits received from states is incomplete or partially missing (see Meyer et al., 2021). In sum, given our data miss several types of income, it is plausible that even our preferred estimates of income are still understated.

4. Results

In this section, we present estimates from each of the five data files: the HRS, the CPS ASEC, the HRS + SSA, the CPS + SSA, and the CPS + SSA + IRS. Table 1 compares the demographic characteristics of the CPS and the HRS samples. The two samples of aged respondents generally exhibit similar demographic characteristics. The two samples differ, however, with respect to the number of persons in the family and the proportion of those who are social security beneficiaries. Similar to findings by Czajka and Denmead (2008) using the 2002 HRS wave, we find that aged HRS respondents in 2016 are less likely to live alone than their CPS counterparts (approximately 28 versus 32%, respectively). Furthermore, according to survey reports, the proportion of respondents who are social security beneficiaries is higher in the HRS than in the CPS (approximately 90 versus 82%, respectively).

Table 1. Demographic characteristics of respondents aged 65 or older in two surveys, by age group, 2016 (in %)

Notes: Authors' calculations based on 2016 wave of HRS and 2016 CPS ASEC. Estimates are weighted using survey weights. Total percentages may not add to 100 due to rounding. The HRS sample removes respondents in institutions to be consistent with the CPS sample. For brevity, in all the tables we refer to CPS ASEC as CPS. For HRS, the public data come from RAND-HRS public file 2016 version 3 (that includes data from 1992 to 2016).

* For the CPS sample, this category includes respondents identified as ‘Asian alone’, other race, and those who report multi- race categories. However, the majority of CPS respondents in this category are Asian alone.

^† We derive these categories using the variable ‘h13hhres’ in the RAND-HRS data file, which indicates the number of residing family members in the household. There is another variable in this file, named the ‘family composition’ variable (‘h13povfam’) that was created in order to define poverty thresholds and poverty rates. We use the latter variable in an earlier version of the paper, but after discussion with RAND researchers, we decided to use the former variable instead to produce this table. It is worth noting that the overall pattern is similar regardless of which variable is used.

‡ These are survey respondents who report receiving social security OASDI benefits.

4.1 Aggregate income

We begin by discussing aggregate income for the population aged 65 and older. Starting first with the CPS, Table 2, panel A reveals that the aggregate total income of the U.S. elderly population is greater when using the administrative records than what is observed in the survey alone. Aggregate total income amounts to $2,100 billion in the CPS and only slightly higher at $2,129 billion (or 1.4% higher) in the CPS + SSA file.³⁹ This reflects the modest aggregate boost from substituting administrative record measures of earnings, OASDI benefits, and SSI. However, when examining the CPS + SSA + IRS file, the aggregate income of the elderly population amounts to $2,520 billion, a substantial 20 percent higher than the CPS alone. This reflects that the survey greatly underestimates retirement income and to a lesser extent asset income.

Table 2. Aggregate income amounts and shares by source for the U.S. population aged 65 or older: estimates from five alternative data files, 2015

Notes: Authors' calculations based on the 2016 wave of HRS and 2016 CPS ASEC, matched with administrative data from SSA and IRS. We refer to CPS ASEC as CPS. For HRS, the public data come from RAND-HRS public file 2016 version 3 (that includes data from 1992 to 2016). The HRS sample removes respondents in institutions to be consistent with the CPS sample. Estimates are based on all respondents with or without a match. Records that are not linkable use survey values throughout. Estimates are weighted using survey weights. SSA administrative data contain wage and salary earnings from Form W-2, self-employment earnings from Form 1040 Schedule SE, OASDI benefits, and SSI benefits. IRS administrative data contain DB pension income and DC withdrawals (excluding rollovers and conversions) from Form 1099-R and interest and dividends from Form 1040. Income reference year is 2015, the calendar year prior to the interview. Dollar amounts are in billions of 2022 dollars, inflation adjusted using CPI-U. Total percentages may not add to 100 due to rounding.

* Reported estimates for CPS data are approved for release by the Census Bureau's Disclosure Review Board (CBDRB-FY24-SEHSD003-010). Analysis of the HRS data is performed at the HRS data enclave at the University of Michigan, and these estimates are approved for release by HRS's Disclosure Review Board in November 2023.

† Pension income includes DC withdrawals but excludes rollovers and conversions.

Another important finding is that the amount of aggregate income in the HRS is substantially higher than in the CPS. This pattern continues to hold with the linked data as aggregate income in the HRS + SSA data file is 13 percent higher compared to the CPS + SSA + IRS file ($2,852 versus $2,520 billion, respectively). With the important exception of pension income, incomes from all other sources are higher in the HRS + SSA data file than in the CPS + SSA + IRS file.⁴⁰ The largest proportional difference between the two data files is the amount of ‘Other’ income,⁴¹ which is 5.3 times higher in the HRS than in CPS. Note that, in the HRS + SSA file, earnings, social security benefits, and asset income are also higher (by approximately 35, 5, and 8%, respectively), whereas pension income is lower (by 28%) than in the CPS + SSA + IRS file.

In terms of each source's share of aggregate income, Table 2, panel B reveals that there are few differences between the CPS and CPS + SSA files. That is, when supplementing with the SSA administrative data, we observe only slight changes in the shares of earnings (from 30 to 29%) and social security (from 35 to 37%). This finding is consistent with the findings of Bee and Mitchell (2017) who found only small aggregate earnings and social security benefit differences between survey and administrative measures. Also of note, the ‘Other’ share of aggregate income changes modestly – the aggregate dollar amount increased by $3 billion because of higher amounts of SSI found in the administrative data.

Pension income accounts for a much larger share of aggregate income in the administrative data than in the survey (23.6% in the CPS versus 33.2% in the CPS + SSA + IRS).⁴² The 10-percentage-point difference suggests that while the CPS redesign may have somewhat improved the reporting of pensions and retirement account withdrawals, its success in improving the measurement of retirement income has been limited relative to IRS data, a finding that is consistent with Bee and Mitchell (2017) and Chen et al. (2018). Particularly noteworthy is that in the CPS + SSA + IRS file, pension income accounts for the largest share of aggregate income (33%), followed by social security benefits (31%). In contrast, when only examining CPS or CPS + SSA data, social security accounts for the largest share (37%) with pensions only the third highest share (23%). Finally, even after the survey redesign, it is noteworthy that the asset income share is still understated in the CPS, although to a lesser extent than the pension income share.

Interestingly, in the HRS data, pension income accounts for almost the same share of aggregate income as in the CPS and CPS + SSA files (23%). Put another way, the HRS pension share is still substantially lower than that in the CPS + SSA + IRS file (22 versus 33%). It is worth emphasizing that, the aggregate dollar amount of HRS pension income is about 32 percent larger than the CPS amount ($656 versus $495 billion); however, this is still approximately 22 percent lower than the amount in the CPS + SSA + IRS file ($835.7 billion).

4.2 Aged population by family income

Table 3, panel A shows the distribution of persons aged 65 or older by family income level. In general, in both surveys, the family income distribution is somewhat similar between the standalone survey and the survey matched with the SSA data file. However, there are noticeable differences in comparison to the CPS + SSA + IRS file, for which the general pattern is a shift in the distribution toward higher incomes. The effect of supplementing with SSA data and particularly with IRS data results in a decrease in proportion of individuals concentrated in the lowest part of the income distribution, with their shares more evenly distributed across the middle and upper parts of the income distribution. Respondents with annual family income below $20,000 comprise about 10 percent of the aged population in the CPS + SSA + IRS file, compared with about 14 and 20 percent in the CPS and HRS, respectively. The same proportions are slightly lower in the CPS + SSA and HRS + SSA files (approximately 13 and 18%, respectively), suggesting that the combination of earnings, social security benefits, and SSI is underreported in the lower part of the distribution. Furthermore, elderly respondents with annual family incomes of $75,000 or higher comprise about 46 percent of the population in the CPS + SSA + IRS data compared with 37 and 31 percent of the population in the CPS and HRS, respectively. These proportions are almost the same when we substitute information in the survey with SSA administrative data suggesting accurate reporting of those income sources in the upper part of the income distribution.

Table 3. Percentage distribution of the population aged 65 or older, by family annual income: estimates from five alternative data files, 2015

Notes: Authors' calculations based on the 2016 wave of HRS and 2016 CPS ASEC, matched with administrative data from SSA and IRS. We refer to CPS ASEC as CPS. For HRS, the public data come from RAND-HRS public file 2016 version 3 (that includes data from 1992 to 2016). The HRS sample removes respondents in institutions to be consistent with the CPS sample. Estimates are based on all respondents with or without a match. Records that are not linkable use survey values throughout. Estimates are weighted using survey weights. SSA administrative data contain wage and salary earnings from Form W-2, self-employment earnings from Form 1040 Schedule SE, OASDI benefits, and SSI benefits. IRS administrative data contain DB pension income and DC withdrawals (excluding rollovers and conversions) from Form 1099-R and interest and dividends from Form 1040. Income reference year is 2015, the calendar year prior to the interview. Dollar amounts are inflation adjusted using CPI-U. For disclosure purposes, in both surveys, reported percentiles are linearly interpolated across intervals of $2,500. Total percentages may not add to 100 due to rounding.

* Reported estimates for CPS data are approved for release by the Census Bureau's Disclosure Review Board (CBDRB-FY24-SEHSD003-010). Analysis of the HRS data is performed at the HRS data enclave at the University of Michigan, and these estimates are approved for release by HRS's Disclosure Review Board in November 2023.

In addition, panel B of Table 3 shows family income at select percentiles. It is notable that despite large differences in aggregate income, at the 10th and 25th percentiles annual family income is almost the same in both HRS and CPS. Even at the median, family income is only modestly higher in the HRS (by approximately $3,000). This pattern continues to hold when comparing the HRS + SSA and CPS + SSA files. There is a much greater difference between the HRS and CPS in the upper half of the distribution. At both the 75th and 90th percentiles, family income is higher in the HRS than in the CPS, and slightly higher in the standalone surveys compared to the surveys linked with SSA administrative data. Finally, relative to the CPS, the CPS + SSA + IRS file shows substantially higher family income at all percentiles. At the median, for example, total family income in the CPS is underreported by approximately 25 percent ($54,840 versus $68,650). Compared to the CPS + SSA + IRS, the HRS and HRS + SSA also fall short at the median by 18 and 14 percent, respectively.

4.3 Sources of income

Table 4 shows the proportion of people aged 65 or older with any family income from each of seven income sources, overall and separately by gender. The estimates indicate that a substantially lower proportion of people aged 65 or older reported income from pensions and retirement account withdrawals in the CPS compared to the CPS + SSA + IRS (48.5 versus 68.4%). In the HRS approximately 60 percent of elderly respondents have family income from pensions or retirement accounts, suggesting that the HRS does a better but still imperfect job with retirement income reporting at the extensive margin.

Table 4. Percentage of individuals aged 65 or older with family income from each source, by gender: estimates from five alternative data files, 2015

Notes: Authors' calculations based on the 2016 wave of HRS and 2016 CPS ASEC, matched with administrative data from SSA and IRS. We refer to CPS ASEC as CPS. For HRS, the public data come from RAND-HRS public file 2016 version 3 (that includes data from 1992 to 2016). The HRS sample removes respondents in institutions to be consistent with the CPS sample. Estimates are based on all respondents with or without a match. Records that are not linkable use survey values throughout. Estimates are weighted using survey weights. SSA administrative data contain wage and salary earnings from Form W-2, self-employment earnings from Form 1040 Schedule SE, OASDI benefits, and SSI benefits. IRS administrative data contain DB pension income and DC withdrawals (excluding rollovers and conversions) from Form 1099-R and interest and dividends from Form 1040. Income reference year is 2015, the calendar year prior to the interview.

* Reported estimates for CPS data are approved for release by the Census Bureau's Disclosure Review Board (CBDRB- FY24-SEHSD003-010). Analysis of the HRS data is performed at the HRS data enclave at the University of Michigan, and these estimates are approved for release by HRS's Disclosure Review Board in November 2023.

† Pension income includes DC withdrawals but excludes rollovers and conversions.

‡ Information on veterans' benefits is not available in the administrative data and thus the percentages in the CPS do not change across the alternative data files.

^§ Cash public assistance includes SSI, which is available in the SSA administrative data. Therefore the percentages change when we supplement with SSA data.

∥ This income category is not available in administrative data and thus the percentages remain the same across alternative data files.

Besides retirement income, there are a few other statistics from Table 4 that are worth highlighting. First, differences in the prevalence of social security and earnings across the CPS files are relatively small. This reflects the fact that the CPS does reasonably well in measuring social security benefits and earnings at the extensive margin. Second, the HRS shows a somewhat higher proportion of aged persons receiving social security benefits, but a lower proportion with earnings, despite our previous finding of a much higher HRS earnings aggregate.⁴³ Similarly, a higher proportion of the aged population in the HRS than in CPS report receiving income from veterans' benefits and public assistance (which includes SSI payments). The increase in the proportion with earnings and public assistance in the HRS + SSA file compared to the HRS file again indicates extensive margin underreporting of income from these sources. Lastly, the breakdown by gender shows that the similarities and differences across the five files are relatively consistent. Still, it is apparent that, across all data files, women are more likely than men to have family income from social security and from public assistance. In contrast, women are less likely than men to have family income from assets, pensions or retirement accounts, earnings, and veterans' benefits.

4.4 Reliance on social security

We next explore the consequences of income misreporting on social security's measured share of total income across families. Table 5 presents social security reliance statistics for each of the five data files. Paralleling Income of the Aged publications, we restrict the sample to persons in social security beneficiary families (i.e., at least one family member, respondent, or other, is a beneficiary), for whom family earnings and family asset income are non-negative, and family total income and family social security benefits are greater than zero.⁴⁴ We use three indicators of reliance. The first reliance threshold encompasses individuals for whom family social security income comprises 50 percent or more of their total family income. The other two reliance thresholds are 75 percent or more and 90 percent or more.

Table 5. Percentage of individuals aged 65 or older for whom social security income represents a selected proportion of family income, by gender: estimates from five alternative data files, 2015*

Notes: Authors' calculations based on the 2016 wave of HRS and 2016 CPS ASEC, matched with administrative data from SSA and IRS. We refer to CPS ASEC as CPS. For HRS, the public data come from RAND-HRS public file 2016 version 3 (that includes data from 1992 to 2016). The HRS sample removes respondents in institutions to be consistent with the CPS sample. Estimates are based on all respondents with or without a match. Records that are not linkable use survey values throughout. Estimates are weighted using survey weights. SSA administrative data contain wage and salary earnings from Form W-2, self-employment earnings from Form 1040 Schedule SE, OASDI benefits, and SSI benefits. IRS administrative data contain DB pension income and DC withdrawals (excluding rollovers and conversions) from Form 1099-R and interest and dividends from Form 1040. Income reference year is 2015, the calendar year prior to the interview.

* In this table we restrict the sample to respondents in social security beneficiary families for whom family earnings and family asset income are non-negative, and family total income and family social security benefits are greater than zero.

† Reported estimates for CPS data are approved for release by the Census Bureau's Disclosure Review Board (CBDRB- FY24-SEHSD003-010). Analysis of the HRS data is performed at the HRS data enclave at the University of Michigan, and these estimates are approved for release by HRS's Disclosure Review Board in November 2023.

We observe that the CPS produces slightly higher reliance figures than the HRS across the board. For example, the proportion of all persons aged 65 or older relying on social security for 50 percent or more of their family income is 52.6 percent in the CPS compared with 49.1 percent in the HRS. It is worth noting that one reason why the HRS might show lower reliance figures than the CPS is that only the net amount of social security benefits are collected in the HRS, and the benefit amounts deducted for Medicare Parts B and D premiums are excluded (see Dushi et al., 2017).

The four files without IRS administrative data all yield similar results at all reliance thresholds as well as when looking separately by gender. However, when examining the CPS + SSA + IRS file, reliance on social security is considerably lower. Most starkly, the proportion of persons aged 65 or older who rely on social security for 90 percent or more of their family income is 25.6 percent in the CPS and 20.5 percent in the HRS but only 13.7 percent in the CPS + SSA + IRS sample. The difference is mainly attributable to better reporting of retirement income in IRS administrative data. In sum, our findings suggest that income from social security is not as large a share of household budgets as previously thought.

We have also examined the robustness of these reliance patterns by limiting our analysis to the subset of beneficiaries that can be linked to administrative data. Following Bee and Mitchell (2017), we first run a logistic regression to model the probability of SSN assignment as a function of survey demographic, geographic, and economic characteristics and estimate the propensity score. We next construct new weights by multiplying the survey weights by the inverse of the estimated propensity score. We then compute the same statistics for the linked subsample using the newly constructed weights. In both surveys, we find that respondents are only 1–2 percentage points less likely to be reliant on social security than the overall sample, across all reliance indicators and gender. For example, in the CPS + SSA + IRS file, among respondents with a linkage, 40.4 percent of them rely on social security benefits for at least half their family income compared to 41.7 percent in the same file among all respondents (with or without linkages). Respective figures in the HRS + SSA file are 50.5 percent for the matched subsample compared to 51.1 percent for the whole sample.

The CPS reliance patterns for the linked subsample can also be directly compared to previous work by Bee and Mitchell to assess the impact of the survey redesign. Prior to the redesign, Bee and Mitchell found that in 2012, 55.5 percent of survey respondents relied on social security for 50 percent or more of their family incomes while the linked data showed a lower estimate of 42.2 percent. Post-redesign, we find that in 2015, the survey number is 52.5 percent while the linked number is 41.7 percent. Thus, the survey only showed a small decrease post-redesign, and the linked data continue to show a significantly lower reliance on social security.

4.5 Poverty rates

Finally, we examine differences in estimated poverty rates across the five files. Table 6 shows that the poverty rate for all persons aged 65 or older is 8.8 percent in the CPS, 7.7 percent in the CPS + SSA file, and only 6.4 percent in the CPS + SSA + IRS file. Interestingly, the HRS shows a lower poverty rate than the CPS at 7.4 percent, but the HRS + SSA shows a still lower poverty rate of 6.5 percent which is very close to the CPS + SSA + IRS estimate. In both the CPS and the HRS, the standalone surveys considerably overestimate the poverty rate relative to the linked administrative data, indicating income at the bottom of the distribution is underreported. The general finding that poverty rate is lower in the HRS than in the CPS, is not surprising given that we already demonstrated that family incomes are generally higher in the HRS in Table 3.⁴⁵

Table 6. Percentage of individuals aged 65 or older in poverty or near poverty, by selected characteristics: estimates from five alternative data files, 2015

Notes: Authors' calculations based on the 2016 wave of HRS and 2016 CPS ASEC, matched with administrative data from SSA and IRS. We refer to CPS ASEC as CPS. For HRS, the public data come from RAND-HRS public file 2016 version 3 (that includes data from 1992 to 2016). The HRS sample removes respondents in institutions to be consistent with the CPS sample. Estimates are based on all respondents with or without a match. Records that are not linkable use survey values throughout. Estimates are weighted using survey weights. SSA administrative data contain wage and salary earnings from Form W-2, self-employment earnings from Form 1040 Schedule SE, OASDI benefits, and SSI benefits. IRS administrative data contain DB pension income and DC withdrawals (excluding rollovers and conversions) from Form 1099-R and interest and dividends from Form 1040. Poverty measures are based on family income in 2015 and the Census Bureau poverty thresholds corresponding to family size and composition. For the HRS, we use the poverty threshold provided in the RAND-HRS file. HRS total family income is similar to the Census Bureau definition as it includes income the other residing family members (see Appendix B for detailed discussion of poverty measures). To the HRS measure we add IRA withdrawals to be consistent with CPS definition. ‘In poverty’ = with income below 100% of the poverty threshold; ‘near poverty’ = with income below 125% of the poverty threshold.

* Reported estimates for CPS data are approved for release by the Census Bureau's Disclosure Review Board (CBDRB-FY24- SEHSD003-010). Analysis of the HRS data is performed at the HRS data enclave at the University of Michigan, and these estimates are approved for release by HRS's Disclosure Review Board in November 2023.

† For the CPS sample, this category includes respondents identified as ‘Asian alone’, other race, and those who report multi- race categories. However, the majority of CPS respondents in this category are Asian alone.

A closer look across the three CPS files shows that poverty rates in the CPS + SSA file are lower than those in the CPS file across all demographic subgroups. However, in the CPS + SSA + IRS file, poverty rates are substantially lower than those in the other two CPS files for nearly all race/ethnicity categories and age groups. While the overall poverty rate is almost the same in the HRS + SSA and CPS + SSA + IRS files, there are some differences across demographic subgroups. For example, the poverty rate is much higher among non-Hispanic Black and Hispanic elderly in the HRS + SSA file (18.8 and 23.5%, respectively) than in the CPS + SSA + IRS file (11.8 and 15.9%, respectively). In contrast, poverty rates in the two surveys are relatively similar across age, gender, and marital status categories.

We can once again compare our CPS results to those of Bee and Mitchell. For 2012, they reported a CPS poverty rate for 65 or over of 9.1 percent versus the linked data rate of 6.9 percent, whereas we find for 2015 a survey poverty rate of 8.8 percent versus the linked data rate of 6.4 percent. Evidently, the CPS redesign has had little effect on poverty estimates for those 65 or over.

5. Conclusions

Previous research on the aged has shown that survey-reported income, particularly retirement income, is subject to substantial underreporting, leading to downwardly biased estimates of financial well-being. In this paper, we examine whether previously documented income measurement issues are confined to the CPS ASEC or whether they also extend to the well-regarded HRS. Our findings show that total family income, particularly pension and asset income, continue to be significantly underreported in the redesigned CPS ASEC. While the HRS performs somewhat better than the CPS ASEC, we find that it still produces lower estimates of retirement income than those generated by the CPS ASEC linked with administrative data.

We also find that according to the CPS ASEC the primary source of aggregate income among the aged is social security, while the administrative records indicate that retirement income is the largest source of aggregate income. Furthermore, social security reliance indicators are also upwardly biased. Thus, the proportion of people aged 65 and over who rely on social security for at least half their family income decreases from 52.5 percent in the CPS ASEC to 41.7 percent when using administrative data. Similarly, the proportion of the elderly who rely on social security for 90 percent or more of their family income is considerably lower in the administrative data (13.7%) compared to the survey (25.6%). For comparison, 20.5 percent of elderly in the HRS rely on social security for 90 percent or more of their family income, which is lower than the CPS ASEC but still well above the CPS + SSA + IRS file.

Finally, mismeasurement of income leads to an upwardly biased estimate of the poverty rate for those 65 and over. Thus, compared to the CPS ASEC, supplementing the CPS with SSA and IRS administrative data resulted in a reduction in the estimated poverty rate among people aged 65 or older from 8.8 to 6.4 percent. In addition, the HRS indicates somewhat lower estimates of poverty rate than the CPS (7.4% in the survey data and 6.5% in the HRS + SSA). These lower rates are in part due to HRS elderly being less likely to live alone and reporting higher family incomes.

Based on these findings, we conclude that declining data quality is not confined to the CPS ASEC, and surveys linked to administrative records remain the most reliable sources for information about the performance of the U.S. retirement system. Therefore, it is our hope that decision makers at both the SSA and the Census Bureau will facilitate the regular usage of linked survey-administrative data for the continued production of the Income of the Aged publications.