ACA Impact on Uninsured | Survey Estimates: RAND Survey

VII. Key Issues: Regulation & Reform >> C. Health Reform >> Affordable Care Act (ACA) >> ACA Impact Analysis >> ACA Impact on Access >> Impact on Uninsured >> Survey Estimates >> RAND Survey  (last updated 4.19.14)

 

Overview

RAND Corporation researchers have conducted surveys of people in September 2013 — before the ACA enrollment started — and earlier in March, interviewing the same 2,425 adults nationwide both times.

Reduction in Uninsured Adults (9.3m). The uninsured rate among non-elderly adults fell from 20.5% in September to 15.8% in March. Researchers applied this to all non-elderly adults in spring 2013 (198.5m), reporting a net reduction of 9.3 million uninsured (+/- 3.5 million).

ACA Role in Reduction of UI. Researchers caution: “Our survey work can’t say for certain which of these shifts are due to the ACA and which are due to other factors.” Table 3 shows the main results reported here. Note that there is a lot of uncertainty around the smallest figures.

• Exchange Enrollment. 1.4m previously uninsured gained coverage through the Exchanges, representing 36% of the 3.9 covered through the Exchanges.
• Non-group Market. Another 0.5m previously uninsured gained coverage in the non-group market, i.e., off-Exchange, representing 6.4% of the 7.8m in adult off-Exchange enrollments. However, this market shrank from 9.4m. in 2013 to 7.8m. in 2014, of which 0.7m. (44%) remained uninsured in the March survey. An alternative way to view this figure is that of the 4.0m who did not remain in this market in 2014 (switched or lost plans), about 18% remain uninsured.
• Medicaid Enrollment. 3.6m previously uninsured obtained Medicaid, representing 61% of the 5.9m increase in Medicaid rolls between 2013 and 2014; since 1m who previously were on Medicaid became uninsured, Medicaid accounts for 2.6m (28%) of the 9.3 net number of uninsured who gained coverage.
• Employer-Sponsored Insurance (ESI). By far, the largest gains in coverage came from ESI, with 7.2m previously uninsured gaining coverage in this fashion; at the same time, 2.1m who previously had ESI became uninsured, so ESI is responsible for a net decline of 5.1 million uninsured (55%) out of the 9.3m. net number who gained new coverage. In light of the delay in the employer mandate and very light individual mandate penalty in 2014, it seems inappropriate to credit the ACA with this entire gain. That said, it is nearly impossible to determine at this juncture what fraction of these gains in ESI can be attributed specifically to the ACA rather than improving economic conditions. The unemployment rate declined from 7.2 to 6.7% during this same period (monthly unemployment fell by 717,000 between September 2013 and March 2014).

 

Who is Not Counted in RAND Survey?

The RAND figures exclude 3 groups. These exclusions would appear to have underestimated the net reduction in uninsured by a minimum of 1.1 million to as many as 5.1 million. Last Minute Enrollment Surge. The authors state “All HROS data collection reported here ended on March 28, and therefore missed the last three days of the open enrollment period, during which time there was a surge in enrollment.” According to ACASignups, average daily Exchange enrollment was 87,400 during March.

• Lower Bound Estimate. One could argue the survey missed a minimum of 250,000 March enrollments (arguably the last 3 days exceeded the daily average), of which 36% were uninsured, changing the count by less than 100,000.
• Upper Bound Estimate. Alternatively, one could argue that on average, the survey depicts coverage as of 3/14 (i.e., the midpoint of the survey period)  rather than a snapshot of 3/28, and therefore it missed the equivalent of 17 days’ worth of enrollments.  That would imply 534,000 previously uninsured who gained coverage but were missed by the survey.

Children. Any uninsured children who gained coverage during this period would not be included in the RAND counts. Roughly 65% of uninsured children are eligible for Medicaid or CHIP (there were 4 million uninsured children who were Medicaid-eligible in 2011 vs. 6.2 million total uninsured children that year). Census reports 6,586,000 uninsured children in spring 2013 (Table C-3).

• Upper Bound Estimate.  If uninsured children experienced the same relative reduction in uninsured risk (23%) as adults in the RAND sample, it would imply 1.5 million gained coverage. But this seems like an improbable upper limit since these children were Medicaid/CHIP eligible (and could have enrolled at essentially no cost well before January 1, 2014) well prior to the ACA whereas a sizable number of uninsured adults became eligible for coverage that heretofore had been unavailable to them. A more realistic upper bound can be calculated as follows. The resultant total–1.0 million–implies that uninsured risk declined by 15% for children.
  • Exchange Enrollment. A total of 251,256 children had enrolled on the Exchanges as of March 1, 2014; total reported exchange enrollment grew from 4.2 million on March 1 to 7.1 million by March 31, it seems implausible that Exchange-enrolled children grew faster than this, giving us an upper bound estimate of 425,000 children on the Exchanges by March 31. If 36% were uninsured, net uninsured children would have declined by 153,000.
  • ESI Enrollment. ESI cannot account for the same share of coverage gains among children as among adults (i.e., 52% using the upper bound figures from the RAND survey), since 72.9% of uninsured adults are full-time workers (Figure 14) and hence have a direct means of securing ESI that children do not. Assuming 72.9% of adults gaining ESI obtained such ESI directly as workers rather than dependents, we can calculate the ratio of remaining adults who gained ESI to adults who obtained Exchange coverage (i.e., ESI x (1-72.9%)/Exchange) and apply this ratio to the number of children with Exchange coverage to estimate a plausible number of children who gained ESI as dependents: 119,000.
  • Medicaid Enrollment. If enrollments into Medicaid occurred at an equivalent pace, then 153,000 Exchange enrollees under age 17 plus 119,000 gaining ESI should represent 35% of the total newly enrolled in either Medicaid or the Exchanges (i.e., 1 minus the 65% figure cited above). If so, we can impute that 742,000 uninsured children had gained coverage through Medicaid or Exchanges as of March 31.
• Lower Bound Estimate. The total calculated as described below–0.8 million–implies that uninsured risk declined by 11.4% for children.
  • Exchange Enrollment. It seems reasonable to assume that the number of children on the Exchanges would have grown in March by at least the same amount as in February (64,329), yielding a total of 316,000 children on Exchange of which 114,000 (36%) can be expected to be uninsured.
  • ESI Enrollment. The number with ESI can be derived using the ratio of uninsured children on ESI to those with Exchange coverage calculated above, resulting in an estimated 88,000 obtaining ESI.
  • Medicaid Enrollment.  The number on Medicaid can be imputed assuming 114,000 + 88,000 equal 35% of the total number of uninsured children gaining coverage: 551,000.

Young Adult Dependents Under Age 25. The RAND survey only accounts for changes in coverage since September 2013. Thus it would miss anywhere from 0.6m to 2.2m  who had already gained coverage as dependents under their parents’ plan due to the ACA mandate to cover such individuals (see Estimates for Young Adults Age 19-25: the 2.2m. figure is the latest available figure using NHIS but also focusing exclusively on changes in private coverage rather than erroneously including gains in public coverage unrelated to the law).

 

Did RAND Survey Overcount the Number Who Gained Coverage? 

The RAND team has calibrated their figures based on the CPS, although this does not entirely rule out a biased sample. But that won’t be known until other surveys are available to compare and contrast to the RAND figures. There are two possible ways in which the survey might have overestimated the number of uninsured who gained coverage.

Unpaid Exchange Enrollments. A McKinsey survey conducted in February (the latest reported so far) showed that only 53% of uninsured enrollees had paid their first month’s premium, versus 86% for those with coverage. Historically the non-group market had a 10% unpaid rate, meaning that only 90% of those who signed up actually retained their coverage by paying premiums for it.  In light of the generally accepted figure that 80-85% have paid and assuming the previously insured currently have a 90% payment rate (the historical average and virtually identical to the McKinsey survey estimate in any case), then if 36% of Exchange enrollments are uninsured, the implied uninsured payment rate is between 62 and 76%. It remains to be seen whether that is a permanent payment rate or something that will eventually rise to the same level as the previously uninsured. The point is that RAND’s estimates of the number of uninsured who gained coverage on the Exchange may be overstated by as much as 32 to 61% given that those who had enrolled in coverage likely reported themselves covered: such individuals will remain uninsured if they fail to pay their first month’s premium (or even fail to pay later down the road). Excluding such individuals would low the net number newly covered by 0.4m to 0.9m.

Hawthorne Effect. The RAND results are based entirely on the same individuals who have been interviewed at monthly intervals since September 2013. These occurred during a period of massive public attention to the rollout of the Exchanges and intensive interest in how many would gain coverage. The survey asks 2 questions about coverage:

• Do you currently have health insurance that will cover you in 2014?
• Which coverage options have you chosen for health insurance in 2014?

No validation was done on these insurance questions, so it is conceivable some respondents offered the socially acceptable answer that they had signed up for coverage even when they had not (or perhaps intended to).  Until the RAND figures can be compared with other surveys, there is no real way of telling how much this phenomenon might have inflated the estimated reduction in the number of uninsured.

 

Is the RAND Survey Accurate? 

Margin of Error. The margin of error for a survey of this size for a 20% sample proportion with 95% confidence is 1.5%. MOE’s are reported for nearly every calculated figure. The RAND survey has a lot of uncertainty, but an equally important question is whether the results are biased such that their point estimate (expected value) of the number who gained coverage is systematically tilted too high or too low.

Consistency With Other Surveys

The baseline uninsured rate for September 2013 is consistent with other (limited) survey evidence. The size of reported changes in uninsured risk appears somewhat higher than other (also limited) survey evidence.

Baseline Uninsured Rate is Very Consistent with Other Evidence. The spring 2013 CPS found an uninsured rate of 20.8% for those age 19-64 (SHADAC-enhanced estimate), nearly identical to the 20.5% reported by RAND for 18-64 year olds in September (most experts view CPS figures as representing coverage at the time of the survey).  Similarly, Gallup reports a rate of 18.0% for all adults in 3rd Q 2013; given that elderly people have an uninsured rate of only 1.5%, the implied rate for 18-64 year olds is 21.7%.

Change in Uninsured Risk Somewhat Consistent with Other Evidence. The Gallup figures show an uninsured rate of 15% in March, which represents a 17% decline from the 2013 Q3 figure of 18%; this is somewhat lower than the RAND figures implying a 23% decline.

Evidence of Bias That Understates Gains in Coverage

The RAND estimate of gains in Exchange coverage appears understated by about 1.4m. If true, this is not proof that the net number who gained coverage also is understated. It may simply be that RAND has inaccurately estimated the source of such gains in coverage, e.g., by overestimating the number who gain ESI and underestimating Exchange enrollment.

Exchange Count Appears Understated. The RAND figures show 3.9m  (+/-1.1m) enrolled on the Exchanges. Using the same logic described above [Who is Not Counted? Last Minute Enrollment Surge], the survey would have missed 1.5 million enrollments occurring between March 15-31, yielding an adjusted figure of 5.4m. But the administration has reported a figure of 7.1 million Exchange enrollments as of March 31.  It is unlikely that all of the “missing” enrollees (24% of the 7.1m) are children, since children constitute only 14% of the uninsured (SHADAC-enhanced estimate), 65% of uninsured children are Medicaid/CHIP-eligible (see above) and only 6% of Exchange enrollments through February were children. Thus, the RAND figures appear understated by 1.4m. (i.e., 5.4m./.94=5.7m.), which is outside their margin of error.

Evidence of Bias That Overstates Gains in Coverage

Counts Are Overstated by 1%. The authors used sample weights to ensure that the sample is representative of the population, benchmarking to the Current Population Survey. They then used the weighted percentage of respondents from the survey multiplied by the 2013 total resident population between the ages of 18 and 64 (198.5 million, evidently derived by subtracting children and elderly from a 2013 resident population figure of 316.1m.) to extrapolate to the national level However, CPS itself is based on the civilian noninstitutionalized population, which in March 2014 was 312.4m., so all the RAND counts are overstated by just over 1% (correcting this would reduce the estimated reduction in uninsured by less than 100,000).

Sampling Frame Potentially Biased.  The survey uses the the American Life Panel (ALP), a group of more than 5,000 individuals ages 18 and over who have agreed to participate in occasional online surveys. A typical interview takes no more than 30 minutes, and respondents are paid an incentive to participate. Many of the respondents use their own computer to log on to the Internet and participate in the surveys. However, the ALP also includes respondents without Internet access, by providing them either a small laptop or a Web TV that allows them to access the Internet using their television and a telephone line. The technology allows respondents who did not have previous Internet access or a computer to participate in the panel; they can also use their laptop or Web TV to access the Internet and use email when not participating in a panel. In principle, this could bias the survey towards low income respondents. However, “data are weighted to match the age, sex, race/ethnicity, education, and income distribution of the 2012 March Supplement of the Current Population Survey (CPS). We also match the joint bivariate distributions of race and sex and education and sex” [Footnote 5]. This weighting should largely offset the effects of inadvertently over-sampling low income adults.

Medicaid Count Appears Overstated. RAND reports a net increase of 5.9m (+/-2.8m) in Medicaid, a count that excludes children. If the estimated number of uninsured children who gained Medicaid are included (see above), these figures would increase to 6.5m (4.7m-9.4m.); this is a conservative estimate since there also would have been previously insured children who also gained Medicaid during this period.

• CMS Estimates. Yet CMS has just reported that the net increase in Medicaid between October 2013 and February 2014 was only 3.0 million, inclusive of children. This is based on comparing the average monthly enrollment from July through September 2013 to total Medicaid/CHIP enrollment figures for February. This count excluded a handful of states that did not report data and obviously missed all of March. But even if March Medicaid enrollments were double the average monthly rate for the 5 months reported, this would only increase the net number to 4.2 million, a figure below even RAND’s lower bound.
• Avalere Health Estimates. Using a similar method to avoid counting routine re-enrollments, Avalere Health likewise reported between 2.4 to 3.5 million newly enrolled in Medicaid due to the ACA between October 2013 through January 2013. Again, if this is extrapolated through March, doubling average monthly enrollments for March, the total would be 4.2 to 6.1m inclusive of children. All these figures may or may not be within RAND’s margin of error depending on what assumed fraction of new enrollments are children.

ESI Growth Appears Implausible. RAND reports a net increase of 8.2m (+/-3.6m) in ESI.

• CBO Predicts No ESI Growth in 2014. In contrast, the CBO projected no net change in ESI during 2014 and a decline in ESI in both 2015 and 2016 (even though those are the first two years that the employer mandate will be in effect).
• Sharp Deviation From Historical Pattern. The CPS shows the ESI for non-elderly adults was 115.7m in spring 2011 and 115.4m in both spring 2012 and 2013 (calculated from Table C-3). Thus, an increase of 7.5m (7.1%) in just six months would be extremely unusual.
• ESI Growth Under Massachusetts Health Reform Much Slower. A 2013 PWC study showed that employees who got insurance through their jobs did increase in Massachusetts from 70.8% in 2006 to 72.1% in 2011 even as the national rate was dropping from 68.2% to 58.3%. Nevertheless, this very modest 1.8% increase over 5 years contrasts sharply with the 7.1% growth in ESI over just 6 months implied by the RAND survey.
• Improved Economy Appears to Account for Only Small Share of ESI Growth. Economic conditions admittedly improved, however, monthly unemployment fell by only 717,000 between September 2013 and March 2014. Total employees on nonfarm payrolls were 137,928,000 in March 2014 vs. 137,037,000 in September 2013, which essentially provides a similar picture. Even if every one of these 900,000 added workers gained ESI, it would leave an increase of 7.3m. to explain.
• Implausibly Large Increase in Take-up Rate for Offered ESI Coverage. Since the employer mandate was delayed and the first year individual mandate penalty is so light ($95 or 1% of income, whichever is higher), it seems improbable that this was sufficient inducement to convince 7.3m additional eligible-but-not-enrolled workers to sign up for offered ESI benefits. In 2013, 80% of eligible workers enrolled in offered ESI coverage. With 115m non-elderly adults covered by ESI, this implies roughly 29m who did not elect to take up offered coverage. So 7.3m implies that 25% of them changed their minds.

Net Assessment: What Was the Net Reduction in Uninsured Numbers by March 31, 2014?

Net Reduction in Uninsured Through March 31, 2014

The table below shows what happens once we adjust the RAND figures to add people who were excluded and subtract those who were overcounted. This raises the estimated reduction in the count of uninsured people to 11.2 million—a figure that could be as low as 8.1 million or as high as 14.2 million.

 

ACA-Attributable Net Reduction in Uninsured Through March 31, 2014

But these figures do not show how much the ACA can be credited with these reductions. The RAND study authors freely conceded “our survey work can’t say for certain which of these shifts are due to the ACA and which are due to other factors.”  Medicaid, the Exchanges and all in the People Not Counted group presumably can be chalked up to the ACA, but together only account for 59% of the 11.1 million. The big unknown is ESI, which accounts for 46% of the adjusted reduction in uninsured people. As noted earlier, it seems highly implausible all or even most of this astonishing (and puzzling) expansion of ESI, if indeed it actually occurred.

• Lower Bound Estimate. If we assume that ¼ of ESI gains can be attributed to the ACA, then the net gain in coverage due to ACA is 7.3m (4.9M-9.7m).
• Upper Bound Estimate. If we assume ¾ of ESI gains were due to the ACA, the net gain rises to 9.9m (7.0m-12.7m).