Government health insurance for people below poverty line in India: quasi-experimental evaluation of insurance and health outcomes
BMJ 2014; 349 doi: https://doi.org/10.1136/bmj.g5114 (Published 25 September 2014) Cite this as: BMJ 2014;349:g5114
All rapid responses
Rapid responses are electronic comments to the editor. They enable our users to debate issues raised in articles published on bmj.com. A rapid response is first posted online. If you need the URL (web address) of an individual response, simply click on the response headline and copy the URL from the browser window. A proportion of responses will, after editing, be published online and in the print journal as letters, which are indexed in PubMed. Rapid responses are not indexed in PubMed and they are not journal articles. The BMJ reserves the right to remove responses which are being wilfully misrepresented as published articles or when it is brought to our attention that a response spreads misinformation.
From March 2022, the word limit for rapid responses will be 600 words not including references and author details. We will no longer post responses that exceed this limit.
The word limit for letters selected from posted responses remains 300 words.
This a response to the comments posted by James D Shelton on our study that evaluated the impact of Vajpayee Aarogyashree Scheme (VAS). We agree with the posted comments on several dimensions including:
1. We agree that our study and its conclusions would be strengthened if we had detailed information on the procedures performed during a hospitalization. In addition to data on procedures performed we would also have liked to have detailed data on the clinical history of patients and presenting conditions. For example, a bypass surgery can be life saving for someone with severe coronary artery disease but can have little effect on health for someone with mild disease. Without such detailed information on procedures, clinical history and presenting conditions for patients in the treatment and comparison areas it is not possible to judge the clinical reasonableness of the study findings. We did not collect such information as it is difficult to collect such information using household surveys and a full blown medical record abstraction was beyond the scope of the current study.
2. We also agree that measurement error in the cause of death is a limitation of this study. A fuller discussion of this issue is presented in the paper and online appendix. However, we note that it is not appropriate to compare the age of death distribution in the comparison area with estimates of the all India age of death distribution for several reasons. First, we survey rural households and government reports suggest large urban-rural disparities in death rates in Karnataka, with death rate for rural residents being about 50% higher than death rate for urban residents. We also survey poor or BPL households and prior studies suggest higher rates of premature mortality for poor households. Finally, in the treatment area, roughly half the deaths occurred below age 60 which matches well with mortality data from Karnataka as reported the government of India.
3. We agree with the comment that a program that supports only tertiary care and does not have a well-functioning system for referring patients to tertiary care would have a much smaller impact on health. As we note in the paper the effects of the program we observe arise from the combination of outreach via health camps and free access to tertiary care at public and private hospitals. Also, outreach via health camps is part of other government sponsored programs in India. The outreach via health camps might lead to greater and potentially earlier diagnosis of disease. Some of the diagnosed patients would seek medical management of their health conditions while others would be appropriate for tertiary care. The program also had a pre-authorization process to screen patients and reduce inappropriate use of care. Patients with better access to tertiary care might be more likely to seek formal medical care for diagnosis of symptoms potentially requiring tertiary care. So the program could benefit patient health through multiple channels including greater access to tertiary care, reduction in undiagnosed disease, earlier diagnosis and treatment, and changes in appropriate use of tertiary care or quality of care more broadly.
4. We also agree and note in the paper that the effects we observe might be larger than true longer term effects of the program due to “pent up” demand for tertiary care. In the long run, the effects would be smaller as the program would target incident rather than prevalent cases. However, we believe that the burden of incident cases of non-communicable diseases is non-trivial and a well-functioning health care system should address this burden of illness.
Although we agree with several comments in the rapid response, not surprisingly, we do not agree with the assessment that the results of our study are implausible. We think it is certainly plausible that a program that provides access to expensive tertiary care (and access to specialists via health camps) for life threatening conditions to poor patients can have meaningful impact on patient health. However, this does not imply that we should adopt this or other similar programs at the expense of primary care interventions. In fact, instead of pitting primary care versus tertiary care the right question is: what is the optimal portfolio of primary, secondary and tertiary care interventions for addressing the burden of illness of a population? The answer to this question is difficult and would require cost-benefit or cost-effectiveness analysis of alternate portfolio of strategies. Future research should focus on this important question.
Competing interests: No competing interests
The asserted impact to reduce adult mortality by 64% by the Valpayee Insurance scheme for tertiary care services in India even for “covered conditions” – mainly cardiac and cancer, as described by Sood et al,1 is simply not credible.
1. We don’t know what health services were actually provided. For the entire 487 households which had hospital admissions for the conditions covered by the plan, our only inkling is a wide-ranging list of 402 covered “procedures” laid out in the appendix. To be really credible, an argument for causality needs to examine the causal pathway, not just the final outcome.
2. Specific examination of those 402 covered services, indicates such a population-level impact of close to 2/3 is highly untenable. The covered services were mainly relatively sophisticated procedures such as cardiac surgery including coronary by-pass surgery, other vascular, brain, thoracic and surgery as well as cancer chemotherapy, radiation, cancer surgery, burn treatment and fairly simple trauma surgery. With the possible exception of severe burn treatment, the medical literature just does not support a decrease in mortality of 2/3 for such procedures among those treated, let alone extending such a level of impact to the entire population of people from whom these patients were drawn.
3. The program didn’t address key primary care interventions with substantial impact on cardiac and cancer mortality, notably high blood pressure, diabetes, heart failure and lifestyle factors such as tobacco, alcohol and diet.
4. The pattern of mortality by age in the comparison area is rather bizarre. As shown in the graph below of the proportion distribution of deaths for all of India, numbers of deaths should peak at about age 70.2 If anything, deaths due to cardiac and cancer should peak even later. As shown in the author’s Figure 4, deaths in the eligible area did peak at about 70, but in the comparison area, they found a very sharp and odd-looking peak in the late 50’s. Thus comparisons made to such an unreliable comparator may well be misleading.
Among various possibilities for these discrepancies, all this suggests some major problems with the measurement of mortality including the verbal autopsy process. We are told little about the process, other than households identified cause of death from a list of 33 lay-translated causes. But verbal autopsy is clearly tricky and susceptible to misclassifications especially for cardiac conditions.
Finally, this insurance scheme was highly atypical in that it included major outreach camps that actually recruited the majority of patients for the hospital services. As the authors allude to, such an outreach process could in a sense pick off the most prevalent instances of remediable conditions, so that any later impact would be largely attenuated. Moreover, such outreach must add substantially to the cost, and is unlikely to be including in actual practical insurance application. Likewise we are not told the cost of these highly sophisticated hospital interventions, though they are described as “costly.” Accordingly in our view, health funding in India and in other low and mid income countries is better spent on other intervention priorities.
References.
1. Sood N, Bendavid E, Mukherji A, Wagner Z ,Nagpal S , Mullen P. Government health insurance for people below poverty line in India: quasi-experimental evaluation of insurance and health outcomes. BMJ 2014;349:g5114.
2. UN Population Division mortality estimates for India 2005-2010. http://esa.un.org/unpd/wpp/Excel-Data/mortality.htm Accessed 17 October, 2014.
Competing interests: No competing interests
The present study shows the health care benefits of an insurance programme in Karnataka, a south Indian state. However, we need to look at some aspects of the study before accepting the results.
An information on how 33 causes for hospitalization were derived at needs detailed explanation since there could be variations in the population regarding morbidity and mortality. And only two-thirds of the study population could tell the self reported cause of admission in the hospital can possibly lead to reporting bias due to a large number of people not reporting the conditions and their characteristics might be different from responders. A clarification regarding this may be given so that possible bias could be eliminated. It is also essential to know how may health centres, tertiary care hospitals had reports of the patients shown in their hospitals who were enrolled for the Vajpayee Arogyashree Scheme (VAS). In general, out patient records in the general government hospitals are not complete and details about the patient records for routine out patient visits might not be available. In this context, it is essential to know about the veracity, quality of records maintained in the empanelled hospitals. Did the investigators checked on this aspect and what is their observation ?
An important indicator is the cause of death used in the study. Was it based on the available records or other methods such as verbal autopsy or combination needs clarity ? If it is based on available records, how many patients had actually had a death certificate certifying the cause of death and what were the proportion of causes of death available in the insured and non insured regions ? Usually, the cause of death report is of much concern and the reporting is incomplete in all the states of India. It is also essential to know the pattern of causes of death in the insured and non insured regions and find out which causes could have been averted or preventable from the perspective of health care delivery system improvements in services. This could be mentioned in the discussion. There are limitations to verbal autopsy reported deaths as well. Verbal autopsy should not be considered as a Gold standard and its limitations should be considered due to reporting bias of respondents, and cause of death ascertainment by coders. Sometimes, the information given by respondents are low and cannot predict the cause of death. Such possible bias should be considered before accepting the results.
Competing interests: No competing interests
Are the mortality effects of health insurance in India credible?
Sood and colleagues are to be commended on the ambition of their evaluation of a government health insurance scheme for people below the poverty line (BPL) in Karnataka, India.1 With some notable exceptions,2 3 there are few opportunities to exploit randomisation in the evaluation of large-scale health financing reforms and thus quasi experimental methods must be relied upon. In this regard, the increasing number of publications in medical journals applying econometric methods to study impacts of such schemes are to be welcomed. Nevertheless, the findings of this study should be interpreted with a number of limitations in mind, particularly given the impressive magnitude of the mortality effect reported.
First, the empirical approach used in the study is more accurately described as matching, the limitations of which are well known.4 A regression discontinuity design that uses a geographical boundary as the assignment variable is based on the idea that there is a discontinuity in the treatment variable (insurance coverage) either side of a “random” boundary but no other discontinuities in relevant variables (household characteristics, community factors, policy environment) conditional on distance to the boundary.5 The study neither demonstrates the discontinuity in insurance coverage6 despite investment in primary data collection nor does it control for distance or geographical location in any way.7 8
Second, the geographical border is far from “arbitrary.” The paper suggests that the insurance scheme was implemented in such a way that the boundary in fact divides different districts. We must therefore ask “what are all the things differing between the two regions other than the treatment of interest?”5 In India, districts are the administrative unit charged with the implementation of health policy, not to mention the administration of the BPL system, and variation in such implementation (due to differences in management, leadership, budget, health personnel etc) must be expected. Yet the analysis is unable to control for such differences between districts. A more credible alternative with the available data would be a difference-in-difference approach – that is, a comparison of poor and non-poor households between treatment and control. It would provide an opportunity to control for geographical location.
Finally, it is hard to reconcile the large mortality effect with the modest (sometimes borderline significant) utilization results. Without substantive evidence of a large increase in hospitalization, it is premature to read too much into the mortality finding.
1. Sood N, Bendavid E, Mukherji A, Wagner Z, Nagpal S, Mullen P. Government health insurance for people below poverty line in India: quasi-experimental evaluation of insurance and health outcomes. BMJ 2014;349:g5114.
2. Baicker K, Taubman SL, Allen HL, Bernstein M, Gruber JH, Newhouse JP, et al. The Oregon experiment--effects of Medicaid on clinical outcomes. N Engl J Med 2013;368(18):1713-22.
3. King G, Gakidou E, Imai K, Lakin J, Moore RT, Nall C, et al. Public policy for the poor? A randomised assessment of the Mexican universal health insurance programme. Lancet 2009;373(9673):1447-54.
4. Imbens GW, Wooldridge JM. Recent Developments in the Econometrics of Program Evaluation. Journal of Economic Literature 2009;47(1):5-86.
5. Lee DS, Lemieux T. Regression Discontinuity Designs in Economics. Journal of Economic Literature 2010;48(2):281-355.
6. Card D, Dobkin C, Maestas N. Does Medicare Save Lives? Quarterly Journal of Economics 2009;124(2):597-636.
7. Black SE. Do Better Schools Matter? Parental Valuation of Elementary Education. Quarterly Journal of Economics 1999;114(2):577-99.
8. Dell M. The Persistent Effects of Peru's Mining Mita. Econometrica 2010;78(6):1863-1903.
Competing interests: No competing interests