Abstract

In 2007, UNAIDS corrected estimates of global HIV prevalence downward from 40 million to 33 million based on a methodological shift from sentinel surveillance to population-based surveys. Since then, population-based surveys are considered the gold standard for estimating HIV prevalence. However, prevalence rates based on representative surveys may be biased because of nonresponse. This article investigates one potential source of nonresponse bias: refusal to participate in the HIV test. We use the identity of randomly assigned interviewers to identify the participation effect and estimate HIV prevalence rates corrected for unobservable characteristics with a Heckman selection model. The analysis is based on a survey of 1,992 individuals in urban Namibia, which included an HIV test. We find that the bias resulting from refusal is not significant for the overall sample. However, a detailed analysis using kernel density estimates shows that the bias is substantial for the younger and the poorer population. Nonparticipants in these subsamples are estimated to be three times more likely to be HIV-positive than participants. The difference is particularly pronounced for women. Prevalence rates that ignore this selection effect may be seriously biased for specific target groups, leading to misallocation of resources for prevention and treatment.

Introduction

Correct estimates of disease prevalence are essential for adequate monitoring and planning of public policies and for the most efficient allocation of resources for health care. In their annual report on the global prevalence of HIV/AIDS in 2007, UNAIDS revised the estimated number of global HIV infections downward from 40 million to 33 million after the inclusion of household-based surveys in the estimation (UNAIDS 2007). Previously, HIV prevalence estimates were largely based on surveillance among certain sentinel groups, such as women who visit public antenatal clinics (ANCs). It was well known that the data thus obtained could hardly be seen as representative for the entire population, but they were, by and large, the only data available (Walker et al. 2004).

More recently, a substantial number of national household surveys have included HIV tests. In particular, Demographic and Health Surveys that include testing for HIV (DHS+ surveys) are now available for at least 31 countries and are ongoing in several more.1 Generally, DHS+ surveys show lower estimates of HIV prevalence than the ones based on ANC sentinel surveillance (Gouws et al. 2008; Montana et al. 2008).

Population-based nationally representative household surveys with HIV testing are currently considered to be the gold standard for HIV prevalence estimations (Mishra et al. 2006), replacing sentinel surveillance data whenever available. Although they represent a significant improvement over extrapolations from ANC-based observations, a number of problems remain (Boerma et al. 2003). There are potentially three major sources of selection bias: (1) nonparticipation in the survey because of absence, (2) unwillingness to participate in any part of the survey, and (3) participation in the survey but refusal to take the HIV test.2 If nonresponse is systematically correlated with the risk of HIV infection, the estimated HIV prevalence will be biased.

A substantial literature has focused on the first source of bias, showing that HIV infection is more prevalent in mobile individuals (e.g., Lydié et al. 2004; Marston et al. 2008; Pison et al. 1993). Much less is known about the second source of bias, which is problematic to identify because data for these individuals are lacking by definition. The focus of this article will be solely on the third source of bias: refusal to participate in the HIV test.

The nonresponse rate in DHS+ surveys resulting from refusal to take the HIV test ranges from 4.4 % to 15.7 % (Mishra et al. 2006). UNAIDS has suggested that corrections can be made using a multivariate logistic regression model based on observable socioeconomic characteristics and sexual behavior indicators, if available (UNAIDS/WHO 2005). Such analyses typically find a negligible effect for adjusted national prevalence rates (Marston et al. 2008; McNaghten et al. 2007; Mishra et al. 2006, 2008).

However, these studies do not take into account potential unobserved differences between individuals, such as knowledge about one’s HIV status. The resulting bias can be large: García-Calleja et al. (2006) calculated that if nonresponders would have twice the HIV prevalence of those who participate, this could result in adjusted HIV rates that are 1.03 (for Rwanda) up to 1.34 (for South Africa) times higher than observed. Reniers and Eaton (2009) estimated the relationship between prior knowledge of HIV status and subsequent refusal to participate in an HIV test in Malawi. They calculated that the underestimation in national prevalence rates can be as large as 13 % and even more than 20 % in urban areas. Using a similar but more extensive approach, Floyd et al. (2013) analyzed refusal bias in a setting in Malawi, where about four-fifths of the population had tested for HIV at least once, exacerbating refusal rates. They estimated that the observed prevalence rates were biased downward by as much as 38 % for men and 25 % for women.3

In this article, we use a Heckman-type selection model with interviewer identities as instrumental variables, exploiting their random assignment to households in the sample to identify the participation selection effect. The article adds to the literature in four respects. First, to our knowledge, our study is the first to correct for refusal bias using truly random instruments. Four other studies have been published that use a Heckman selection model to correct for unobserved characteristics of nonrespondents. Studies by Bärnighausen et al. (2011) and Hogan et al. (2012) are most similar to ours. They also used interviewer codes as instruments. However, interviewers in these surveys were generally assigned to respondents based on sex, province, and language, which can introduce biases even when these factors are controlled for in the regressions. Reniers et al. (2009) quantified refusal bias in HIV prevalence estimates in a clinic-based study in Ethiopia, using nonrandomly assigned interviewer codes as well as ward of admission (including the gynecology ward) and respondent education as instruments. These last two variables are likely to have a direct relationship with HIV status. Lachaud (2007) used labor market participation and rural/urban residence to identify the selection effect in Burkina Faso: again, two variables that can be related to the likelihood of HIV infection.

Second, we calculate confidence intervals for HIV prevalence taking into account that the population prevalence rate is derived from individual predictions. The individual predictions themselves are estimates with a corresponding standard error. Therefore, standard errors of the population prevalence rate should be estimated by, for example, bootstrapping or the delta method (Davidson and MacKinnon 2004). HIV prevalence estimates in most other studies are not corrected for this uncertainty leading to an underestimation of confidence intervals. The study by Hogan et al. (2012) is an exception. They approximated the distribution of predicted prevalence for individuals without HIV test results and with HIV test results separately, by simulations from multivariate normal distributions. They then introduced a correlation between these distributions by the so-called copula method, for which bootstrapped correlation coefficients are used. However, this method is computationally very intensive and, as noted by the authors, makes strong distributional assumptions. By contrast, the delta method we use is easy to compute and does not make such assumptions.

Our third contribution is that we explore in more detail the characteristics of nonparticipants and their relative contribution to HIV prevalence estimates. Kernel density plots of the predicted likelihood of infection show that nonparticipants are overrepresented both in the high-risk and low-risk regions. This implies that bias resulting from nonresponse can both be positive and negative, depending on the risk profile of refusers.

Finally, we are able to test the validity of our estimation results by taking advantage of a follow-up survey among the same households, which includes HIV testing and allows for HIV incidence estimations (Aulagnier et al. 2011).

The next section describes the data set. Subsequently, we compare participation rates and response rates in the sample to gain a first qualitative understanding of potential refusal bias. Next, we describe the econometric specification. Results of both a simple probit and the Heckman model are then presented. Results are validated using follow-up data. We subsequently explore the characteristics of nonrespondents in more detail. Finally, we conclude.

Description of the Data Set

In 2006, a longitudinal household survey was initiated in Windhoek, the capital of Namibia, with follow-up surveys in 2008 and 2009. This survey was the first and, at that time, the only one in the country to include an anonymous HIV test. The 2006 baseline survey consisted of an extensive socioeconomic questionnaire at the end of 2006 and a biomedical component, including an HIV test, in early 2007. The two components were not conducted simultaneously because of lengthy ethical clearance procedures and validation of the oral HIV test, which was new in the country (Hamers et al. 2008).

The voluntary, anonymous, and saliva-based HIV test was administered by trained nurses on all individuals aged 12 and older. Parental consent was sought in addition to children’s consent for those between ages 12 and 18. Participants did not receive their HIV test results because HIV status should not be diagnosed on the basis of a single test. Those interested in learning their HIV status were referred to the standard voluntary testing and counseling centers. Ethical clearance was obtained from the Namibian Ministry of Health and Social Services.

The socioeconomic component of the baseline survey was based on a representative clustered sample of 1,769 households in 99 primary sampling units (PSUs) that were randomly sampled proportional to size. It included 5,639 eligible individuals age 12 and older. Unfortunately, only 53 % of them (i.e., 2,964 respondents) could be tracked and interviewed for the biomedical component of the baseline, primarily because of its coincidence with a large-scale relocation project in Windhoek.4

Attrition was not random. Respondents lost between the socioeconomic and biomedical baseline survey were, on average, younger, more affluent, and better educated. There may have been other, unobserved correlates with attrition. Therefore, the corrected estimates of HIV prevalence rates in this article will not be representative of the population as a whole but only of the subsample of individuals interviewed in both the socioeconomic and the biomedical survey. Because of the lack of appropriate instruments, we cannot correct for the nonrandom absence of individuals in our sample, despite the availability of information on their socioeconomic characteristics.

The participation rate in the HIV test was 82.4 % of the individuals who were present during the biomedical visit. Of the individuals who were asked to participate in the test, 13.8 % refused. An additional 3.8 % of the observations could not be used for analysis because of mislabeling or technical problems in the field or in the laboratory. The analysis does not include individuals younger than 18 years because their participation in the HIV test depended not only on their own consent but also on their parents’ or guardians’ consent. The total sample includes 2,031 individuals aged 18 and older. The analysis is restricted to the 1,992 individuals with complete information on all explanatory variables used in the estimations excluding HIV status: 1,710 participants and 282 nonparticipants in the test.

Observed HIV Prevalence and Response Rates

Columns 5–8 of Table 1 show the observed rates of HIV prevalence for the participants in the HIV test by demographic and socioeconomic characteristics. The sample’s overall prevalence rate is 10.5 %. The HIV infection rate is higher for women than for men (11.8 % vs. 8.9 %), but the difference is not statistically significant (p value = .117). The age distribution of HIV infection follows the standard bell-shaped pattern, with women being at risk at an earlier age (p value = .004) and men following about five years later. As in other African settings (De Walque 2009), prevalence rates in this urban sample appear lower at the highest education levels for both men and women. Among the poorest 40 % of the sample, rates are more than twice as high as among the top 40 % of the consumption distribution. Looking at the wealth index,5 differences between the poor and the rich become even more pronounced. These findings are in contrast with those of other studies that have found either positive or insignificant relationships between socioeconomic status and HIV prevalence (Fortson 2008; Lachaud 2007; Mishra et al. 2007).

Columns 9–12 of Table 1 show a 14.2 % nonresponse rate resulting from refusal. Nonresponse is significantly higher for males than for females (16.3 % vs. 12.4 %; p value = .018). The table also shows that nonparticipation is higher among the older population, which is less likely to be infected with HIV. Refusal rates show a steady rise with increasing per capita consumption and wealth among females. For males, the relationship between refusal rates and per capita consumption or wealth is nonlinear.

Table 2 shows a number of HIV-related variables in relation to participation and HIV status in order to shed more light on the selectivity of nonresponse. The first variable in panel A is the HIV/AIDS knowledge score. This score measures the number of correct answers to the question, “How can you get AIDS?” It ranges from 0 to a maximum of 3. As expected, individuals with better knowledge on how to prevent HIV infection are significantly less likely to be infected. They are also significantly more likely to participate in the HIV test.

The second set of variables in panel A consists of six biomarkers that are related to HIV infection: having coughed for more than a week in the past 12 months, ever had an X-ray taken of the lungs, ever had an AFB test for tuberculosis, ever been diagnosed with tuberculosis, lost weight in the past 12 months, and suffered from diarrhea in the past four weeks. All biomarkers are positively correlated with HIV infection and negatively correlated with test participation, although not always significantly. The correlation of biomarkers with participation is more pronounced for women. Panel B shows a biomarker score derived from a factor analysis that combines the six individual markers. Its correlation with HIV infection and participation is large and significant. It confirms that the biomarkers are strongly and positively correlated with HIV infection rates but negatively correlated with test participation rates, especially for women.

Finally, Table 2 includes information on stigmatizing attitudes regarding HIV/AIDS: whether the respondent would not buy food from an HIV-positive shopkeeper, would not kiss an HIV-positive person, or would not take care of an HIV-positive person (panel A). These attitudes are not related to HIV infection, but they are strong predictors of refusal to participate in the HIV test. The stigma score derived from a factor analysis of the three attitudes confirms these relationships (panel B).

Econometric Specification

Heckman-Type Selection Model

To correct for bias resulting from unobserved characteristics, we use a Heckman-type selection model, which is defined as follows (Van de Ven and Van Praag 1981):
formula
(1)
formula
(2)
where and are latent variables of individual HIV status and participation in the HIV test, respectively. Contained in the vector Xi are a number of demographic and socioeconomic characteristics (gender, age, age squared, marital status, education level, employment status, household size, number of children, log per capita consumption, and asset-based wealth), variables that capture individual risk of HIV infection (biomarkers and HIV knowledge), stigmatizing attitudes, and neighborhood dummy variables.

To identify the selection effect, the vector Zi consists of the identity (ID) codes of the nurses who administered the HIV test in the biomedical survey. These variables are not included in the main equation. Although these nurses can influence the participation rate (some of them are more persuasive than others), they cannot influence the outcome of the test directly, and they are randomly assigned to households, such that there is no systematic correlation between nurse assignment and HIV status.

The error terms and are assumed to be bivariate normal, such that , , and for all i. Because of the survey design, we take into account clustering of the error terms at the PSU level to account for intracluster correlations. When , selection is dependent on HIV status, and estimating the main equation independently from the selection equation can bias the estimates.

Calculating Correct Point Estimates and Confidence Intervals

Because , it follows from Eq. (1) that
formula
(3)
where is the cumulative standard normal distribution. After the Heckman model is estimated using maximum likelihood, can be calculated for all N individuals in the sample (participants as well as nonparticipants in the test), so that is the estimate of the HIV prevalence rate.
When constructing the confidence interval of , one should account for the uncertainty in , for example by using the delta method or by bootstrapping. The delta method is in essence a chain rule specifying how the variance of a vector of parameters can be transformed into the variance of a function of . Suppose that is a vector of continuously differentiable monotonic functions of . The delta method shows that the covariance matrix of , , can be consistently estimated by
formula
(4)
where the matrix is of the form , and is the estimated covariance matrix of (Davidson and MacKinnon 2004).6

Because is such a function of , we can use the delta method to construct the confidence interval around the HIV prevalence estimate. Standard statistical methods that do not take into account the uncertainty in will report underestimated confidence intervals.

Using Eqs. (1) and (2), we can derive the probabilities of HIV infection conditional on test participation:
formula
(5)
formula
(6)
where is the cumulative bivariate standard normal distribution. Conditional estimates of HIV prevalence and their confidence intervals are calculated in line with the methods described earlier for the unconditional estimates and standard errors. Note that the uncertainty in the unconditional Heckman estimate of P(HIVi = 1) depends on the uncertainty of the parameter only (Eq. (3)). On the other hand, the uncertainty in the conditional Heckman estimates depends on the uncertainty of all estimated parameters in the Heckman model: namely , , and (Eqs. (5) and (6)).

Nurse ID Codes as Instrumental Variables

To identify the participation effect, we use the nurse ID codes. To be valid instruments, they must be nontrivially correlated with participation and should not be correlated with the HIV status of the respondent other than through the participation effect. We will discuss these two assumptions in turn.

It seems likely that some nurses were more capable of motivating individuals to participate in the test than other nurses. Indeed, abundant evidence exists on the relationship between response rates and interviewer identity (Durrant et al. 2010; O’Muircheartaigh and Campanelli 1999). Table 3 shows that participation rates among the seven interviewing nurses exhibit significant differences (final row). Four of the seven correlation coefficients between the participation dummy variable and the nurse-identifier dummy variables are significant at the 1 % level, and all are significant at the 10 % level (columns 3–4). Column 5 shows the coefficients of nurse ID codes in a probit regression of the likelihood of test participation controlling for other characteristics as described in the section Heckman-Type Selection Model. The coefficients are jointly highly significant, with p value < .001.

In addition, the supervisor of the nurses was asked to assess their interview skills (columns 7–8, Table 3). The subjective performance scores correlate perfectly with the participation rates, except for those of nurse F. The supervisor assessment in column 7 explains that because of the low performance of nurse F, the supervisor went back to her respondents and increased participation rates in the HIV test. From these results, we conclude that the nurse ID codes fulfill the first requirement (nontrivial correlation with participation) for being used as instruments in the participation equation.7

The second condition requires that the nurse ID codes are not systematically correlated with HIV status other than through their effect on participation. A priori, it seems highly unlikely that individual nurses can affect the outcome of the HIV test. Respondents self-administer the test by rubbing a toothbrush-like test instrument over their gums. This procedure is easy to explain and carry out. The “toothbrush” is subsequently put in a test tube, labeled with an ID sticker, and sent to the laboratory, where the test is conducted by lab technicians. Test results are matched to the socioeconomic and self-reported health data after all identifying information is deleted from the database. Hence, apart from obvious fraudulent behavior (Janssens et al. 2010), nurses cannot influence the outcome of the HIV test directly.

Another source of correlation between nurses and HIV status would arise if nurses were systematically assigned to PSUs, households, or individuals with characteristics that are correlated with HIV status. This would be the case, for example, if older nurses were more likely to be assigned to the elderly, or if male nurses went more often to PSUs without electricity out of safety considerations for the female interviewers in the evenings.

However, a protocol was developed such that nurses were randomly assigned to PSUs and to households within these PSUs. We cannot rule out that nurses may have been assigned accidentally to PSUs with systematically different characteristics. However, a regression of PSU-level averages of 10 respondent and household characteristics on nurse ID codes shows that these characteristics are evenly distributed over the nurses. Only 3 of 60 coefficients are statistically significant on a 5 % level, which is to be expected with purely random assignment (Online Resource 1, section 1).

Finally, there should be no unobservable variables that are correlated both with nurses’ ability to motivate individuals to participate in the test and with the probability of being HIV-positive. If nurse effects are heterogeneous by HIV status—that is, if Zi is correlated with εi1—the second requirement will be violated (Manski 1989). Nurse heterogeneity is further explored in Online Resource 1, section 2.

In sum, test participation rates differ significantly by nurse ID codes, satisfying the first requirement of nontrivial correlation. Moreover, nurses are randomly assigned to households, and cannot influence test outcomes directly. Therefore, the ID codes are valid instruments for Heckman analyses as long as any bias introduced by heterogeneous nurse effects is negligible. The remainder of this article rests on this assumption.

Results

This section presents estimates of the HIV prevalence rate corrected for refusal bias. The Heckman model takes into account both observable and unobservable factors related to HIV status and participation. If refusal is nonselective with respect to unobservable characteristics, HIV prevalence can be estimated with a simple probit model. Effectively, this method assumes that missing values are random given the observed variables, and thus the method corrects for nonresponse bias based on observed characteristics only.

Panel A of Table 4 shows the observed and the predicted HIV prevalence for both the probit and the Heckman model, for all individuals and for males and females separately (column 1).8 Columns 2–5 give the 95 % confidence intervals calculated with, respectively, the delta method and the standard method (which ignores uncertainty in the individual predictions) to allow for a comparison of the two.9

The results show that the probit correction for test refusal based on observable differences has only a minor impact. The difference between the observed and the probit predicted HIV prevalence rates is 0.01 percentage point for the total population. This is in line with findings from other studies (e.g., Floyd et al. 2013; Mishra et al. 2006, 2008).

A much larger difference is found between the observed and the Heckman-corrected prevalence estimates. The estimated prevalence rate increases by 1.33 percentage points for the total population, and by 0.82 and 2.85 percentage points for males and females, respectively.10 These findings are of the same order of magnitude as the predictions of Floyd et al. (2013), who estimated adjusted prevalence rates to be 1.3 to 2.7 percentage points higher for men and 1.4 to 2.9 percentage points higher for women compared with observed rates in Malawi. Reniers and Eaton (2009) calculated biases of 3.1 and 5.2 percentage points in urban Zimbabwe and urban Malawi, respectively.

However, the increases in the point estimates are not statistically significant (columns 2–3): all 95 % delta method confidence intervals of the Heckman predictions encompass the HIV prevalence rates as observed among participants. These findings are confirmed by the statistically nonsignificant parameter estimates of the Heckman model that represent the correlation between the unobserved determinants of participation and the unobserved determinants of HIV infection (Table 5).11 By the definition of , this implies that there is no statistical evidence of sample selection—that is, participation is not significantly related with the outcome variable. Therefore, the results do not yield general evidence of significant refusal bias resulting from unobservable characteristics for the total population and for males, and only weak evidence for females.

Panel B of Table 4 shows the predicted probabilities of HIV infection for participants and nonparticipants based on the Heckman selection model. The predicted probabilities are conditional on participation status, in line with Eqs. (3) and (4). The point estimates of HIV prevalence for nonparticipants are almost twice as large as for participants at predicted infection rates of 19.66 versus 10.56, respectively—a deviation of 86 %. The estimated bias for males is lower at 14.07 versus 8.95—a 57 % deviation. Female nonparticipants are estimated to be almost three times more likely to be infected than female participants, with predicted infection rates of 33.30 versus 11.80, respectively. Although these results are suggestive of substantial refusal bias, all delta method confidence intervals (columns 2–3) encompass the observed HIV prevalence rate.

For comparison, columns 4 and 5 of Table 4 show the confidence intervals as calculated with the standard method. They are narrower than the delta method intervals, especially for the Heckman estimates. Inference based on the standard method may unduly lead to the conclusion that the Heckman prevalence rates are significantly different from the observed prevalence rates.

The estimated prevalence rates among nonparticipants are similar in magnitude to the estimates corrected for unobservable characteristics in Bärnighausen et al. (2011), but are reversed for men and women. They predict that nonparticipating males are more than four times as likely to be infected as participating males and that the bias for nonparticipating females is about 70 %. The different patterns may be due to specific mechanisms that vary between Zambia and Namibia, to methodological differences such as the nonrandom assignment of interviewers in Bärnighausen et al. (2011), or to other reasons, such as the sensitivity of the Heckman selection model.

Validation of the Model

In 2008, a follow-up survey was conducted among the same sample of respondents as in 2006, allowing us to validate our estimation results. A total of 105 refusers in 2006 agreed to participate in the HIV test in 2008. The observed prevalence among those 105 individuals was 18.1 % (n=19) in 2008. The 2008 prevalence rate among those who participated in both years was substantially lower, at 13.9 %.12 As shown in the previous section, the Heckman predictions in Table 4, although not statistically significant, indeed point to a higher HIV prevalence rate among initial test refusers.

If we use the seroconversion rate of 3.87 % between 2006 and 2008 in the balanced sample of participants, we would expect that 89 refusers were HIV-negative in 2006 and 16 were HIV-positive, implying a 14.8 % prevalence rate in 2006. The conditional Heckman model predicts a 23.7 % prevalence rate for the 105 individuals in 2006, with a corresponding 95 % confidence interval of (7.1 %, 40.2 %). The unconditional Heckman model predicts a prevalence of 15.6 %, with 95 % confidence interval (11.4 %, 19.9 %). The confidence intervals of both predictions contain the estimated 14.8 % prevalence rate.13 As a comparison, the probit model predicts an HIV prevalence of 13.6 % in 2006 for the 105 refusers, with a 95 % confidence interval of (10.6 %, 16.5 %).

If the Heckman selection model improves the prediction of the likelihood of HIV infection among the 2006 nonparticipants, one would expect that the predicted individual HIV status in 2006 would be highly correlated with their actual HIV status in 2008. We conduct a probit regression of observed HIV status in 2008 on the conditional and unconditional Heckman predictions of HIV infection in 2006 for the 105 refusers (Table 6, columns 1 and 2). For comparison, the regression using the probit prediction is included in the table as well (column 3).

The coefficients of predicted HIV status are large and highly significant for all three estimations. Goodness of fit is slightly better for the Heckman prediction conditional on nonparticipation in 2006, as measured by the pseudo-R2 and log-likelihood estimates. The unconditional Heckman estimates correctly predict 78.1 % of all HIV-negative cases (82 of 95) and 50 % of all HIV-positive cases (5 of 10). Both the conditional Heckman and the probit estimates correctly predict 77.1 % of all HIV-negative cases (81 of 95) and 50 % of all positive cases (5 of 10). The imperfect fit of the three models can, to a large extent, be due to the seroconversion between the two survey rounds.

These findings suggest that the Heckman selection model has been able to capture essential and distinctive characteristics of nonrespondents in relation to their HIV status. Nevertheless, the probit model performs similarly well.

A Detailed Look at Nonparticipants

The findings in the Results section—although suggestive of refusal bias—are not statistically significant. A potential explanation why the analysis might fail to detect a bias, even if one existed, is related to the possibility that the various reasons for refusal may have opposite effects on prevalence rates. The literature generally points toward an underestimation of HIV prevalence because HIV-infected individuals are more likely to refuse. However, Table 1 suggests that other groups of nonparticipants—namely, the older and the richer population—are less likely to be infected. These individuals may be relatively more empowered to refuse test participation. Similarly, section 5 of Online Resource 1 shows that the refusal rate is highest in both the highest and the lowest stratum of probit predicted HIV prevalence. This heterogeneity may lead to offsetting biases that leave the overall HIV prevalence estimates unchanged but result in biased prevalence rates for specific subgroups.

Figure 1 presents the kernel density estimates for the Heckman linear prediction of HIV infection, , for participants and nonparticipants separately. To compare how the kernel density of corresponds to the Heckman prediction , the cumulative standard normal distribution is shown on the same domain as the kernel density of . For example, the median of is –1.503, which corresponds to an HIV infection probability of (–1.503) = .066, or 6.6 %. That is, the individuals located on the left side of the median in Fig. 1 have an estimated near-zero likelihood of infection. In contrast, for those individuals with a greater than 0 linear prediction of HIV, , the estimated likelihood of being infected is 50 % or more, based on the cumulative standard normal distribution.

Figure 1 clearly shows that the bias due to refusal is bidirectional. The kernel density of refusers is broader on both sides; that is, among test refusers, relatively more persons have either a very low or a very high likelihood of infection compared with test participants.

The left side of the kernel plot depicts individuals with a predicted likelihood of infection that is lower than for the median respondent. On this side of the density plot, where estimated infection rates are very low throughout, the nonparticipants are significantly older than the participants (Table 7). Participation is not correlated with age on the right side of the kernel plot, where the estimated likelihood of infection is above the median. Higher wealth, limited knowledge on HIV/AIDS, and stronger stigmatizing attitudes are predictors of refusal on both the left and the right side of the kernel. The difference in wealth is especially pronounced on the left side. Biomarkers are predictors of refusal on the right side of the kernel only.

These differences provide reason to believe that refusal bias may be more one-sided when looking at the young and the old or the poor and the nonpoor separately. We therefore construct the following four subgroups: the “young” subgroup consists of all individuals aged 18–39, and the “old” group includes all persons aged 30 or older. The subgroups “poor” and “nonpoor” are constructed such that they include all persons, respectively, in the bottom 75 % and the top 75 % of the per capita consumption distribution.14

Figure 2 shows the kernel density estimates of the Heckman linear prediction of HIV infection for individuals in the four subsamples. The model is estimated separately for each subsample. Nonparticipants in the “young” and “poor” subsamples are predicted to be more at risk of infection, and nonparticipants in the “old” and “nonpoor” subsamples are less likely to be infected compared with participants.15 Thus, the disaggregation in subgroups has enabled splitting the bidirectional bias in the total sample into two distinct parts.

Panel A of Table 8 presents the HIV prevalence estimates by subsample. The Heckman estimates for the young and the poor subsamples are about 3 to 4 percentage points higher than the observed HIV prevalence, except estimates for poor males, for whom the Heckman prediction is only 0.29 percentage points higher. Nevertheless, the observed HIV prevalence falls within the Heckman 95 % confidence interval for all subgroups.

Panel B of Table 8 shows the estimated HIV prevalence rate for participants and nonparticipants separately for the four subgroups. Nonparticipants in the young and poor subsamples are estimated to be about three times more likely to be HIV-infected. The estimated infection rates of nonparticipants are significantly different from the observed rates for the total young and poor subsamples and for poor females. These strong indications of refusal bias are confirmed by the significant parameters in the Heckman regressions: –.651 (p value = .010), –.600 (p value = .015), and –.794 (p value = .009) for the young, poor, and poor female subsamples, respectively (Table 5).

For the old and nonpoor subsamples, the results are quite different. Both the probit and Heckman predictions are similar to the observed HIV prevalence (panel A), and the likelihood of infection is comparable between participants and nonparticipants (panel B). That is, for the old and nonpoor subgroups, there is no statistical evidence of refusal bias. This is confirmed by the nonsignificance of the parameters in the Heckman regressions, with p values ranging from .334 to .924 (Table 5). Although the kernels for participants and nonparticipants in these subgroups do not overlap on the left side, this does not lead to a significant bias in the prevalence estimate because the refusers’ probability of HIV infection does not substantially differ in absolute size from the participants’ probability.

Conclusion

HIV prevalence estimates based on nationally representative population surveys are widely used for policy-making and resource allocation. However, population-based surveys face substantial nonresponse rates resulting from refusal to participate in the HIV test. If refusal is correlated with HIV status, the estimated HIV prevalence rate will be biased. This article explores the extent of nonresponse bias resulting from refusal, exploiting the random assignment of interviewers to respondents. Interviewers differed in their ability to elicit consent for the HIV test, they could not directly influence the result of the test itself, and their assignment was not correlated with HIV prevalence. Thus, both the requirement of nontrivial correlation and the exclusion restriction are satisfied, allowing us to use the interviewer ID codes as valid instruments to identify the selection effect.

We use a Heckman-type selection model to estimate the potential refusal bias resulting from unobservable variables and use the delta method to calculate the confidence intervals of the prevalence estimates. In line with other studies, estimated HIV prevalence rates do not significantly differ from observed HIV prevalence when controlling for observables only. The Heckman point estimates, corrected for unobservable factors, suggest that individuals who refuse to take the test are more likely to be HIV-positive than those who participate. However, the refusal bias is not statistically significant, given that the observed HIV infection rate among participants falls within the delta method confidence intervals of the estimated prevalence rate among nonparticipants. Inference based on the narrower—and incorrect—standard method confidence intervals, which fail to account for the uncertainty in predicted individual rates, would have led to wrong conclusions in this respect.

A detailed look at the kernel density of the Heckman selection model shows that the nonsignificance of the bias is related to counterbalancing effects in the overall population. On the one hand, and as expected, individuals who may fear exposure because they suspect being HIV-positive are less likely to participate. On the other hand, participation rates are also significantly lower for subsamples of people who may be both more empowered to refuse (e.g., the older and the wealthy) and have a low probability of HIV infection.

An analysis by subgroups shows that differences in infection rates between participants and nonparticipants, corrected for unobservable characteristics, are negligible for older and richer individuals, but they are large and statistically significant for young adults and for the poor. For these last two groups, the likelihood of infection is three to four times higher for nonparticipants than for participants and is especially pronounced for females.

Information on HIV status from a follow-up survey allows us to validate the Heckman selection model for a group of 105 individuals who refused testing at baseline but who agreed to be tested at follow-up. The results show that the baseline Heckman estimates for these individuals are good predictors of their actual HIV status at follow-up. The probit model, however, performs equally well.

Finding appropriate instruments to account for refusal bias in a Heckman selection model is difficult. The ID codes of interviewers may be used as instruments to detect such biases, particularly if they are randomly assigned to respondents. Therefore, we strongly recommend that both interviewer ID codes and assignment rules are routinely included when survey data are made available for analyses. Worldwide predictions of HIV prevalence by WHO and UNAIDS will benefit from this information because corrections can be made for unobservable characteristics that influence participation of surveyed individuals in HIV testing. And even if by coincidence, as in this article, no bias is found in the estimates for the overall population, the observed demographic or socioeconomic profile of the HIV-positive population may be flawed as a result of offsetting biases in subgroups. Prevalence rates that ignore selection bias may thus be seriously underestimated or overestimated for specific target groups, which can lead to misallocation of resources for prevention and treatment.

Acknowledgments

This work was supported by the Dutch Ministry of Development Cooperation (Grant No. 13298) and the Dutch Organization of Scientific Research (NWO) (Rubicon Grant No. 446-08-004 to W.J.). The survey data used in this article were collected by the University of Namibia (UNAM) and the National Institute of Population (NIP), with technical assistance from PharmAccess International and the Amsterdam Institute of International Development. Special thanks are due to Ingrid De Beer, Gert van Rooy, and Christa Schier for organizing the fieldwork and providing detailed insights into the data collection process. We are also grateful to Chris Elbers, Angus Deaton, and Aico van Vuren for helpful discussions on technical aspects of the estimation. We would like to thank participants at the 2007 AIID workshop on “The Economic Consequences of HIV/AIDS,” the 2008 Tinbergen Annual Conference in Amsterdam, the 2009 CSAE Conference in Oxford, and the 2012 Scientific EUDN Conference in Paris for useful comments and suggestions. Finally, we thank the editors of this journal and three anonymous reviewers for the positive and constructive feedback on earlier versions of this article.

Notes

1

Information about DHS+ surveys is available online (http://www.measuredhs.com/What-We-Do/).

2

In repeat population-based surveys, a fourth source of bias may arise: attrition (Obare 2010). A fifth bias inherent to household surveys is the sampling frame when only people residing in households are included.

3

Estimates are from authors’ own calculations.

4

The test results of 384 individuals were dropped from the sample because of fraudulent practices by one of the interviewers (Janssens et al. 2010). Because the interviewers were randomly assigned to households, this should not affect the results.

5

The wealth index is calculated based on the first factor loadings of a principal component analysis of 28 assets and 7 dwelling characteristics, with missing values imputed.

6

This can be calculated with the margins command in STATA version 11.0.

7

The Staiger and Stock (1997) rule of thumb to assess the strength of instrumental variables is not applicable in the case of a Heckman model. Instead, we calculate the likelihood ratio for the first stage with and without instruments. This shows that including the instrumental variables substantially increases the ratio from 187 to 298.

8

See Online Resource 1, sections 3 and 4, for the detailed regression results of the probit and Heckman model, respectively.

9

Another way of correctly calculating standard errors is by bootstrapping. The bootstrapped confidence intervals are not reported in the table because in about 10 % of the bootstrap iterations, the probit and Heckman models do not converge and it cannot be ruled out that nonconvergence is selective. The other 90 % yield intervals that are very similar to the intervals calculated with the delta method.

10

Online Resource 1, section 5, compares observed with predicted prevalence rates by stratum of probit predicted HIV prevalence, which is presumably less sensitive to probit misspecification. The results show that the probit predictions are very similar to the observed rates in all strata. The Heckman predictions are increasingly higher than the probit estimates for each consecutive stratum, suggesting that the bias in the population prevalence increases with the propensity of HIV-infection. However, refusal is most common in the first stratum with the lowest HIV infection rate. The section A Detailed Look at Nonparticipants explores the characteristics of nonparticipants in more detail.

11

Although the estimates are large and negative at –.329, –.198, and –.622, respectively, for all individuals and males and females, they are not significant at the 5 % level. For females, the p value is .061. A negative sign of indicates that individuals less likely to participate are more likely to be HIV-positive.

12

Attrition rates, which may be selective with respect to HIV status (Obare 2010), were very similar for HIV-negative versus HIV-positive participants: 36 % versus 35 %.

13

Section 6 of Online Resource 1 discusses how increases in sample size affect confidence intervals.

14

The subgroups overlap because the Heckman model does not converge for our data when taking a strict boundary between subgroups.

15

For comparison, Online Resource 1, section 7, shows the same plots including the densities from the probit model. The Heckman model shifts the kernel to the right compared with the probit model for the young and poor, but not the old and nonpoor subsamples.

References

Aulagnier, M., Janssens, W., De Beer, I., van Rooy, G., Gaeb, E., Hesp, C., . . . Rinke de Wit, T. F. (
2011
).
Incidence of HIV in Windhoek, Namibia: Demographic and socio-economic associations
.
PLoS ONE
,
6
(
10
),
e25860
. doi:10.1371/journal.pone.0025860
Bärnighausen, T., Bor, J., Wandira-Kazibwe, S., & Canning, D. (
2011
).
Correcting HIV prevalence estimates for survey nonparticipation using Heckman-type selection models
.
Epidemiology
,
22
,
27
35
. 10.1097/EDE.0b013e3181ffa201
Boerma, J. T., Ghys, P. D., & Walker, N. (
2003
).
Estimates of HIV-1 prevalence from national population-based surveys as a new gold standard
.
The Lancet
,
362
,
1929
1931
. 10.1016/S0140-6736(03)14967-7
Davidson, R., & MacKinnon, J. G. (
2004
).
Econometric theory and methods
.
New York, NY
:
Oxford University Press
.
De Walque, D. (
2009
).
Does education affect HIV status? Evidence from five African countries
.
World Bank Economic Review
,
23
,
209
233
. 10.1093/wber/lhp005
Durrant, G. B., Groves, R. M., Staetsky, L., & Steele, F. (
2010
).
Effects of interviewer attitudes and behaviors on refusal in household surveys
.
Public Opinion Quarterly
,
74
,
1
36
. 10.1093/poq/nfp098
Floyd, S., Molesworth, A., Dube, A. Crampin, A. C., Houben, R., Chihana, M., . . . Glynn, J. R. (
2013
).
Underestimation of HIV prevalence in surveys when some people already know their status, and ways to reduce the bias
.
AIDS
,
27
,
233
242
.
Fortson, J. (
2008
).
The gradient in sub-Saharan Africa: Socioeconomic status and HIV/AIDS
.
Demography
,
45
,
303
322
. 10.1353/dem.0.0006
García-Calleja, J. M., Gouws, E., & Ghys, P. D. (
2006
).
National population based HIV prevalence surveys in sub-Saharan Africa: Results and implications for HIV and AIDS estimates
.
Sexually Transmitted Infections
,
82
(
Suppl. 3
),
iii64
iii70
.
Gouws, E., Mishra, V., & Fowler, T. B. (
2008
).
Comparison of adult HIV prevalence from national population-based surveys and antenatal clinic surveillance in countries with generalized epidemics: Implications for calibrating surveillance data
.
Sexually Transmitted Infections
,
84
(
Suppl. 1
),
i17
i23
. 10.1136/sti.2008.030452
Hamers, R. L., de Beer, I. H., Kaura, H., van Vugt, M., Caparos, L., & Rinke de Wit, T. F. (
2008
).
Diagnostic accuracy of two oral fluid-based tests for HIV surveillance in Namibia
.
AIDS
,
48
,
116
118
.
Hogan, D. R., Salomon, J. A., Canning, D., Hammitt, J. K., Zaslavsky, A. M., & Bärnighausen, T. (
2012
).
National HIV prevalence estimates for sub-Saharan Africa: Controlling selection bias with Heckman-type selection models
.
Sexually Transmitted Infections
,
88
(
Suppl. 2
),
i17
i23
. doi:10.1136/sextrans-2012-050636
Janssens, W., de Beer, I., Coutinho, H. M., Van Rooy, G., van der Gaag, J., & Rinke de Wit, T. F. (
2010
).
Estimating HIV prevalence: A cautious note on household surveys in poor settings
.
BMJ
,
341
,
c6323
10.1136/bmj.c6323
Lachaud, J. P. (
2007
).
HIV prevalence and poverty in Africa: Micro- and macro-econometric evidences applied to Burkina Faso
.
Journal of Health Economics
,
26
,
483
504
. 10.1016/j.jhealeco.2006.10.007
Lydié, N., Robinson, N. J., Ferry, B., Akam, E., De Loenzien, M., & Abega, S. (
2004
).
Mobility, sexual behavior, and HIV infection in an urban population in Cameroon
.
Journal of Acquired Immune Deficiency Syndromes
,
35
,
67
74
. 10.1097/00126334-200401010-00010
Manski, C. F. (
1989
).
Anatomy of the selection problem
.
Journal of Human Resources
,
24
,
343
360
. 10.2307/145818
Marston, M., Harriss, K., & Slaymaker, E. (
2008
).
Non-response bias in estimates of HIV prevalence due to mobility of absentees in national population-based surveys: A study of nine national surveys
.
Sexually Transmitted Infections
,
84
(
Suppl.1
),
i71
i77
. 10.1136/sti.2008.030353
McNaghten, A. D., Herold, J. M., Dube, H. M., & St Louis, M. E. (
2007
).
Response rates for providing a blood specimen for HIV testing in a population-based survey of young adults in Zimbabwe
.
BMC Public Health
,
7
,
145
150
. 10.1186/1471-2458-7-145
Mishra, V., Barrere, B., Hong, R., & Khan, S. (
2008
).
Evaluation of bias in HIV seroprevalence estimates from national household surveys
.
Sexually Transmitted Infections
,
84
(
Suppl.1
),
i63
i70
. 10.1136/sti.2008.030411
Mishra, V., Bignami-Van Asshe, S., Greener, R., Vaessen, M., Hong, R., Ghys, P. D., . . . Rutstein, S. (
2007
).
HIV infection does not disproportionately affect the poorer in sub-Saharan Africa
.
AIDS
,
21
(
Suppl.7
),
S17
S28
.
Mishra, V., Vaessen, M., Boerma, J. T., Arnold, F., Way, A., Barrere, B., . . . Sangha, J. (
2006
).
HIV testing in national population-based surveys: Experience from the Demographic and Health Surveys
.
Bulletin of the World Health Organization
,
84
,
537
545
.
Montana, L. S., Mishra, V., & Hong, R. (
2008
).
Comparison of HIV prevalence estimates from antenatal care surveillance and population-based surveys in sub-Saharan Africa
.
Sexually Transmitted Infections
,
84
(
Suppl.1
),
i78
i84
. 10.1136/sti.2008.030106
Obare, F. (
2010
).
Nonresponse in repeat population-based voluntary counseling and testing for HIV in rural Malawi
.
Demography
,
47
,
651
665
. 10.1353/dem.0.0115
O’Muircheartaigh, C., & Campanelli, P. (
1999
).
A multilevel exploration of the role of interviewers in survey non-response
.
Journal of the Royal Statistical Society
,
162
,
437
446
. 10.1111/1467-985X.00147
Pison, G., Le Guenno, B., Lagarde, E., Enel, C., & Seck, C. (
1993
).
Seasonal migration: A risk factor for HIV infection in rural Senegal
.
Journal of Acquired Immune Deficiency Syndromes
,
6
,
196
200
.
Reniers, G., Araya, T., Berhane, Y., Davey, G., & Sanders, E. J. (
2009
).
Implications of the HIV testing protocol for refusal bias in seroprevalence surveys
.
BMC Public Health
,
9
(
1
),
163
10.1186/1471-2458-9-163
Reniers, G., & Eaton, J. (
2009
).
Refusal bias in HIV prevalence estimates from nationally representative seroprevalence surveys
.
AIDS
,
23
,
621
629
. 10.1097/QAD.0b013e3283269e13
Staiger, D., & Stock, J. (
1997
).
Instrumental variables regression with weak instruments
.
Econometrica
,
65
,
557
586
. 10.2307/2171753
UNAIDS
. (
2007
).
AIDS epidemic update
.
Geneva, Switzerland
:
UNAIDS
.
UNAIDS/WHO
. (
2005
).
Guidelines for measuring national HIV prevalence in population-based surveys
.
Geneva, Switzerland
:
UNAIDS/WHO Working Group on Global HIV/AIDS and STI Surveillance
.
Van de Ven, W. P. M. M., & Van Praag, B. M. S. (
1981
).
The demand for deductibles in private health insurance: A probit model with sample selection
.
Journal of Econometrics
,
17
,
229
252
. 10.1016/0304-4076(81)90028-2
Walker, N., Grassly, N. C., Garnett, G. P., Stanecki, K. A., & Ghys, P. D. (
2004
).
Estimating the global burden of HIV/AIDS: What do we really know about the HIV pandemic?
.
The Lancet
,
363
,
2180
2185
. 10.1016/S0140-6736(04)16511-2

Supplementary data