On 2017 Apr 10, Paul Sullins commented:

Reported findings of "no differences" by parent type in this study are an artifact of a well-known sampling error which conflates same-sex couples with a larger group of miscoded different-sex couples. Large disparities between the reported sample and same-sex couple population data reported by Statistics Netherlands strongly confirm this conclusion. The remainder of this comment presents detailed analysis supporting these claims. A longer critique, with standard citations and a table, is available at http://ssrn.com/author=2097328 .

The authors report that same-sex couples were identified using “information about the gender of the participating parent and the gender of the participant’s partner” (p. 5). However, validation studies of the use of this procedure on other large representative datasets, including the 2000 U.S. Census, the U.S. National Health Interview Survey (NHIS), and the National Longitudinal Study of Adolescent to Adult Health (“Add Health”), have found that most "same-sex couples" identified in this way are actually misclassified different-sex couples.

The problem stems from the fact that, like all survey items, the indication of one’s own sex or the sex of one’s partner is subject to a certain amount of random error. Respondents may inadvertently mark the wrong box or press the wrong key on the keyboard, thus indicating by mistake that their partner is the same sex as themselves. Black et al., who examined this problem on the U.S. Census, explains that “even a minor amount of measurement error, when applied to a large group, can create a major problem for drawing inferences about a small group in the population. Consider, for example, a population in which 1 out of 100 people are HIV-positive. If epidemiologists rely on a test that has a 0.01 error rate (for both false positives and false negatives), approximately half of the group that is identified as HIV-positive will in fact be misclassified” The measurement of same-sex unmarried partner couples in the 2000 US Census. Since same-sex couples comprise less than one percent of all couples in the population of Dutch parent couples studied by Bos et al., even a small random error in sex designation can result in a large inaccuracy in specifying the members of this tiny subpopulation.

A follow up consistency check can effectively correct the problem; however without this it can be quite severe. When the NHIS inadvertently skipped such a consistency check for 3.5 years, CDC estimated that from 66% to 84% of initially identified same-sex married couples were erroneously classified different-sex married couples Division of Health Interview Statistics, National Center for Health Statistics. 2015. Changes to Data Editing Procedures and the Impact on Identifying Same-Sex Married Couples: 2004-2007 National Health Interview Survey. Likewise, Black reported that in the affected portion of the 2000 Census “only 26.6 percent of same-sex female couples and 22.2 percent of same-sex male couples are correctly coded” Black et al, p. 10. The present author found, in an Add Health study that ignored a secondary sex verification, that 61% of the cases identified as “same-sex parents” actually consisted of different-sex parent partners The Unexpected Harm of Same-sex Marriage: A Critical Appraisal, Replication and Re-analysis of Wainright and Patterson’s Studies of Adolescents with Same-sex Parents. British Journal of Education, Society & Behavioural Science, 11(2)..

The 2011 Statistics Netherlands data used by Bos et al. are based on computer assisted personal interviews (CAPI), in which the respondent uses a computer keyboard to indicate his or her responses to interview questions that are presented by phone, website or in person. Sex of respondent and partner is indicated is indicated by the respondent entering "1" or "2" on the keyboard, a procedure in which a small rate of error, hitting the wrong key, would be quite normal. The Statistics Netherlands interview lacks any additional verification of sex designation, making sample contamination very probable. [Centraal Bureau voor de Statistiek Divisie Sociale en Ruimtelijke Statistieken Sector Dataverzameling. (2010). Jeugd en Opgroeien (SCP) 2010 Vraagteksten en schema’s CAPI/CATI. The Hague].

Several key features of the reported control sample strongly confirm that sample contamination has occurred. First, in the Netherlands in 2011, the only way for a same-sex co-parent to have parent rights was to register an adoption, so we would expect one of the partners, for most same-sex couples, to be reported as an adoptive parent [Latten, J., & Mulder, C. H. (2012). Partner relationships at the dawn of the 21st century: The case of the Netherlands. In European Population Conference pp. 1–19]. But in Bos et al.'s sample, none of the same-sex parents are adoptive parents, and both parents indicate that the child is his/her "own child" (eigen kind). This is highly unlikely for same-sex couples, but what we would expect to see if a large proportion of the "same-sex" couples were really erroneously-coded opposite-sex couples. Second, the ratio of male to female same-sex couples in the Bos et al. sample is implausibly high. In every national and social setting studied to date, far fewer male same-sex couples raise children than do female ones. Statistics Netherlands reports that in 2011 the disparity in the Netherlands was about seven to one: Of the (approximately) 30,000 male and 25,000 female same-sex couples counted in that year “[o]nly 3% (nearly 800) of the men's pairs had one or more children, compared to 20% (almost 5000) of the female couples.” [de Graaf, A. (2011). Gezinnen in cijfers, in Gezinsrapport 2011: Een portret van het gezinsleven in Nederland. The Hague: The Netherlands Institute for Social Research.] Yet Bos et al. report, implausibly, that they found about equal numbers of both lesbian and gay male couples with children, actually more male couples (68) than female (63) with children over age 5. They also report that 52% of Dutch same-sex parenting couples in 2011 were male, but Statistics Netherlands reports only 14%. The Bos sample is in error exactly to the degree that we would expect if these were (mostly) different-sex couples that were inaccurately classified as being same-sex due to random errors in partner sex designation.

Third, according to figures provided by Eurostat and Statistics Netherlands [Eurostats. (2015). People in the EU: who are we and how do we live? - 2015 Edition. Luxembourg: Publications Office of the European Union.] [Nordholt, E. S. (2014). Dutch Census 2011: Analysis and Methodology. The Hague: Statistics Netherlands.] (www.cbs.nl/informat), same-sex parents comprised an estimated 0.28 percent of all Dutch parenting couples in 2011, but in the Bos sample the prevalence is more than three times this amount, at 0.81 percent. From this disparity, it can be estimated roughly that about 65% of the Bos control sample consisted of misclassified different-sex parents. This rate of sample contamination is very similar to that estimated for the three datasets discussed above (61% for Add Health; 66% or higher for NHIS, and about 75% for the 2000 U.S. Census.)<br> The journal Family Process has advised that it is not interested in addressing errors of this type in its published studies. I therefore invite the authors to provide further population evidence in this forum, if possible, showing why their findings should be considered credible and not spurious.

Paul Sullins, Ph.D. Catholic University of America sullins@cua.edu

On 2017 Apr 10, Paul Sullins commented:

None

On 2017 Apr 10, Paul Sullins commented:

None

On 2017 Apr 10, Paul Sullins commented:

Reported findings of "no differences" by parent type in this study are an artifact of a well-known sampling error which conflates same-sex couples with a larger group of miscoded different-sex couples. Large disparities between the reported sample and same-sex couple population data reported by Statistics Netherlands strongly confirm this conclusion. The remainder of this comment presents detailed analysis supporting these claims. A longer critique, with standard citations and a table, is available at http://ssrn.com/author=2097328 .

The authors report that same-sex couples were identified using “information about the gender of the participating parent and the gender of the participant’s partner” (p. 5). However, validation studies of the use of this procedure on other large representative datasets, including the 2000 U.S. Census, the U.S. National Health Interview Survey (NHIS), and the National Longitudinal Study of Adolescent to Adult Health (“Add Health”), have found that most "same-sex couples" identified in this way are actually misclassified different-sex couples.

The problem stems from the fact that, like all survey items, the indication of one’s own sex or the sex of one’s partner is subject to a certain amount of random error. Respondents may inadvertently mark the wrong box or press the wrong key on the keyboard, thus indicating by mistake that their partner is the same sex as themselves. Black et al., who examined this problem on the U.S. Census, explains that “even a minor amount of measurement error, when applied to a large group, can create a major problem for drawing inferences about a small group in the population. Consider, for example, a population in which 1 out of 100 people are HIV-positive. If epidemiologists rely on a test that has a 0.01 error rate (for both false positives and false negatives), approximately half of the group that is identified as HIV-positive will in fact be misclassified” The measurement of same-sex unmarried partner couples in the 2000 US Census. Since same-sex couples comprise less than one percent of all couples in the population of Dutch parent couples studied by Bos et al., even a small random error in sex designation can result in a large inaccuracy in specifying the members of this tiny subpopulation.

A follow up consistency check can effectively correct the problem; however without this it can be quite severe. When the NHIS inadvertently skipped such a consistency check for 3.5 years, CDC estimated that from 66% to 84% of initially identified same-sex married couples were erroneously classified different-sex married couples Division of Health Interview Statistics, National Center for Health Statistics. 2015. Changes to Data Editing Procedures and the Impact on Identifying Same-Sex Married Couples: 2004-2007 National Health Interview Survey. Likewise, Black reported that in the affected portion of the 2000 Census “only 26.6 percent of same-sex female couples and 22.2 percent of same-sex male couples are correctly coded” Black et al, p. 10. The present author found, in an Add Health study that ignored a secondary sex verification, that 61% of the cases identified as “same-sex parents” actually consisted of different-sex parent partners The Unexpected Harm of Same-sex Marriage: A Critical Appraisal, Replication and Re-analysis of Wainright and Patterson’s Studies of Adolescents with Same-sex Parents. British Journal of Education, Society & Behavioural Science, 11(2)..

The 2011 Statistics Netherlands data used by Bos et al. are based on computer assisted personal interviews (CAPI), in which the respondent uses a computer keyboard to indicate his or her responses to interview questions that are presented by phone, website or in person. Sex of respondent and partner is indicated is indicated by the respondent entering "1" or "2" on the keyboard, a procedure in which a small rate of error, hitting the wrong key, would be quite normal. The Statistics Netherlands interview lacks any additional verification of sex designation, making sample contamination very probable. [Centraal Bureau voor de Statistiek Divisie Sociale en Ruimtelijke Statistieken Sector Dataverzameling. (2010). Jeugd en Opgroeien (SCP) 2010 Vraagteksten en schema’s CAPI/CATI. The Hague].

Several key features of the reported control sample strongly confirm that sample contamination has occurred. First, in the Netherlands in 2011, the only way for a same-sex co-parent to have parent rights was to register an adoption, so we would expect one of the partners, for most same-sex couples, to be reported as an adoptive parent [Latten, J., & Mulder, C. H. (2012). Partner relationships at the dawn of the 21st century: The case of the Netherlands. In European Population Conference pp. 1–19]. But in Bos et al.'s sample, none of the same-sex parents are adoptive parents, and both parents indicate that the child is his/her "own child" (eigen kind). This is highly unlikely for same-sex couples, but what we would expect to see if a large proportion of the "same-sex" couples were really erroneously-coded opposite-sex couples. Second, the ratio of male to female same-sex couples in the Bos et al. sample is implausibly high. In every national and social setting studied to date, far fewer male same-sex couples raise children than do female ones. Statistics Netherlands reports that in 2011 the disparity in the Netherlands was about seven to one: Of the (approximately) 30,000 male and 25,000 female same-sex couples counted in that year “[o]nly 3% (nearly 800) of the men's pairs had one or more children, compared to 20% (almost 5000) of the female couples.” [de Graaf, A. (2011). Gezinnen in cijfers, in Gezinsrapport 2011: Een portret van het gezinsleven in Nederland. The Hague: The Netherlands Institute for Social Research.] Yet Bos et al. report, implausibly, that they found about equal numbers of both lesbian and gay male couples with children, actually more male couples (68) than female (63) with children over age 5. They also report that 52% of Dutch same-sex parenting couples in 2011 were male, but Statistics Netherlands reports only 14%. The Bos sample is in error exactly to the degree that we would expect if these were (mostly) different-sex couples that were inaccurately classified as being same-sex due to random errors in partner sex designation.

Third, according to figures provided by Eurostat and Statistics Netherlands [Eurostats. (2015). People in the EU: who are we and how do we live? - 2015 Edition. Luxembourg: Publications Office of the European Union.] [Nordholt, E. S. (2014). Dutch Census 2011: Analysis and Methodology. The Hague: Statistics Netherlands.] (www.cbs.nl/informat), same-sex parents comprised an estimated 0.28 percent of all Dutch parenting couples in 2011, but in the Bos sample the prevalence is more than three times this amount, at 0.81 percent. From this disparity, it can be estimated roughly that about 65% of the Bos control sample consisted of misclassified different-sex parents. This rate of sample contamination is very similar to that estimated for the three datasets discussed above (61% for Add Health; 66% or higher for NHIS, and about 75% for the 2000 U.S. Census.)<br> The journal Family Process has advised that it is not interested in addressing errors of this type in its published studies. I therefore invite the authors to provide further population evidence in this forum, if possible, showing why their findings should be considered credible and not spurious.

Paul Sullins, Ph.D. Catholic University of America sullins@cua.edu