On 2016 Aug 31, Robert Speth commented:

The primary reason this study is scientifically invalid is that there is no evidence that a protein the size of apoaequorin, administered in a pill form can enter the body via absorption from the digestive tract. Indeed, there is abundant evidence that proteins are metabolized in the digestive tract into component amino acids. To make a simple comparison, this is the reason insulin is injected into the body rather than taken orally. Additionally, even if apoaequorin was administered parenterally, it would not be able to cross the blood-brain-barrier and enter into the brain to exert its therapeutic effects in brain neurons that mediate cognitive functions. It is noteworthy that in the manuscript the authors cite to show the putative benefits of apoaequorin to protect hippocampal neurons from stroke damage[1], the apoaequorin was administered intracranially since this was the only way that the drug could gain access into the brain. Additionally, the protective effects described for apoaequorin in that study related to hypoxic/glucose deprivation conditions associated with stroke leading to massive increases in intracellular calcium, as opposed to normoxic/glucose available conditions. Notably missing from the article is any mention of a mechanism by which orally administered apoaequorin could gain access to neurons in the brain to mediate the improvement in cognitive function.

Additionally, if by some remarkable circumstance apoaequorin was able to enter the body from the GI tract, e.g., through an open sore in the mouth, accidental inhalation of the pill, or a damaged esophagus, it would pose a serious health hazard to the person taking this pill because foreign proteins are immunogenic and could produce a harmful immunological response. Worse yet, it might impair the calcium homeostasis of every cell in the body in which it gained entry.

Another reason that this article is scientifically invalid is that there was no statistical comparison made between the apoaequorin treated group and the placebo control group. The primary reason for including a placebo control group in clinical trials is to determine if the treated group improves significantly more than the placebo group. The omission of this information invalidates any claims the authors might try to make regarding the efficacy of orally administered apoaequorin.

Additional statistical inadequacies that make the article scientifically invalid are: there is no representation of the error variance for the data presented in Figures 1 – 6 of the manuscript, nor is such error variance reported in the text of the manuscript. The Statistical analysis section describes the use of paired and independent t tests, a repeated-measures analysis of covariance (ANCOVA) test, Mann-Whitney U test and Wilcoxon signed-ranks test to examine group differences. However, the only tests presented in the results are paired t tests with degrees of freedom that differ from the degrees of freedom expected based upon the group sizes that were reported in Table 1. Only 2 subjects are indicated as not having completed the testing, yet the number of subjects per group which should be the number of degrees of freedom plus 1, suggest that the values for 51 subjects are missing from the statistical analyses.

The description of the tests of cognition is vague and does not allow the reader to know how many different tests were administered to the participants in “The Madison Memory Study”. There is no indication of the number of computerized tasks from the CogState Research Battery that were administered to the study participants. The lack of such information makes it impossible to determine if the preponderance of the tasks included in the CogState Research Battery showed no difference between the apoaequorin group and the control group. It also makes it impossible to determine the error rate that should be applied to t tests to account for multiple comparison mediated increases in the Type I error rate. Furthermore, there are much better and more meaningful ways to describe an effect than Cohen’s d test, for example the 95% confidence interval around the measured effect.

In view of the statistical anomalies in this article one must consider the possibility of subjective bias in the conduct of this research. Since all of the authors of the study are associated with the company that is marketing apoaequorin as a memory enhancer, and there is no acknowledgment of participation by any other persons, let alone an independent third party, there is a financial conflict of interest in the outcome of the study that should disqualify the authors from being able to claim that this study was objectively double blinded. It is troublesome that the control group, from which the values for 13% of the participants are missing, experienced a dramatic reduction in their level of performance enhancement between days 60 and 90, for which no explanation is provided, while the apoaequorin group for which the results for 2 subjects are missing, showed a dramatic increase in performance from 60 to 90 days. Looking only at the 90-day performance enhancement rather than the enhancements at shorter time intervals, which seems to have been part of the original study design by ANCOVA, also creates a selection bias that compromises the validity of the study.

Another potential conflict of interest is the occurrence of a paid advertisement for Prevagen® brand of apoaequorin prior to the table of contents page of the journal issue. The possibility that publication of this article in Advances in Mind-Body Medicine was associated with financial compensation to the journal for placement of this advertisement is at the very least an apparent financial conflict of interest.

Finally, there is an inaccuracy in the characterization of apoaequorin “… having an amino acid sequence similar to human calcium binding proteins.” This uncited statement on page 5 is misleading. A BLAST sequence analysis of apoaequorin run using blastp, revealed a small sequence homology with a single isoform of the human calcium binding protein plastin-3. By no means can the inference be made that apoaequorin has a high homology with human calcium binding proteins.

1. Detert JA, Adams EL, Lescher JD, Lyons JA, Moyer JR, Jr. Pretreatment with apoaequorin protects hippocampal CA1 neurons from oxygen-glucose deprivation. PloS one. 2013;8(11):e79002. Epub 2013/11/19. doi: 10.1371/journal.pone.0079002. PubMed PMID: 24244400; PubMed Central PMCID: PMCPMC3823939.

On 2016 Aug 31, Robert Speth commented:

The primary reason this study is scientifically invalid is that there is no evidence that a protein the size of apoaequorin, administered in a pill form can enter the body via absorption from the digestive tract. Indeed, there is abundant evidence that proteins are metabolized in the digestive tract into component amino acids. To make a simple comparison, this is the reason insulin is injected into the body rather than taken orally. Additionally, even if apoaequorin was administered parenterally, it would not be able to cross the blood-brain-barrier and enter into the brain to exert its therapeutic effects in brain neurons that mediate cognitive functions. It is noteworthy that in the manuscript the authors cite to show the putative benefits of apoaequorin to protect hippocampal neurons from stroke damage[1], the apoaequorin was administered intracranially since this was the only way that the drug could gain access into the brain. Additionally, the protective effects described for apoaequorin in that study related to hypoxic/glucose deprivation conditions associated with stroke leading to massive increases in intracellular calcium, as opposed to normoxic/glucose available conditions. Notably missing from the article is any mention of a mechanism by which orally administered apoaequorin could gain access to neurons in the brain to mediate the improvement in cognitive function.

Additionally, if by some remarkable circumstance apoaequorin was able to enter the body from the GI tract, e.g., through an open sore in the mouth, accidental inhalation of the pill, or a damaged esophagus, it would pose a serious health hazard to the person taking this pill because foreign proteins are immunogenic and could produce a harmful immunological response. Worse yet, it might impair the calcium homeostasis of every cell in the body in which it gained entry.

Another reason that this article is scientifically invalid is that there was no statistical comparison made between the apoaequorin treated group and the placebo control group. The primary reason for including a placebo control group in clinical trials is to determine if the treated group improves significantly more than the placebo group. The omission of this information invalidates any claims the authors might try to make regarding the efficacy of orally administered apoaequorin.

Additional statistical inadequacies that make the article scientifically invalid are: there is no representation of the error variance for the data presented in Figures 1 – 6 of the manuscript, nor is such error variance reported in the text of the manuscript. The Statistical analysis section describes the use of paired and independent t tests, a repeated-measures analysis of covariance (ANCOVA) test, Mann-Whitney U test and Wilcoxon signed-ranks test to examine group differences. However, the only tests presented in the results are paired t tests with degrees of freedom that differ from the degrees of freedom expected based upon the group sizes that were reported in Table 1. Only 2 subjects are indicated as not having completed the testing, yet the number of subjects per group which should be the number of degrees of freedom plus 1, suggest that the values for 51 subjects are missing from the statistical analyses.

The description of the tests of cognition is vague and does not allow the reader to know how many different tests were administered to the participants in “The Madison Memory Study”. There is no indication of the number of computerized tasks from the CogState Research Battery that were administered to the study participants. The lack of such information makes it impossible to determine if the preponderance of the tasks included in the CogState Research Battery showed no difference between the apoaequorin group and the control group. It also makes it impossible to determine the error rate that should be applied to t tests to account for multiple comparison mediated increases in the Type I error rate. Furthermore, there are much better and more meaningful ways to describe an effect than Cohen’s d test, for example the 95% confidence interval around the measured effect.

In view of the statistical anomalies in this article one must consider the possibility of subjective bias in the conduct of this research. Since all of the authors of the study are associated with the company that is marketing apoaequorin as a memory enhancer, and there is no acknowledgment of participation by any other persons, let alone an independent third party, there is a financial conflict of interest in the outcome of the study that should disqualify the authors from being able to claim that this study was objectively double blinded. It is troublesome that the control group, from which the values for 13% of the participants are missing, experienced a dramatic reduction in their level of performance enhancement between days 60 and 90, for which no explanation is provided, while the apoaequorin group for which the results for 2 subjects are missing, showed a dramatic increase in performance from 60 to 90 days. Looking only at the 90-day performance enhancement rather than the enhancements at shorter time intervals, which seems to have been part of the original study design by ANCOVA, also creates a selection bias that compromises the validity of the study.

Another potential conflict of interest is the occurrence of a paid advertisement for Prevagen® brand of apoaequorin prior to the table of contents page of the journal issue. The possibility that publication of this article in Advances in Mind-Body Medicine was associated with financial compensation to the journal for placement of this advertisement is at the very least an apparent financial conflict of interest.

Finally, there is an inaccuracy in the characterization of apoaequorin “… having an amino acid sequence similar to human calcium binding proteins.” This uncited statement on page 5 is misleading. A BLAST sequence analysis of apoaequorin run using blastp, revealed a small sequence homology with a single isoform of the human calcium binding protein plastin-3. By no means can the inference be made that apoaequorin has a high homology with human calcium binding proteins.

1. Detert JA, Adams EL, Lescher JD, Lyons JA, Moyer JR, Jr. Pretreatment with apoaequorin protects hippocampal CA1 neurons from oxygen-glucose deprivation. PloS one. 2013;8(11):e79002. Epub 2013/11/19. doi: 10.1371/journal.pone.0079002. PubMed PMID: 24244400; PubMed Central PMCID: PMCPMC3823939.