Reviewer #1 (Public review):
Summary
In their paper Zhan et al. have used Pf genetic data from simulated data and Ghanaian field samples to elucidate a relationship between multiplicity of infection (MOI) (the number of distinct parasite clones in a single host infection) and force of infection (FOI). Specifically, they use sequencing data from the var genes of Pf along with Bayesian modeling to estimate MOI individual infections and use these values along with methods from queueing theory that rely on various assumptions to estimate FOI. They compare these estimates to known FOIs in a simulated scenario and describe the relationship between these estimated FOI values and another commonly used metric of transmission EIR (entomological inoculation rate).
This approach does fill an important gap in malaria epidemiology, namely estimating force of infection, which is currently complicated by several factors including superinfection, unknown duration of infection, and highly genetically diverse parasite populations. The authors use a new approach borrowing from other fields of statistics and modeling and make extensive efforts to evaluate their approach under a range of realistic sampling scenarios. However, the write-up would greatly benefit from added clarity both in the description of methods, and in the presentation of the results. Without these clarifications, rigorously evaluating whether the author's proposed method of estimating FOI is sound remains difficult. Additionally, there are several limitations that call into question the stated generalizability of this method that should at minimum be further discussed by authors and in some cases require a more thorough evaluation.
Major comments:
(1) Description and evaluation of FOI estimation procedure.
a. The methods section describing the two-moment approximation and accompanying appendix is lacking several important details. Equations on line 891 and 892 are only a small part of the equations in Choi et al. and do not adequately describe the procedure notably several quantities in those equations are never defined some of them are important to understand the method (e.g. A, S as the main random variables for inter-arrival times and service times, aR and bR which are the known time average quantities, and these also rely on the squared coefficient of variation of the random variable which is also never introduced in the paper). Without going back to the Choi paper to understand these quantities, and to understand the assumptions of this method it was not possible to follow how this works in the paper. At minimum, all variables used in the equations should be clearly defined.
b. Additionally, the description in the main text of how queueing procedure can be used to describe malaria infections would benefit from a diagram currently as written it's very difficult to follow.
c. Just observing the box plots of mean and 95% CI on a plot with the FOI estimate (Figures 1, 2 and 10-14) is not sufficient to adequately assess the performance of this estimator. First, it is not clear whether authors are displaying the bootstrapped 95%Cis or whether they are just showing the distribution of the mean FOI taken over multiple simulations, and then it seems that they are also estimating mean FOI per host on an annual basis. Showing a distribution of those per host estimates would also be helpful. Second, a more quantitative assessment of the ability of the estimator to recover the truth across simulations (e.g. proportion of simulations where the truth is captured in the 95% CI or something like this) is important in many cases it seems that the estimator is always underestimating the true FOI and may not even contain the true value in the FOI distribution (e.g. figure 10, figure 1 under the mid IRS panel). But it's not possible to conclude on way or the other based on this visualization. This is a major issue since it calls into question whether there is in fact data to support that these methods give good and consistent FOI estimates.
d. Furthermore authors state in the methods that the choice of mean and variance (and thus second moment) parameters for inter arrival times are varied widely, however, it's not clear what those ranges are there needs to be a clear table or figure caption showing what combinations of values were tested and which results are produced from them, this is an essential component of the method and it's impossible to fully evaluate its performance without this information. This relates to the issue of selecting the mean and variance values that maximize the likelihood of observing a given distribution of MOI estimates, this is very unclear since no likelihoods have been written down in the methods section of the main text, which likelihood are the authors referring to, is this the probability distribution of the steady state queue length distribution? At other places the authors refer to these quantities as Maximum Likelihood estimators, how do they know they have found the MLE? There are no derivations in the manuscript to support this. The authors should specify and likelihood and include in an appendix why their estimation procedure is in fact maximizing this likelihood preferably with evidence of the shape of the likelihood, and how fine the grid of values they tested are for their mean and variance since this could influence the overall quality of the estimation procedure.
(2) Limitation of FOI estimation procedure.
a. The authors discuss the importance of duration of infection to this problem. While I agree that empirically estimating this is not possible, there are other options besides assuming that all 1-5 year olds have the same duration of infection distribution as naïve adults co-infected with syphilis. E.g. it would be useful to test a wide range of assumed infection duration and assess their impact on the estimation procedure. Furthermore, if the authors are going to stick to the described method for duration of infection, the potentially limited generalizability of this method needs to be further highlighted in both the introduction, and the discussion. In particular, for an estimated mean FOI of about 5 per host per year in the pre-IRS season as estimated in Ghana (Figure 3) it seems that this would not translate to 4 year old being immune naïve, and certainly this would not necessarily generalize well to a school-aged child population or an adult population.
b. The evaluation of the capacity parameter c seems to be quite important, and is set at 30, however, the authors only describe trying values of 25 and 30, and claim that this does not impact FOI inference, however it is not clear that this is the case. What happens if carrying capacity is increased substantially? Alternatively, this would be more convincing if the authors provided a mathematical explanation of why the carrying capacity increasing will not influence the FOI inference, but absent that, this should be mentioned and discussed as a limitation.
Comments on revisions:
The authors have adequately responded to all comments.