Reviewer #3 (Public review):
Summary
This paper introduces EMUsort, an extension of Kilosort4 designed to sort motor unit action potentials from high-density intramuscular EMG recordings. Using rat and monkey forelimb recordings, the authors generate realistic simulated datasets with known ground truth and demonstrate that EMUsort substantially outperforms Kilosort4 and MUedit, particularly during periods of high motor unit overlap.
Strengths
This is a timely study in light of recent advances in intramuscular muscle recording technologies and the growing interest in automated methods for decoding neural and neuromuscular signals. The work leverages state-of-the-art electrode arrays and combines them with advanced signal processing tools to address a challenging and relevant problem in motor unit analysis.
Weaknesses
There are several aspects of the study that substantially limit the interpretation of the main results and conclusions. The following major points should be carefully considered by the authors.
(1) Choice of experimental model and validation framework: The study aims to validate a new methodology for EMG decomposition, yet the rationale for the chosen experimental models is unclear. Specifically, it is not evident why the authors focused on intramuscular recordings from two animal models performing dynamic tasks. Given the extensive literature on the development and validation of EMG decomposition methods, the choice of an experimental design that substantially deviates from established approaches is insufficiently justified. In particular, it is unclear why the authors did not consider more standard validation paradigms based on (i) isometric contractions, (ii) human data, (iii) surface EMG recordings, or (iv) combinations of their recording technologies with previously validated motor unit identification methods. This methodological divergence makes it difficult to interpret the findings in the context of existing evidence.
(2) Lack of manual EMG decomposition as reference: Related to the previous point, it is unclear why standard manual EMG decomposition methods were not used to generate reference datasets for validation. Manual decomposition has been shown to be reliable under specific conditions (low contraction levels, slow dynamics, etc.) and would have substantially strengthened the validation of the proposed algorithm.
(3) Neglect of muscle deformation effects: While the manuscript discusses several factors that complicate EMG decomposition relative to brain recordings, it does not address the well-known effects of muscle deformation during contractions on motor unit action potential shapes. There is extensive literature demonstrating that dynamic muscle contractions lead to systematic changes in action potential morphology, representing a major challenge for EMG decomposition and a fundamental difference from brain recordings. Additionally, even small relative movements of intramuscular electrodes can produce waveform changes that are large relative to muscle fiber dimensions. These issues are particularly relevant given the highly dynamic tasks studied here (e.g., treadmill walking in rats), yet they are not discussed or incorporated into the analysis.
(4) Exclusive reliance on simulated data for validation: The primary validation of EMUsort is based on simulated data, which represents a major limitation of the study. This reliance should be clearly and explicitly stated in the abstract, introduction, and discussion. Moreover, the simulation approach itself raises concerns. The simulated EMG signals are generated using templates derived from the same sorting framework being validated, which introduces a potential methodological bias. The linear combination of components used to synthesize waveforms constitutes an unjustified modeling assumption that may favor template-based approaches such as EMUsort. Additionally, the spike time generation procedure appears unnecessarily complex and insufficiently justified. Previous validation studies typically modeled motor units as firing at relatively stable levels along their recruitment curves, producing long spike trains with pseudo-random relative timing and diverse overlap conditions. This framework would likely provide a more robust and interpretable validation. If the authors believe their simulation approach is superior, a stronger justification is required. Finally, the limited number of simulated motor units is difficult to reconcile with the expected level of motor unit recruitment during the studied behaviors, and this choice is not adequately justified.
(5) Incomplete reporting and visualization of experimental data: The manuscript would benefit from a clearer description of the number of rats and monkeys used, which should be reported explicitly in the abstract. In addition, visualizations of the raw multichannel EMG data across different task phases and activation levels would substantially improve transparency. Providing comprehensive visualizations of motor unit action potential shapes across all channels and identified units (for both rats and monkeys) would also help readers assess the spatiotemporal features that underpin unit identification and sorting reliability.
(6) Physiological limitations of conduction delay correction: The proposed method for correcting conduction delays across channels is physiologically suboptimal. First, motor unit conduction velocities differ substantially across units, implying that delay correction should be applied at the unit level rather than uniformly across channels. Second, conduction delays depend on fiber orientation and distance relative to electrode geometry; if fibers are oriented at different angles with respect to the array, a single delay correction becomes invalid. Additionally, the schematic in Figure 2A appears to contradict the proposed correction approach: if electrode threads are arranged perpendicular to muscle fibers, conduction delays across channels within a single thread should be minimal.
(7) Clarity issues in Figure 4: Figure 4 (panels A-D) is potentially misleading. It should be clearly stated whether the signals shown are artificial examples or derived from real recordings; ideally, real data should be used to illustrate the advantages of dynamic thresholds. In panels B-D, the depiction of overlapping action potentials is difficult to interpret due to the thickness of the traces, and it is unclear whether overlaps with neighboring action potentials are absent by design or expected to occur in real data. If overlaps are expected, one would also expect to observe contamination in the extracted waveforms, which is not evident in the figure.
(8) Concerns regarding method comparisons: The comparison with existing methods raises methodological concerns. It appears that EMUsort was carefully optimized, whereas the competing algorithms were not equivalently fine-tuned. The literature clearly shows that EMG decomposition performance depends strongly on adapting algorithms to the signal type (intramuscular vs. surface, species, electrode geometry). Furthermore, it is surprising that MUedit is reported to perform particularly poorly during periods of motor unit overlap, as blind source separation methods were explicitly developed to handle convolutive mixtures and overlapping sources, especially in surface EMG (which is an extreme case of motor unit overlapping). This discrepancy requires further explanation.
(9) Insufficient characterization of motor unit firing properties: The study does not provide sufficient information about the firing characteristics of the identified motor units in experimental data. Relevant metrics that should be reported include average, minimum, and maximum firing rates; coefficients of variation of discharge rate; signal-to-noise ratios of motor unit action potentials; potential evidence of motor unit rotation over time; and stability of firing behavior across recording intervals.
(10) Lack of theoretical framing: Given the scope and claims of the paper, it would be valuable to include a more theory-driven introduction explaining why different sorting approaches (e.g., template matching vs. blind source separation) may be more or less suitable depending on the nature of the recorded signals. A clearer conceptual rationale for why the proposed approach is expected to outperform existing methods would substantially strengthen the manuscript.
(11) Limitations of validation metrics: Some of the metrics used to evaluate performance are not ideal. For example, reporting 0% accuracy for a unit is misleading and should instead be described as a failure to identify that unit. Similarly, comparisons of total spike counts are of limited interpretive value and may be misleading, as correct spike counts do not necessarily imply correct spike identities.
(12) Clarification of computational performance claims: Finally, the discussion of computation times should clarify that some existing methods require substantial time for offline estimation of projection vectors but can operate in near real time once these vectors are learned and remain stable. This distinction is important for a fair comparison of practical usability.