Comment from Timothy Barry

Timothy Barry, Luca Pinello, Danillo Pellin, and Daniel Bauer Boston Children's Hospital and Massachusetts General Hospital The FDA draft guidance provides a useful experimental and analytical framework for assessing the safety and efficacy of therapeutic gene editors. However, we believe that the guidelines on experimental design and statistical analysis are underdeveloped. Targeted sequencing analysis. Targeted sequencing assays (e.g., pooled amplicon sequencing, hybrid-capture sequencing) assess editing at the on-target locus or a prespecified set of off-target loci. The draft guidance states that “the on-target editing rate can be determined by sequencing the on-target site and evaluating the proportion of reads harboring the intended edit over total reads covering the target site.” However, reads may harbor a modification due to either editing or a technical event (e.g., library preparation, sequencing, or alignment errors). Thus, we recommend that sponsors distinguish between the (unobserved) editing rate and (observed) modified-read fraction and use control samples, when possible, to adjust the modified-read fraction for technical artifacts. Second, the draft guidance states that sponsors “should use adequate sequencing depth” to carry out a targeted sequencing analysis. We recommend that sponsors conduct a comprehensive prospective power analysis to select the sequencing depth and replicate count required to estimate the editing rate (or modified-read fraction) at a desired level of statistical precision. The power analysis could incorporate genomic context, with high-risk regions (e.g., oncogenes) potentially requiring a higher level of statistical precision. We furthermore recommend that the editing rate (or modified-read fraction) be estimated using a statistical model that remains accurate for proportions near the boundary of the parameter space (close to 0 or 1). Finally, we recommend that sponsors report a confidence interval, credible interval, or other statistical measure of uncertainty alongside the point estimate of the editing rate. Unbiased, genome-wide analysis. Unbiased, genome-wide assays assess off-target activity genome-wide without requiring a prespecified locus list. These assays tend to be noisy: reads typically map to a small number of true editing sites and a much larger number of “background” sites. Background events can result from sequencing and alignment errors, spontaneous DNA cleavage, or other confounding technical and biological factors. The draft guidance recommends providing a “a detailed description of filtering step(s) used to either subset reads or variants based on base quality or mapping quality, should be supported with scientific justification.” We recommend to the extent possible avoiding use of arbitrary read count and homology thresholds to filter background sites, which can inflate false positive and negative results. Moreover, we recommend reporting a statistical measure of evidence (e.g., a p-value or likelihood ratio) of editing at each site to which reads map. These measures could incorporate read or UMI count, recurrence across replicates, local read structure around the putative cut site, homology, or other factors. Finally, we recommend that sponsors conduct a prospective power analysis to determine the replicate count and sequencing depth required to detect low-frequency editing events at a desired level of statistical sensitivity. Conclusion. Providing additional guidance on statistical analysis and experimental design could help sponsors conduct more reliable and interpretable preclinical therapeutic gene editing experiments.