Comment from Anonymous

I am submitting this comment as a private citizen. I live in Washington, DC and have seen the impact of reductions in the federal workforce and am concerned on behalf of my friends, neighbors, and colleagues in regards to these surveys. I also have experience with human subjects research and survey design, and my comments below reflect the protections that should exist for any federal workplace survey. The Paperwork Reduction Act (PRA) directs OMB to evaluate whether a proposed collection has practical utility and whether its quality can be enhanced. The quality of these data depends on respondents trusting that their responses will not be identifiable. Federal employees who fear that their responses can be tied back to them may give neutral to positive reviews even if, or especially if, actual conditions are poor or leadership has produced a toxic workplace environment. The risk is that the resulting data may systematically overstate leadership effectiveness and workforce satisfaction, a well-documented effect in survey methodology research (Tourangeau and Yan, 2007). Providing anonymity reduces this distortion and serves the PRA's requirement that collections enhance the quality and utility of the information collected. The human subjects research community has spent decades addressing this challenge. The Common Rule at 45 CFR 46 codifies the protections researchers apply when collecting information from human subjects, precisely because data collection of this kind creates potential for harm to participants. Strong protections are also what make this kind of research possible. Without protections participants can trust, they either decline to participate or shape their responses to manage perceived risk, and the research produces distorted or unusable results. Although the Common Rule does not formally apply to administrative or programmatic data collections, the data collection and findings from these surveys present risk for the participants. Federal employees are being asked to provide candid assessments of their leadership and workplace conditions, and those assessments, if linked to individual respondents, could affect their employment, their working relationships, and their professional standing. This is the kind of harm that the protections in the Common Rule can mitigate. The Common Rule becomes even more salient when results will be shared publicly, and the recommendations below draw on protections typical to IRB-supervised research. Strikingly, if OPM funded a university researcher to administer these exact surveys and publish the findings, IRB approval and confidentiality protections would be required. The activity OPM proposes to conduct internally is substantively identical and warrants the same protections. Minimize the risk of identification. OMB should condition approval on four related design requirements that mitigate the risk of identification of individual responses. First, surveys should use generic distribution mechanisms, made available to all federal employees through posted URLs or all-staff announcements, rather than personalized invitations tied to individual employee identities. This eliminates the existence of individual-level participation data entirely. There are trade-offs, including the inability to send individual reminders which might reduce response rate. This trade-off is proportional to the protections gained. Second, collect only the minimum demographic information necessary for the specific analyses each survey is designed to support. All questions should be optional. Third, the survey platform should not capture metadata, including IP addresses or browser identifiers, beyond what is operationally necessary to administer the survey. Fourth, suppress reporting for units with fewer than ten respondents. OMB should make a minimum response threshold of ten respondents a condition of approval for both collections, with results from smaller units aggregated upward to the next organizational level to minimize identification of individuals in small offices. Require timely public release of aggregated results. OMB should condition approval on a requirement that aggregated results from each survey administration be released publicly within a defined timeframe, accompanied by methodology documentation. These protections will produce data that is less precise but more accurate. Researchers have long understood that anonymity is what makes honest responses possible, and without it the data describes something other than what it claims to measure. Ultimately if OMB and OPM want to enhance the quality, utility, and clarity of the information to be collected, then these are the protections and commitments that should be put in place. I appreciate the opportunity to comment.