Comment from Wicked Problem Wizards LLC

Wicked Problem Wizards Comments on Docket No. FDA-2026-N-4390 June 29, 2026 Dockets Management Staff (HFA-305) Food and Drug Administration 5630 Fishers Lane, Room 1061 Rockville, MD 20852 Submitted electronically via Regulations.gov Re: Docket No. FDA-2026-N-4390. Request for Information: AI-Enabled Optimization of Early-Phase Clinical Trials Pilot Program. Dear Members of the FDA Review Team, Wicked Problem Wizards is responding to three questions in this RFI. Wicked Problem Wizards is an advisory firm focused on data quality and fit-for-purpose clinical trial data under ICH E6(R3). We co-authored and co-led the eClinical Forum's ATRA working group that resulted in the 2026 position paper [2] cited in our response to A.4.b. Our work this year also includes kicking off BRAVE (Bridging Regulations and AI Validation Evidence), a pre-competitive cross-industry working group on AI validation. We have responded only to the three questions in the RFI where we can speak from direct experience. The argument has two parts. The pilot needs data that the AI can use, and a way to validate the AI itself. The data requires RBQM (Risk-Based Quality Management) applied at the process level. Without it, the AI produces confident outputs from broken inputs. The validation framework requires pre-competitive cross-industry work that the pilot cannot produce on its own. FDA's contribution to that work is most useful while the framework is being built, not after. A.3.b (collaboration). Pre-competitive cross-industry collaboration is how you produce frameworks for problems no single sponsor or vendor will solve on their own. The eClinical Forum's audit trail review work, which Nechama Katan co-led, is one example. BRAVE kicked off in June 2026 with 22 participants from pharma, CROs, and technology vendors, including an FDA participant. The most useful thing FDA can do is keep showing up as the regulatory anchor while the framework is being built. A.4.b (infrastructure). The infrastructure the pilot needs most is not technical. It is the organization's capability to run Risk-Based Quality Management and Audit Trail Review Analytics on the data the AI will use. AI deployed on data nobody has reviewed at the process level produces confident outputs from broken inputs. The eClinical Forum's 2026 ATRA [2] position paper gives the pilot a maturity model it can use as an infrastructure readiness check, with a minimum threshold on the Analytical Capability domain. FDA can suggest this threshold as an infrastructure expectation for all pilot participants, with documented readiness roadmaps for sponsors operating below it. B.5.a (validation evidence). What "valid and reliable" means for generative and agentic AI is an open question. Existing validation frameworks were built for deterministic software, trained ML models with characterizable behavior, or they explicitly carve out the kinds of AI the pilot will likely deploy. The April 2026 FDA warning letter on AI over-reliance shows the gap is real. The pilot will not answer this question, but FDA can contribute by supporting the work that will. The detailed responses follow. Respectfully submitted, Nechama Katan Founder, Wicked Problem Wizards nechama@wickedproblemwizards.com www.wickedproblemwizards.com