NICE is actively developing an AI transparency checklist to support consistent and auditable use of AI in evidence submissions, running parallel pilots examining reproducibility, validation, error detection, and human-in-the-loop requirements for AI-assisted HTA dossiers. [1] This is not a future development. It is a project underway now that will produce formal documentation standards for AI-assisted submissions at the HTA body that reviews the majority of new drug applications in Europe's largest pharmaceutical market. For HEOR teams preparing submissions today, the practical question is not whether an audit trail will eventually be required for AI-assisted HTA evidence. It is whether the audit trail they can currently produce will satisfy the standard NICE, G-BA, and EMA are converging on.
At Pienomial, we built auditable AI output traceability into KnolAI's architecture from the first line of code, and we built KnolForge's governance layer to generate the complete AI data provenance documentation that ALCOA++ and the FDA-EMA joint Guiding Principles now require. This post provides a practical, actionable checklist for building an audit trail that satisfies the current and emerging documentation standards across NICE, G-BA, FDA, and EMA for AI-assisted HTA submissions, and explains how KnolAI's architecture generates this audit trail as a standard operational output rather than a retrospective documentation exercise.[9]
1. Why the Audit Trail Standard for AI Has Changed Fundamentally in 2025 and 2026
The regulatory standard for audit trails in pharmaceutical contexts has always been high. What changed in 2025 and 2026 is that regulators have explicitly extended those existing standards to cover AI systems, and in some cases have tightened them specifically to address the unique risks that AI introduces: model version drift, data provenance opacity, and the risk of AI-generated outputs being presented as human-authored conclusions.[5]
Three regulatory developments define the current standard. First, COSO published guidance in February 2026 specifying that effective monitoring of AI-driven processes requires a complete audit trail capturing prompts, inputs, outputs, model and configuration versions, and evidence of human review, sufficient to reconstruct what the AI acted on and demonstrate that the control functioned as designed. [2] Second, the EU AI Act's full enforcement of high-risk AI requirements from August 2026 requires deployers of high-risk AI to maintain logs for at least six months, with specific requirements around traceability, accuracy of inputs, identification of natural persons involved, and the reference database used. [2] Third, EU GMP Chapter 4's July 2025 draft revision codified ALCOA++ as the binding data integrity standard for the first time, adding Traceable as a tenth principle requiring full record-history reconstruction for all data and metadata modifications.[3]
Together, these three developments mean that an AI-assisted HTA submission produced in 2026 without a complete, ALCOA++-compliant audit trail is not merely best-practice-deficient. It is non-compliant with standards that are now either binding law or imminently binding guidance.[5]
2. The ALCOA++ Framework Applied to AI: What Each Principle Requires
ALCOA++ defines ten data integrity principles that apply fully to AI-generated data in GxP contexts. Understanding what each principle requires specifically for AI-assisted HTA evidence generation clarifies precisely what an adequate audit trail must document.[8]
A, Attributable: Every AI action, query, extraction, screening decision, and synthesis output, must be attributed to the specific AI system, the specific model version, and the specific human user who initiated or reviewed the action. A generic reference to 'AI' is not attributable. The specific platform, version, and operator identity must be captured for every action in the trail.[4]
L, Legible: The audit trail must be readable and understandable by an independent reviewer, including a NICE technical reviewer or G-BA scientific advisor, without requiring specialist AI knowledge. Audit records expressed in machine code or in technical formats that cannot be rendered in human-readable form for review do not satisfy legibility.
C, Contemporaneous: Every action must be recorded at the time it occurs, not reconstructed afterward. A retrospective summary of what the AI did is not a contemporaneous audit trail. The timestamp must reflect when the action was performed, not when the documentation was compiled.
O, Original: The audit record must be the primary, unaltered record of what the AI system actually did, not a processed or summarised version. Where the AI processes a source document, the original document and the specific version of it that was processed must be identifiable.
A, Accurate: The audit record must correctly reflect what the AI system actually did. If the AI's screening decision or extraction result was subsequently reviewed and modified by a human, both the original AI output and the human modification must be recorded accurately, with the rationale for any modification documented.
C, Complete: The audit trail must cover every step in the AI-assisted workflow from initiation to final output. There must be no gaps in the chain from the search query through source retrieval, screening, extraction, synthesis, and human review to the final submission content.[4]
C, Consistent: The audit trail must be internally consistent: the records of search execution, screening decisions, and extraction results must be consistent with each other and with the final evidence outputs they supported. Inconsistencies between the audit trail and the submission content are a significant inspection risk.
E, Enduring: The audit trail must be retained for the period required by the applicable regulation. EU AI Act Article 12 requires a minimum of six months. GxP requirements typically require retention for the life of the product plus an additional period. Records must be stored in formats that remain accessible and readable throughout the retention period.[2]
A, Available: The audit trail must be available for review on demand by regulatory authorities. A submission accompanied by an audit trail stored in a format that requires specialist software to access, or that is held by a third-party vendor without guaranteed retrieval, does not satisfy availability.
T, Traceable: Added to ALCOA++ in 2023 and codified in EU GMP Chapter 4's July 2025 draft, Traceable requires that the complete history of every data record can be reconstructed from the audit trail. For AI-assisted HTA evidence, this means the chain from source document to extracted data point to evidence table to submission content must be fully reconstructable from the audit records alone, without relying on undocumented institutional memory.[3]
3. The Practical Audit Trail Checklist for AI-Assisted HTA Evidence
The following checklist translates the ALCOA++ framework and the current regulatory guidance from FDA, EMA, NICE, and G-BA into the specific documentation elements an audit trail for AI-assisted HTA evidence must contain. Each element addresses a specific regulatory requirement and includes the rationale for why it satisfies that requirement.[9]
Checklist Item 1, AI System Identification Record: Document the specific AI platform name, version number, and deployment date for the deployment that produced the submission evidence. Include the knowledge base or data layer version if the platform uses a separately versioned evidence base. This record satisfies the Attributable principle by identifying exactly which system, in which configuration, produced the evidence.
Checklist Item 2, Search Strategy Execution Log: A timestamped record of every search query executed, across every database or source type queried, including the exact search string, the execution date and time, and the record counts returned at each stage. This record must be detailed enough that an independent reviewer could re-execute the search and verify that the documented record set matches the actual results.[6]
Checklist Item 3, Deduplication and Initial Filtering Record: Documentation of the deduplication methodology applied to the combined record set from all databases, the number of duplicates removed, and the deduplication criteria. Where AI classification was used to flag duplicates, the classification criteria must be documented at a level of specificity that allows the classification logic to be independently assessed.
Checklist Item 4, Title and Abstract Screening Record: A record for every record in the search result set documenting the screening decision, the specific inclusion or exclusion criterion applied, and the date of the decision. For AI-assisted screening, this record must capture whether the decision was made by the AI or by a human reviewer, and, for AI decisions, the confidence score or classification probability if the AI system provides one.[7]
Checklist Item 5, Full-Text Screening Record: For every record that passed title and abstract screening, a record documenting the full-text screening decision, the specific criterion applied, and the date. Records excluded at this stage must have their exclusion reason documented at the level of specificity the PRISMA 2020 flow diagram requires.
Checklist Item 6, Data Extraction Record with Source Attribution: For every included study, a record of every data point extracted, the specific location within the source document from which it was extracted at the section, paragraph, or table level, and the date of extraction. This is the record that satisfies the Traceable principle: it allows any specific value in the evidence table to be traced back to its exact location in its primary source.
Checklist Item 7, Inter-Rater Reliability Documentation: Where dual screening or dual extraction was performed, documentation of the inter-rater reliability calculation including the specific methodology, the kappa coefficient or equivalent measure, and the procedure for resolving discordant decisions. Where AI simulated one reviewer role, this must be documented as AI simulation rather than independent human review.[1]
Checklist Item 8, Human Review and Override Record: A record of every stage at which a human reviewer assessed AI output, the reviewer's identity and qualification, the date of review, and, critically, a record of every AI output that was modified by human review including the original AI output and the human-approved version. The absence of this record is one of the most common audit trail gaps found in AI-assisted submissions.
Checklist Item 9, Evidence Synthesis Methodology Record: Documentation of the synthesis approach applied, whether narrative synthesis, pairwise meta-analysis, network meta-analysis, or another method, the specific statistical methods used, the software and version used for analysis, and the date of analysis execution. Where AI generated elements of the narrative synthesis, these sections must be identified.
Checklist Item 10, Version Control and Change Log: A record of every version of the evidence table, evidence synthesis, and submission section that was generated during the preparation process, with timestamps and the identity of the user who generated or approved each version. Where new evidence became available during preparation and required an update, the change and its rationale must be documented.[2]
4. The AI Data Provenance Layer: Documenting Your Knowledge Foundation
Beyond the workflow-level audit trail, AI data provenance documentation for HTA submissions must address the knowledge foundation the AI system operated on: what sources were in the AI's knowledge base, how they were validated before inclusion, when they were last updated, and what the version history of the knowledge base is.[5]
This data provenance layer matters because a technically correct workflow audit trail can coexist with an inadequate knowledge base. An AI that correctly executed a search against a knowledge base that had not been updated for six months, or that contained sources that had not been validated against their stated primary references, produces an audit trail that documents a flawed evidence synthesis correctly.[4]
The AI data provenance documentation for an HTA submission must therefore include: the data sources covered by the AI's knowledge layer, the last update date for each source type, the validation methodology applied to each source at ingestion, the source coverage gaps for the therapeutic area in question, and the version of the knowledge layer that was active during the submission preparation period. This documentation is the equivalent of the PRISMA 2020 search date and database coverage documentation that is already required for manual systematic literature reviews.[9]
5. What NICE's HTA Lab Pilots Signal About the Coming Documentation Standard
NICE's HTA Lab pilots on AI-assisted submissions are the most direct signal of where HTA body documentation standards are heading. The two parallel pilots examine an AI-assisted version of a historic submission and an External Assessment Group's review of an AI-assisted submission, with both pilots specifically focusing on reproducibility, validation, error detection, and human-in-the-loop requirements. [1] The expected output, an AI transparency checklist for consistent and auditable AI use in evidence submissions, will translate these pilot findings into a formal documentation standard.
The pilot design reveals what NICE considers the core risks of AI-assisted submissions: reproducibility failure, where an independent reviewer cannot re-execute the methodology to verify the result; validation gaps, where the AI's performance on the specific task was not demonstrated before use; error introduction, where AI errors passed into the evidence base undetected; and human oversight deficiencies, where the human review process did not adequately assess AI output reliability. An audit trail that documents all ten checklist items above directly addresses each of these four risk categories.
6. The Three Most Common Audit Trail Gaps in Current AI-Assisted Submissions
Based on the regulatory inspection experience documented across FDA warning letters, NICE technical queries, and G-BA clarification requests, three audit trail gaps appear most consistently in current AI-assisted submissions and carry the most significant consequences when identified.[4]
Gap 1, No record of AI screening decisions at the individual document level: Many HEOR teams document the total number of records screened and the total number included and excluded, without documenting the screening decision and criterion for each individual record. This satisfies the PRISMA flow diagram requirement in form but not in substance: an independent reviewer cannot verify that the inclusion and exclusion criteria were correctly applied to any specific paper without per-record documentation.[6]
Gap 2, No distinction between AI-generated and human-reviewed outputs in the evidence table: Evidence tables that mix AI-extracted values and human-reviewed and corrected values without distinguishing which values were modified in human review, and what the original AI-extracted value was, do not satisfy the Accurate principle. A reviewer who identifies an error in the evidence table cannot determine from an undifferentiated table whether the error was in the AI extraction or survived a human review that should have caught it.
Gap 3, Knowledge base version not documented: Submissions that document the search execution date and databases queried without documenting the version of the AI's knowledge base active at the time of the search leave an AI data provenance gap that satisfies the search date requirement of PRISMA but not the knowledge layer documentation requirement of the ALCOA++ Traceable principle. If the knowledge base was updated between the initial search and a rerun, the version difference must be documented.[3]
7. How KnolAI Generates the Complete Audit Trail Automatically
The most significant operational difference between KnolAI and a manual or generic AI-assisted HTA evidence process is not the quality of the search or the accuracy of the extraction. It is that KnolAI generates the complete ALCOA++-compliant audit trail as a structural by-product of the workflow execution, rather than requiring a separate documentation exercise after the evidence has been compiled.[9]
Every KnolAI action is logged in real time, not reconstructed afterward, satisfying the Contemporaneous principle. Every logged action is attributed to the specific KnolAI version, the specific knowledge layer version, and the specific operator session that initiated it, satisfying the Attributable principle. Every data extraction is recorded at the specific source document location level, creating the per-record source attribution that satisfies both the Traceable principle and the individual-document-level documentation that the most common audit trail gap currently reflects.
The human review log captures every stage at which a HEOR team member reviewed KnolAI output, the reviewer's platform identity, the date of review, and any modifications made to AI-generated outputs with the original AI output preserved alongside the human-approved version. This is the Override Record that most manual AI-assisted processes lack entirely, because it requires the audit infrastructure to be active during the review itself rather than documented afterward.[9]
8. Practical Implementation: Getting Your Audit Trail Ready Before the Next Submission
For HEOR teams preparing their next HTA submission and evaluating whether their current audit trail practices will satisfy the emerging standard, a gap assessment against the ten checklist items above is the most direct way to identify the documentation work required.[9]
The assessment should address four questions for each checklist item. First, does our current AI-assisted workflow generate this record at the time of the action, or would we need to reconstruct it retrospectively? Retrospective reconstruction fails the Contemporaneous principle and will be identified as a gap in any inspection. Second, is the record at the individual-action level or only at the aggregate level? Aggregate-level records fail the per-record documentation standard that NICE's pilot findings indicate will be required. Third, is the AI platform version and knowledge base version documented separately from the search execution date? Many current processes document the latter but not the former. Fourth, is the human override record separate from the final approved output, with the original AI output preserved alongside the human-modified version?[4]
9. How Fast Can Your Team Deploy Audit-Trail-Ready AI Evidence Generation with KnolAI?
Building an auditable AI output traceability infrastructure that satisfies the ALCOA++ standard from scratch requires significant documentation system development alongside the AI capability itself. KnolForge eliminates this development requirement by providing the audit trail infrastructure as a built-in platform capability from the first session.[9]
Sprint 1, Weeks 1 to 2, Complete audit trail active from the first HTA evidence workflow: KnolAI is configured for your therapeutic area and primary HTA submission use cases. The ALCOA++-compliant audit trail is active from the first search execution: every query, screening decision, extraction, and human review action is logged contemporaneously with full attribution. Your team reviews the first audit trail output and confirms it satisfies the ten checklist items above.
Sprint 2, Weeks 3 to 4, PALISADE and TRIPOD+AI documentation generated from audit data: The PALISADE and TRIPOD+AI checklist completion for your submission is generated using KnolAI's audit trail data as the primary source material. The methodology sections that these checklists require can be completed directly from the audit records, rather than requiring separate recall and reconstruction of the AI methods used.[6]
Sprint 3, Weeks 5 to 6, EU AI Act and NICE documentation package ready: The EU AI Act Article 12 six-month log retention is configured and confirmed. The AI data provenance documentation package, covering knowledge layer version, source coverage, and update date, is generated and appended to the audit trail. Your submission documentation package satisfies ALCOA++ across all ten principles and aligns with the documentation standard NICE's HTA Lab pilots are converging on.[1][2]
Conclusion
The audit trail standard for AI-assisted HTA submissions is not a future requirement that HEOR teams can monitor and prepare for when it arrives. NICE is actively piloting AI transparency checklists, ALCOA++ is being codified in EU binding regulation, and the EU AI Act's logging requirements are fully in force from August 2026. The gap between current practice and the emerging standard is already visible in the three common audit trail deficiencies identified across regulatory inspections and HTA technical queries.
At Pienomial, we built auditable AI output traceability into KnolAI as a foundational architectural property because we believe the audit trail is not a documentation burden separate from the evidence generation work. It is the mechanism by which AI-assisted evidence earns the trust of the HTA reviewers who will accept or reject the submission. An audit trail that a reviewer can follow from source to submission conclusion is not just a compliance requirement. It is what transforms AI from a speed tool into a credibility asset. [9]
CTA: See how KnolAI generates a complete, ALCOA++-compliant audit trail for every HTA submission workflow. Book a demo with the Pienomial team today.
