This is a ready-to-use record for a single change to an already validated AI tool: an edit to the prompt, a move to a different model version, a change to the output schema, or a refresh of the retrieval corpus. It exists because of a common finding in this area, and one worth stating plainly: a team validates version 1, and then quietly edits the prompt or swaps the underlying model without assessing what that change did to the behavior they validated. This record is the executed artifact that the version-control and re-evaluation policy in the LLM evaluation and test plan requires, and it feeds the change control process in SOP: change control for validated systems. Replace every <<FILL: ...>> placeholder with your own specifics, set your document numbers and dates, and route it through your normal document control, review, and approval. A worked filled specimen follows the template so you can see how a completed version reads. Verify each cited regulation against the current source before you rely on it, and treat AI-specific guidance as evolving rather than settled.
Control header
| Field | Entry |
|---|---|
| Document title | AI Prompt, Model Version, and Retrieval Corpus Change Control Record |
| Record number | <<FILL: e.g. AICC-2026-014>> |
| Linked change request | <<FILL: CR number from your change control system, or "none, see section 9">> |
| AI tool name and ID | <<FILL: TOOL NAME / SYSTEM ID>> |
| Validated state reference | <<FILL: validation summary report number and version this change acts on>> |
| Document owner | <<FILL: role, e.g. AI System Owner>> |
| Record raised by | <<FILL: name, role>> |
| Date raised | <<FILL: date>> |
| Status | Draft / Under assessment / Approved / Implemented / Closed / Rejected / Rolled back |
1. Purpose of this record
This record documents one change to a validated AI tool and the assessment that establishes whether the tool remains fit for its approved intended use after the change. It captures what changed, who asked for it and why, how far the change reaches into validated behavior, the regression evidence that the acceptance criteria still hold, how to get back to the previous state, the Quality Assurance decision made before the change reached production, and what will be watched afterwards.
The record does not restate the governing policy. Risk tiering and the standing governance rules sit in Policy: AI governance for GxP systems. The intended-use boundary sits in Form: AI intended-use and use-pattern classification. The full validation method sits in Protocol: AI/ML system validation. This record is the per-event artifact those documents call for.
2. Scope
Raise this record for any change to a validated AI tool in the following categories, whether the change originates internally or arrives from the model vendor:
- Prompt text, system instruction, rubric, or in-context examples.
- Model identifier, model version, vendor, hosting region, or serving configuration.
- Output schema, field names, enumerated values, or the parsing and validation logic applied to model output.
- Retrieval corpus content, corpus version, chunking, embedding model, or retrieval parameters.
- Model parameters that alter output distribution, including sampling temperature, top-p, maximum output tokens, and stop sequences.
Changes to the surrounding deterministic code, infrastructure, or hosting that do not touch the five categories above go through your normal change control route and do not need this record, though they may still need an impact statement against this tool.
3. Field definitions
| Field | Format | Required | Who | When |
|---|---|---|---|---|
| Record number | Controlled sequence | Yes | Document control or system owner | On raise |
| AI tool name and ID | Text matching the system inventory | Yes | Requester | On raise |
| Validated state reference | Report number and version | Yes | System owner | On raise |
| Change category | One or more of the five scope categories | Yes | Requester | On raise |
| Prompt version from / to | Version identifier plus content hash | Yes when prompt changes | Requester | On raise |
| Model identifier from / to | Vendor, model family, exact version string | Yes when model changes | System owner or IT | On raise |
| Output schema version from / to | Schema version identifier plus hash | Yes when schema changes | Requester | On raise |
| Retrieval corpus version from / to | Corpus version identifier, document count, cut-off date | Yes when corpus changes | Data owner | On raise |
| Requester and business reason | Name, role, free text | Yes | Requester | On raise |
| Change classification | Tier 1, 2, or 3 per section 6 | Yes | System owner, confirmed by QA | Before testing |
| Regression evidence | Test reference, pass or fail per criterion | Yes for Tier 2 and Tier 3 | Validation lead | Before approval |
| Intended-use impact | Yes or No plus justification | Yes | System owner and QA | Before approval |
| Rollback plan | Text plus named artifacts | Yes | IT or system administrator | Before approval |
| QA assessment and approval | Signature and date | Yes | Quality Assurance | Before production |
| Post-change monitoring commitment | Metric, threshold, duration, owner | Yes for Tier 2 and Tier 3 | System owner | Before production |
| Implementation confirmation | Date, time zone, who implemented | Yes | IT or system administrator | On implementation |
| Closure | Signature and date | Yes | Quality Assurance | After monitoring period |
4. What changed
Complete every row that applies. Where a row does not apply, write “No change” rather than leaving it blank, because a blank row does not prove the item was considered.
| Element | From | To | Content hash or identifier | Evidence location |
|---|---|---|---|---|
| Prompt version | <<FILL: e.g. PROMPT-DEV-TRIAGE v2.3>> | <<FILL: e.g. v2.4>> | <<FILL: hash of both>> | <<FILL: repository path and commit>> |
| Prompt text diff | <<FILL: attach or reference the exact diff>> | <<FILL>> | ||
| Model vendor | <<FILL: vendor name>> | <<FILL: vendor name>> | <<FILL>> | |
| Model identifier and exact version string | <<FILL: full version string as returned by the API, not a marketing name>> | <<FILL>> | <<FILL: API response capture>> | |
| Model parameters | <<FILL: temperature, top-p, max output tokens, stop sequences>> | <<FILL>> | <<FILL: configuration file and commit>> | |
| Output schema version | <<FILL: e.g. SCHEMA-TRIAGE v1.1>> | <<FILL>> | <<FILL: hash>> | <<FILL>> |
| Schema field changes | <<FILL: fields added, removed, renamed, or with changed enumerations>> | <<FILL>> | ||
| Retrieval corpus version | <<FILL: corpus ID and version>> | <<FILL>> | <<FILL: manifest hash>> | <<FILL>> |
| Retrieval corpus scope | <<FILL: document count, source systems, cut-off date>> | <<FILL>> | <<FILL>> | |
| Embedding model and retrieval parameters | <<FILL: embedding model version, chunk size, top-k>> | <<FILL>> | <<FILL>> | |
| Deterministic code around the model | <<FILL: version, or "No change">> | <<FILL>> | <<FILL>> |
Record the exact version string the vendor API returns, not the marketing name. A marketing name that stays constant while the served weights change underneath it is a known trap and is the reason your log records the returned identifier rather than the name a person typed into a configuration file.
5. Who requested the change and why
| Field | Entry |
|---|---|
| Requester name and role | <<FILL>> |
| Date requested | <<FILL>> |
| Trigger for the change | <<FILL: e.g. vendor deprecation notice, monitoring signal, override-rate breach, new document type in scope, defect report, periodic review action>> |
| Business reason, stated plainly | <<FILL: what problem this change is meant to solve>> |
| Was the change forced by the vendor | Yes / No. If Yes, state the vendor notice reference and the deprecation date: <<FILL>> |
| Related deviation, monitoring alert, or CAPA | <<FILL: reference, or "none">> |
| Alternatives considered | <<FILL: including "do nothing" and why it was rejected>> |
A vendor-forced change is still a change. The fact that a version is being deprecated on a fixed date is a schedule constraint, not a reason to skip the assessment. If the deprecation date is close, that is a planning failure to log, not a justification to shorten the testing.
6. Change classification and the re-validation decision rule
Answer every question. The classification is the highest tier triggered by any Yes.
6.1 Screening questions
| # | Question | Answer | If Yes, tier |
|---|---|---|---|
| Q1 | Does the change alter the approved intended use, the decision the tool supports, or the population of records it processes? | Yes / No | Tier 3 |
| Q2 | Does the change touch a behavior classified as high risk in the risk assessment? | Yes / No | Tier 3 |
| Q3 | Does the change alter the output schema in a way a downstream consumer reads, including field names, types, or enumerated values? | Yes / No | Tier 3 |
| Q4 | Does the change alter the human checkpoint, what the reviewer sees, the disposition options, or the reason-capture behavior? | Yes / No | Tier 3 |
| Q5 | Does the change move to a different model family, a different vendor, or a different hosting arrangement? | Yes / No | Tier 3 |
| Q6 | Does the change alter the retrieval corpus in scope, source, or cut-off date, such that the tool can now cite material it could not cite before? | Yes / No | Tier 3 |
| Q7 | Does the change alter instructional content of the prompt, the rubric, the in-context examples, or a model parameter that affects the output distribution? | Yes / No | Tier 2 |
| Q8 | Does the change move to a different version string within the same model family, including a vendor patch? | Yes / No | Tier 2 |
| Q9 | Does the change add, remove, or reword a constraint the model is expected to obey? | Yes / No | Tier 2 |
| Q10 | Is the change confined to non-instructional text, code comments, logging labels that are not record fields, or formatting with no effect on model input or output? | Yes / No | Tier 1 |
6.2 What each tier requires
| Tier | Name | What it means | Required evidence before production |
|---|---|---|---|
| Tier 1 | Documentation only | The change cannot alter model input or output. Nothing the model sees has changed and nothing downstream reads a different value. | This record completed through section 9, a stated basis for the “cannot alter behavior” conclusion, and QA acknowledgement. No test execution required. A Tier 1 claim that turns out to have changed behavior is a deviation. |
| Tier 2 | Targeted regression | Behavior may shift, but intended use, schema, high-risk behaviors, and the human checkpoint are untouched. | Re-run of the reference case set, including the false-negative subset and the negative-path cases, against the approved acceptance criteria. Before and after results side by side. QA approval before production. Post-change monitoring per section 10. |
| Tier 3 | Full re-validation | The change reaches intended use, a high-risk behavior, the schema, the human checkpoint, the model family, or the retrieval corpus scope. | Fresh or updated risk assessment per Risk assessment: AI/ML system where intended use changed, re-execution of the affected protocol sections per Protocol: AI/ML system validation, an updated model card, and a validation summary or addendum approved before production. |
Two rules that stop this decision from being gamed. First, the tier is proposed by the system owner and confirmed by QA; the requester does not self-certify their own change as Tier 1. Second, a sequence of Tier 2 changes to the same prompt within one review period is assessed cumulatively, because ten small edits that each pass a targeted regression can still walk the tool away from its validated behavior. Set the cumulative trigger in section 10 of your own procedure and record it here.
| Field | Entry |
|---|---|
| Proposed tier | <<FILL: 1 / 2 / 3>> |
| Proposed by (name, role, date) | <<FILL>> |
| Cumulative-change check: prompt edits since last full re-validation | <<FILL: count and dates>> |
| Cumulative trigger reached | Yes / No |
| QA confirmed tier | <<FILL: 1 / 2 / 3>> |
| QA rationale where the tier differs from the proposal | <<FILL: or "not applicable">> |
7. Regression evidence
Complete for Tier 2 and Tier 3. Name the exact case set and its version, because “we tested it” without a versioned case set is not evidence.
| Field | Entry |
|---|---|
| Reference case set identifier and version | <<FILL: e.g. REF-TRIAGE-CASES v3, 120 cases>> |
| Case set held out from prompt development | Yes / No. If No, explain why the result is still meaningful: <<FILL>> |
| Cases re-run | <<FILL: count and selection basis>> |
| Negative-path cases included | Yes / No (malformed output, model unavailable, timeout, empty retrieval result) |
| False-negative subset included | Yes / No (known-positive cases the tool must not miss) |
| Execution date and executor | <<FILL>> |
| Raw results location | <<FILL: path or record reference>> |
7.1 Results against acceptance criteria
| Criterion | Threshold | Result before change | Result after change | Delta | Met | Comment |
|---|---|---|---|---|---|---|
<<FILL: e.g. recall on known-positive cases>> | <<FILL: e.g. not less than 0.95>> | <<FILL>> | <<FILL>> | <<FILL>> | Yes / No | <<FILL>> |
<<FILL: e.g. schema validation pass rate>> | <<FILL: e.g. 100 percent, malformed output must route to a human>> | <<FILL>> | <<FILL>> | <<FILL>> | Yes / No | <<FILL>> |
<<FILL: e.g. false-flag rate>> | <<FILL>> | <<FILL>> | <<FILL>> | <<FILL>> | Yes / No | <<FILL>> |
<<FILL: e.g. negative-path routing to human>> | <<FILL: 100 percent of induced failures route to a human>> | <<FILL>> | <<FILL>> | <<FILL>> | Yes / No | <<FILL>> |
7.2 Behavior differences worth noting even where criteria are met
| Case reference | Output before | Output after | Assessment |
|---|---|---|---|
<<FILL>> | <<FILL>> | <<FILL>> | <<FILL: benign wording change / material change in reasoning / requires investigation>> |
A criterion met on aggregate can still hide a changed behavior on a specific case class. Read the per-case differences, not only the summary number, and record what you found even when the answer is “no material difference.”
7.3 Regression outcome
| Field | Entry |
|---|---|
| All acceptance criteria met | Yes / No |
| Where not met, deviation reference | <<FILL: or "not applicable">> |
| Recommendation | Proceed / Proceed with additional controls / Do not proceed |
| Additional controls proposed | <<FILL: e.g. temporary increase in review sampling, tightened monitoring threshold>> |
8. Intended-use impact
| Question | Answer | Justification |
|---|---|---|
| Does the tool now do anything it was not approved to do? | Yes / No | <<FILL>> |
| Does the tool now process a record type, product, or process it was not approved for? | Yes / No | <<FILL>> |
| Does the output now carry more decision weight than before? | Yes / No | <<FILL>> |
| Has the human checkpoint changed in any way? | Yes / No | <<FILL>> |
| Does the intended-use statement need re-approval? | Yes / No | <<FILL>> |
| Is a fresh risk assessment required? | Yes / No | <<FILL: if Yes, reference the new assessment>> |
If the answer to any of the first four questions is Yes, the intended-use statement is re-approved and a fresh risk assessment is completed before this record can be approved. A change to intended use is not a prompt edit with paperwork; it is a new validated use of the tool.
9. Rollback plan
| Field | Entry |
|---|---|
| Previous state artifacts retained | <<FILL: prompt file and hash, model version string, schema file and hash, corpus manifest and hash, configuration file and commit>> |
| Where the previous state is stored | <<FILL: repository and tag, or controlled archive location>> |
| Rollback method, step by step | <<FILL: exact steps, including who executes and what is verified afterwards>> |
| Rollback tested | Yes / No. Date tested: <<FILL>> |
| Estimated time to revert | <<FILL>> |
| Rollback authority | <<FILL: role authorized to order the revert without waiting for a governance meeting>> |
| Rollback trigger conditions | <<FILL: e.g. schema failure rate above threshold, override rate breach, any missed known-positive case in the post-change window>> |
| Constraint on rollback | <<FILL: e.g. previous model version is deprecated by the vendor on a stated date, after which rollback is unavailable>> |
| Records created under the changed version if rolled back | <<FILL: how those records are identified, reviewed, and dispositioned>> |
The last two rows are the ones teams skip. If the previous model version will be withdrawn by the vendor, your rollback has an expiry date and you should say so before you approve the change, not discover it on the day you need it. And if you do roll back, the records the tool produced while the changed version was live still exist and still need a disposition.
10. Post-change monitoring commitment
Complete for Tier 2 and Tier 3.
| Metric | Baseline before change | Threshold that triggers action | Monitoring frequency | Duration of heightened monitoring | Owner |
|---|---|---|---|---|---|
| Override rate | <<FILL>> | <<FILL>> | <<FILL: e.g. weekly>> | <<FILL: e.g. 4 weeks or 200 calls, whichever is later>> | <<FILL>> |
| Schema validation failure rate | <<FILL>> | <<FILL>> | <<FILL>> | <<FILL>> | <<FILL>> |
| Reviewer-reported false flags | <<FILL>> | <<FILL>> | <<FILL>> | <<FILL>> | <<FILL>> |
| Findings from false-negative sampling | <<FILL>> | <<FILL>> | <<FILL>> | <<FILL>> | <<FILL>> |
<<FILL: tool-specific metric>> | <<FILL>> | <<FILL>> | <<FILL>> | <<FILL>> | <<FILL>> |
| Field | Entry |
|---|---|
| Where the monitoring data comes from | <<FILL: name your call-level log, the one that records each request, the output returned, and the human disposition taken on it>> |
| Routine monitoring procedure this hands back to | <<FILL: e.g. /templates/sop-ml-model-monitoring-drift>> |
| Who reviews the heightened-monitoring result and closes it | <<FILL>> |
Heightened monitoring after a change is a defined period with an end, not an open-ended intention. State when it stops and who signs that it stopped cleanly.
11. QA assessment and approval
Approval is recorded before the change reaches production. A change implemented before this section is signed is an uncontrolled change to a validated system and is handled as a deviation under SOP: deviation management.
| Field | Entry |
|---|---|
| QA assessment summary | <<FILL: what QA reviewed and what it concluded>> |
| Tier confirmed | <<FILL>> |
| Regression evidence adequate | Yes / No |
| Intended use unchanged or re-approved | <<FILL>> |
| Rollback plan adequate | Yes / No |
| Monitoring commitment adequate | Yes / No |
| Conditions of approval | <<FILL: or "none">> |
| QA decision | Approved / Approved with conditions / Rejected |
| Role | Name | Signature | Date |
|---|---|---|---|
| Requester | <<FILL>> | ||
| System owner | <<FILL>> | ||
| Validation lead (Tier 2 and 3) | <<FILL>> | ||
| Quality Assurance approver | <<FILL>> |
12. Implementation and closure
| Field | Entry |
|---|---|
| Implemented by (name, role) | <<FILL>> |
| Implementation date and time, with time zone | <<FILL>> |
| First call executed under the new version (call ID) | <<FILL: from the call log, establishes the boundary between versions>> |
| Post-implementation verification performed | <<FILL: what was checked immediately after go-live and by whom>> |
| Training or communication to reviewers | <<FILL: what reviewers were told, and when>> |
| Documents updated | <<FILL: model card, intended-use statement, system inventory, SOP, training material>> |
| Heightened monitoring period ended | <<FILL: date>> |
| Monitoring outcome | <<FILL: within thresholds / action taken>> |
| Rolled back | Yes / No. If Yes, date and reason: <<FILL>> |
| Record closed by (QA name, signature, date) | <<FILL>> |
The “first call executed under the new version” row matters more than it looks. It is what lets an investigator, months later, sort the call log into before and after without guessing. It is also the join between this record and whatever call-level log your tool writes, the log that captures each request, the output returned, and the human decision taken on that output, so a reviewer can trace any individual result back to the exact prompt text and model version that produced it.
13. Instructions for use
- Raise the record before the change is made, not after. A record written to describe something already in production is a deviation record, not a change record, and should be labeled as one.
- Capture the exact artifacts and hashes in section 4 at the moment you raise it, while the previous state is still live and easy to capture.
- Get the tier confirmed by QA in section 6 before you spend effort on testing, so the test scope matches the tier.
- Execute the regression against a versioned case set. If you do not have one, build it before you make the change; a change is a bad time to discover you have no reference behavior to compare against.
- Do not sign section 11 on the same day the change went live unless the timestamps genuinely support that order of events. Inspectors read timestamps.
- Close the record only after the heightened monitoring period ends, so the record carries the outcome and not just the intent.
14. Retention
| Field | Entry |
|---|---|
| Retention period | <<FILL: not less than the retention of the GxP records the tool influenced, and not less than the life of the system plus your defined period>> |
| Retention basis | <<FILL: reference your records retention schedule>> |
| Storage location | <<FILL>> |
| Format | <<FILL: electronic record under access control, with audit trail>> |
| Attachments retained with the record | Prompt files and hashes, schema files, corpus manifest, raw regression results, vendor notices, approval signatures |
This record is itself a GxP record. It is subject to the same electronic record controls as anything else the quality system holds, including audit trail, access control, and protection from alteration. If it is held in a system that permits silent editing, it is not adequate.
15. References
21 CFR Part 11 (electronic records and signatures), for the record and signature controls applied to this record and to the tool’s own outputs. 21 CFR 211.68 (automatic, mechanical, and electronic equipment), 211.100 and 211.160 (written procedures and controls), and 211.180 and 211.194 (records), as applicable to the GMP activity the tool supports. EU GMP Annex 11 (computerised systems), current version effective June 2011, with a draft revision issued for public consultation in July 2025 and a companion draft Annex 22 addressing artificial intelligence; both are drafts and are not final, so treat them as direction of travel rather than requirement. EU GMP Chapter 4 (documentation) and the change control expectations in EU GMP Part I Chapter 1. FDA guidance, “Computer Software Assurance for Production and Quality Management System Software” (current version issued 3 February 2026). The 2022 draft, dated 13 September 2022, and the final issued 24 September 2025 were both titled “Computer Software Assurance for Production and Quality System Software”; the current version added “Management” to the title and supersedes the 2025 version. ICH Q9(R1), Quality Risk Management, for sizing assessment and test effort to risk. ICH Q10, Pharmaceutical Quality System, for the change management system this record sits inside. GAMP 5 Second Edition (ISPE, 2022), for the software category logic and the change control expectations applied to custom applications; described here in original wording, not quoted. EU Artificial Intelligence Act, Regulation (EU) 2024/1689, which applies in phases with obligations staged over time. Confirm the current application dates and whether your system falls in scope before relying on them.
This is general guidance to adapt to your own quality system, not legal or regulatory advice. Confirm the current version and clause numbers of each reference, and the current status of any draft instrument, before issue.
Filled specimen
The following shows the same record completed for an illustrative internal deviation-triage assistant that suggests a preliminary criticality tier for a QA reviewer to confirm or override. The company, tool, version strings, and results are invented for illustration. The vendor is described generically; where a real vendor is named anywhere in this template it is a neutral example only.
Control header (filled)
| Field | Entry |
|---|---|
| Record number | AICC-2026-014 |
| Linked change request | CR-2026-0873 |
| AI tool name and ID | Deviation Triage Assistant, DTA-01 |
| Validated state reference | VSR-AI-004 v2.0, approved 14 February 2026 |
| Record raised by | J. Okafor, AI System Owner |
| Date raised | 08 June 2026 |
| Status | Closed |
What changed (filled)
| Element | From | To | Hash | Evidence |
|---|---|---|---|---|
| Prompt version | PROMPT-DTA v2.3 | PROMPT-DTA v2.4 | 9f1c…a072 to 4b8e…c119 | Repo dta-tool, commit 7ad20c9 |
| Prompt text diff | Added two worked examples covering aseptic-fill environmental excursions; no change to the criticality rubric or the output instruction | Diff attached, Attachment 1 | ||
| Model vendor | Commercial LLM API provider, unchanged | Unchanged | Supplier assessment SA-2025-11 | |
| Model identifier and exact version string | vendor-lm-3.5-20260112 | vendor-lm-3.5-20260112, no change | API response capture, Attachment 2 | |
| Model parameters | temperature 0.0, top-p 1.0, max output tokens 800 | Unchanged | config/dta.yaml, commit 7ad20c9 | |
| Output schema version | SCHEMA-DTA v1.1 | Unchanged | 2c40…ff81 | Repo dta-tool |
| Retrieval corpus version | CORPUS-DTA-2026Q1, 412 documents, cut-off 31 March 2026 | Unchanged | Corpus manifest | |
| Deterministic code around the model | dta-core v1.6 | Unchanged | Repo dta-tool |
Requester and reason (filled)
| Field | Entry |
|---|---|
| Requester | M. Ibrahim, QA Deviation Coordinator |
| Trigger | Monitoring signal: override rate on environmental monitoring excursions from the aseptic fill area reached 31 percent over eight weeks against a threshold of 25 percent, while the site-wide rate stayed at 9 percent |
| Business reason | The tool consistently under-rates environmental excursions in the fill area because the prompt examples do not cover that record shape. Reviewers are correcting the same category of miss repeatedly |
| Vendor-forced | No |
| Related alert | MON-2026-031, override-rate threshold breach |
| Alternatives considered | Do nothing was rejected because the override pattern is systematic, not random. Retraining reviewers was rejected because the reviewers are already correcting it, which is the control working; the tool is the thing that is wrong. Removing fill-area deviations from the tool’s scope was considered and held in reserve if this change does not resolve the pattern |
Classification (filled)
| # | Question | Answer |
|---|---|---|
| Q1 | Intended use, decision, or record population altered | No |
| Q2 | High-risk behavior touched | No. Criticality suggestion is advisory; QA confirms every tier with a reason |
| Q3 | Output schema altered | No |
| Q4 | Human checkpoint altered | No |
| Q5 | Different model family, vendor, or hosting | No |
| Q6 | Retrieval corpus scope altered | No |
| Q7 | Instructional content, rubric, or examples altered | Yes |
| Q8 | Different version string within the same family | No |
| Q9 | Constraint added, removed, or reworded | No |
| Q10 | Confined to non-instructional text | No |
| Field | Entry |
|---|---|
| Proposed tier | Tier 2, targeted regression |
| Proposed by | J. Okafor, 08 June 2026 |
| Prompt edits since last full re-validation | 2 (v2.2 on 03 April 2026, v2.3 on 11 May 2026) |
| Cumulative trigger reached | No. Site trigger is 4 prompt edits or 6 months since full re-validation, whichever comes first |
| QA confirmed tier | Tier 2 |
| QA rationale | Agreed. Examples added, rubric untouched, schema and checkpoint untouched. QA added a condition that the false-negative subset be expanded to include ten fill-area cases before execution |
Regression evidence (filled)
| Field | Entry |
|---|---|
| Reference case set | REF-DTA-CASES v4, 140 cases, expanded from v3 by 10 fill-area cases per the QA condition |
| Held out from prompt development | Yes for 130 cases. The 10 new fill-area cases were drawn from closed 2025 deviations not used to write the new examples; the two examples added to the prompt came from a separate 2024 set |
| Cases re-run | 140, full set |
| Negative-path cases included | Yes, 8 cases: malformed JSON, truncated output, API timeout, API 500, empty retrieval, oversized input, non-English free text, null required field |
| False-negative subset included | Yes, 24 known-critical cases the tool must not under-rate |
| Execution date and executor | 15 to 16 June 2026, R. Delacroix, Validation Lead |
| Raw results | VAL-EXEC-2026-119, attached |
| Criterion | Threshold | Before | After | Delta | Met |
|---|---|---|---|---|---|
| Recall on known-critical cases (must not under-rate) | Not less than 0.95 | 0.92 (22 of 24) | 1.00 (24 of 24) | +0.08 | Yes |
| Agreement with QA-confirmed tier, all cases | Not less than 0.80 | 0.81 | 0.86 | +0.05 | Yes |
| Agreement, fill-area subset | Not less than 0.80 | 0.60 | 0.85 | +0.25 | Yes |
| Schema validation pass rate on valid input | 100 percent | 100 percent | 100 percent | 0 | Yes |
| Negative-path routing to a human | 100 percent of 8 induced failures | 8 of 8 | 8 of 8 | 0 | Yes |
| Over-rating rate (false escalation) | Not more than 0.15 | 0.09 | 0.11 | +0.02 | Yes |
| Case reference | Before | After | Assessment |
|---|---|---|---|
| REF-DTA-071 | Minor, rationale cites “single excursion within alert limit” | Major, rationale cites “Grade A excursion, action limit, aseptic fill in progress” | Material change, and the intended one. QA-confirmed tier for this closed deviation was Major |
| REF-DTA-112 | Major | Major, rationale reworded | Benign wording change, no change in output tier |
| REF-DTA-129 | Minor | Major | Material change, and incorrect. QA-confirmed tier was Minor. Over-rating, absorbed by the reviewer, counted in the over-rating rate above and flagged for monitoring |
| Field | Entry |
|---|---|
| All acceptance criteria met | Yes |
| Deviation raised | None |
| Recommendation | Proceed with additional controls |
| Additional controls | Over-rating rate rose from 0.09 to 0.11. Monitor weekly rather than monthly for the first four weeks and report to the AI governance forum |
Intended-use impact (filled)
| Question | Answer | Justification |
|---|---|---|
| Does anything new | No | Same output, same records, same reviewer step |
| New record type, product, or process | No | Fill-area deviations were already in scope; the tool was performing badly on them, which is not the same as them being out of scope |
| More decision weight | No | Suggestion remains advisory, QA confirms every tier with a reason |
| Human checkpoint changed | No | |
| Intended-use statement re-approval needed | No | |
| Fresh risk assessment required | No. RA-AI-004 v1.2 reviewed on 12 June 2026 and confirmed still applicable, noted in the assessment file |
Rollback (filled)
| Field | Entry |
|---|---|
| Previous state retained | PROMPT-DTA v2.3 (hash 9f1c…a072), config/dta.yaml at commit 3e91b40, repository tag dta-v2.3-validated |
| Storage | Repository dta-tool, protected tag, plus controlled archive QA-ARC/AI/DTA |
| Rollback method | IT reverts to tag dta-v2.3-validated, restarts the service, runs the 12-case smoke set, and confirms the prompt hash in the first live call record matches 9f1c…a072. Executed by IT, verified by the system owner |
| Rollback tested | Yes, 17 June 2026, in the qualified test environment, 6 minutes end to end |
| Estimated time to revert | Under 30 minutes including verification |
| Rollback authority | QA Manager or AI System Owner, either acting alone, without waiting for the governance forum |
| Rollback triggers | Any known-critical case under-rated in the post-change window; over-rating rate above 0.20 in any monitored week; schema failure rate above 1 percent; override rate above 35 percent in any week |
| Constraint on rollback | None. The model version is unchanged and the prompt artifact is held internally, so rollback does not depend on the vendor |
| Records created under the changed version if rolled back | Identified by prompt hash in the call log. All affected deviations would be re-triaged by a QA reviewer within 5 working days and the re-triage recorded on each deviation record |
Monitoring commitment (filled)
| Metric | Baseline | Action threshold | Frequency | Duration | Owner |
|---|---|---|---|---|---|
| Override rate, fill-area subset | 31 percent | Above 25 percent, or below 3 percent | Weekly | 6 weeks | M. Ibrahim |
| Override rate, all deviations | 9 percent | Above 25 percent, or below 3 percent | Weekly | 6 weeks | M. Ibrahim |
| Over-rating (false escalation) rate | 0.11 in regression | Above 0.20 in any week | Weekly | 6 weeks | R. Delacroix |
| Schema validation failure rate | 0 percent | Above 1 percent | Weekly | 6 weeks | IT system administrator |
| False-negative sampling: 20 unflagged-as-Major deviations per month reviewed by QA | 0 misses in the last cycle | Any single miss of a case QA rates Major | Monthly | 3 cycles | J. Okafor |
Approval and closure (filled)
| Field | Entry |
|---|---|
| QA assessment summary | Tier 2 confirmed. Regression executed on an expanded case set with the fill-area gap closed and the false-negative subset clean at 24 of 24. Over-rating rose slightly and is controlled by weekly monitoring and the reviewer step. Rollback tested and independent of the vendor. Approved with the monitoring condition |
| QA decision | Approved with conditions |
| Requester | M. Ibrahim, signed 19 June 2026 |
| System owner | J. Okafor, signed 19 June 2026 |
| Validation lead | R. Delacroix, signed 19 June 2026 |
| QA approver | S. Whitfield, QA Manager, signed 22 June 2026 |
| Implemented by | IT system administrator, T. Lindqvist |
| Implementation date and time | 23 June 2026, 07:12 Central European Time |
| First call under the new version | Call ID DTA-2026-06-23-0007 |
| Post-implementation verification | Prompt hash in the first three live call records confirmed as 4b8e…c119 by the system owner on 23 June 2026 |
| Reviewer communication | Briefing note issued to the 6 QA deviation reviewers on 22 June 2026 explaining that fill-area suggestions should improve and that they must continue to confirm every tier with a reason |
| Documents updated | Model card REC-AI-004 v1.3, system inventory entry DTA-01, reviewer job aid JA-QA-018 |
| Monitoring period ended | 04 August 2026 |
| Monitoring outcome | Fill-area override rate settled at 11 percent, site-wide at 8 percent, over-rating 0.12, no schema failures, false-negative sampling clean across 3 cycles. Returned to routine monitoring |
| Rolled back | No |
| Record closed by | S. Whitfield, QA Manager, signed 07 August 2026 |
Reading it: the change was triggered by a monitored metric rather than by someone deciding the prompt could be better, the tier was confirmed by QA rather than self-certified, the regression was run against a versioned case set that included the cases the tool must never miss, a small degradation was found and honestly recorded rather than smoothed over, the rollback was tested before approval, and the record was not closed until the monitoring period ended with an outcome. That sequence, signal to assessment to evidence to approval to monitored outcome, is what an inspector is looking for.
Common inspection findings this record prevents
- A validated AI tool is running a prompt that does not match the prompt in the validation package, and nobody can say when or why it changed.
- The model version in production differs from the version in the validation report because the vendor updated it and nobody assessed the update.
- Prompt changes are made under a general IT change ticket that contains no assessment of validated behavior and no regression evidence.
- Every change is classified as low impact by the person who made it, with no independent quality confirmation of the classification.
- Regression testing is claimed but there is no versioned reference case set, so “it still works” cannot be demonstrated.
- The tool’s output schema changed and a downstream consumer silently broke, discovered later during a record review.
- The retrieval corpus was refreshed with new source documents, changing what the tool can cite, with no assessment.
- No rollback path exists, or the rollback depends on a model version the vendor has already withdrawn.
- Change records are approved after the change was already live, with signature dates that prove it.
- A series of small prompt edits accumulates until the production behavior no longer resembles the validated behavior, with each individual edit defensible and the aggregate never assessed.
- The change record is closed on the day of implementation, so it records an intention to monitor rather than a monitored outcome.
How to adapt this record
- Set your record numbering, owner, and the change control procedure this record feeds in the control header and section 11.
- Rewrite the five scope categories in section 2 to match how your tool is actually built. If you do not use retrieval, delete those rows rather than leaving them permanently marked “not applicable.”
- Set your own cumulative-change trigger in section 6. Four prompt edits or six months is a starting point, not a rule; base it on how much your tool’s behavior moves per edit.
- Replace the acceptance criteria in section 7.1 with the criteria from your own validation package, including the exact thresholds. The criteria here are illustrative.
- Name your reference case set and its version in section 7. If you do not have one, build it first; this record depends on it.
- Point section 10 at your routine monitoring procedure so heightened monitoring hands back cleanly rather than trailing off.
- Set the retention period in section 14 against your own records retention schedule and the retention of the GxP records the tool influenced.
- Confirm every regulation in section 15 against the current published version before issue, and check the status of the draft Annex 11 revision and draft Annex 22, which were not final at the time of writing.