Data Criticality and Data Risk: Classifying Records to Right-Size Controls

Not every record carries the same weight. A GMP batch release calculation that feeds a disposition decision is not in the same category as the line in a maintenance log noting that a fan filter was wiped down. Treat them identically and you waste effort on the trivial while under-protecting the records that actually move quality and patient-safety decisions. The discipline that prevents both failure modes is data classification: scoring each record stream by how much it matters (criticality) and how likely it is to be wrong or manipulated (risk), then setting controls in proportion.

This is the model the UK regulator built its data integrity guidance around, and it is one of the most heavily tested topics in a data integrity interview. Get the distinction between criticality and risk right, show you can apply it to a real instrument, and you signal that you understand the whole point of risk-based quality: finite control budget, deployed where it changes outcomes.

The regulatory basis: where data criticality and data risk come from

The model has a clear origin and a clear lineage. You should be able to name the documents.

MHRA “GxP Data Integrity Definitions and Guidance for Industry” (March 2018). This is the primary source. It defines data criticality and data risk as separate concepts and states that the effort applied to assuring data integrity should be commensurate with the risk to product quality and patient safety. Two ideas from it are worth carrying into an interview, in your own words:

The guidance makes the point that data is generated across a spectrum, from simple machines through to complex, highly configurable computerised systems, and that the inherent risk to data integrity differs depending on how far the system (or the data it generates) can be configured, and therefore potentially manipulated.

It also separates the two concepts: criticality reflects how the data is used to influence the decisions being made, while the risk to data integrity is a combination of that criticality, the inherent risks in how the data is generated, and the controls applied to manage those risks.

So criticality is about consequence (what decision does this record drive, how bad is a wrong decision), and risk is the broader combination that includes criticality plus how the data is generated and what controls already sit on it.

PIC/S PI 041-1 “Good Practices for Data Management and Integrity in Regulated GMP/GDP Environments” (July 2021). The global inspectorate equivalent. It carries the same logic and is what many non-UK inspectors cite. It is explicit that data governance and the level of control should be based on a documented risk assessment that considers data criticality and the vulnerability of the data to alteration.

WHO Technical Report Series 996, Annex 5 (2016), “Guidance on good data and record management practices.” Earlier than the MHRA 2018 final, and an early influential statement that controls should be proportionate to risk, with worked criticality language. The risk-based lineage is older still: the MHRA published GMP data integrity definitions and guidance in 2015, before WHO TRS 996, so this is one strand of a developing consensus rather than the single origin point.

FDA “Data Integrity and Compliance With Drug CGMP, Questions and Answers” (final guidance, December 2018). The FDA does not use the exact “criticality versus risk” vocabulary, but it endorses a risk-based approach to audit trail review and to the level of validation and control. FDA expects you to focus effort on the records of highest consequence.

EU GMP Annex 11 and Chapter 4, plus 21 CFR Part 11. These are the control-requirement layers that your classification feeds into. Annex 11 paragraph 1 requires risk management throughout the lifecycle of a computerised system “taking into account patient safety, data integrity and product quality.” Your classification is how you decide which Annex 11 and Part 11 controls apply at what depth.

The quality rationale underneath all of it: data integrity controls cost money, time, and operational friction. Apply them flat across every record and you will either bankrupt the effort or dilute it. Apply them in proportion to consequence and likelihood and you protect the records that protect the patient.

Cross-links: data-integrity-foundations, ALCOA+ in detail, quality-risk-management.

The two concepts, kept straight

The single most common interview failure on this topic is blurring criticality and risk. Keep them clean.

Data criticality

Criticality answers one question: if this record is wrong, how bad is the decision that goes wrong because of it?

It is a property of the record’s use, not of the system. A weighment that feeds a batch formulation is critical because it drives a release decision; the same balance used to weigh a non-GMP cleaning sample is low criticality even though it is the identical instrument. Criticality is driven by:

• The decision the data supports (batch release, dose, sterility assurance, stability shelf-life, clinical safety) versus a supporting or informational use.
• The directness of the link to product quality and patient safety.
• Whether the record is a primary GMP record (the data of record) or a secondary check.
• Whether it is part of a regulatory submission or a release certificate.

Data risk (data integrity risk)

Risk answers: how likely is this record to be wrong, missing, or manipulated, given how it is generated and what controls already exist?

Risk is a property of the system and process. It is driven by:

• Generation method: manual paper entry, hybrid (instrument prints, human transcribes), or fully electronic.
• Configurability and manipulability: a simple bench instrument with a fixed display versus a highly configurable chromatography data system where integration parameters, clocks, and audit trails can be altered by a privileged user.
• Complexity and number of human touchpoints: every transcription, re-entry, or manual calculation is an opportunity for error or falsification.
• Existing controls: access control, audit trail, locked methods, automated transfer to a LIMS, second-person review. Strong controls lower residual risk; their absence raises it.

The MHRA model ties them together: risk to data integrity is a function of criticality, the inherent risks of generation, and the controls applied. So criticality is an input to risk, not a synonym for it. A low-criticality record on a wide-open, easily edited spreadsheet can carry meaningful integrity risk; a high-criticality result locked behind unique logins, a validated method, and an automatic transfer can carry low residual risk despite its high consequence.

The output you care about is the residual risk after controls, because that is what tells you whether you have done enough.

The classification procedure, step by step

This is the part interviewers want to see you operationalise. Here is a sequence that holds up in an audit.

Step 1: Inventory the data and define the units

You cannot classify what you have not listed. Build a data inventory or data flow map first. Work at the level of a record stream, meaning a defined output from a process or system (for example: “HPLC assay results for finished-product release”, “environmental monitoring plate counts”, “balance weighments for dispensing”). Do not try to classify individual values; classify the stream.

For each stream capture: the source system or instrument, how the data is generated, where it flows (intermediate files, manual transcription, LIMS, report), where the data of record lives, and the decision it ultimately feeds. The data-lifecycle-and-metadata and data-governance-framework work feeds this directly.

Step 2: Score criticality

Rate the consequence of the record being wrong. A defensible three-tier scale:

Anchor each tier with worked examples in your procedure so two assessors land in the same place.

Step 3: Assess inherent data integrity risk (before controls)

Score how vulnerable the stream is to error or manipulation as generated, ignoring controls for now. Useful dimensions:

• Generation: fully electronic with automated transfer (lower) → hybrid with transcription (higher) → fully manual paper (highest for transcription error, though sometimes lower for manipulability than a configurable system).
• Manipulability: fixed-display instrument (low) → configurable software where data can be deleted, reprocessed, or the clock changed (high).
• Human touchpoints: count the transcriptions, manual calculations, and re-keys.
• Complexity: number of systems the data crosses before reaching its record of record.

Step 4: Evaluate existing controls and derive residual risk

List the controls already on the stream: unique user accounts and access levels, audit trail enabled and reviewed, locked/validated methods, automated interface to LIMS, second-person verification, time synchronisation, backup and restore. Reduce the inherent risk by the strength of these controls to land on residual risk.

A practical scoring approach is a matrix. Map criticality (consequence) against residual likelihood of error or manipulation to get a priority band:

Step 5: Decide the control set (right-sizing)

For each priority band, define the expected controls. This is where classification earns its keep: it tells you what is mandatory, what is recommended, and what is unnecessary.

The decision is not “do we control this” (everything gets baseline GxP control) but “how deep.” Audit-trail review frequency is the classic dial: daily-equivalent review of every critical chromatographic run, periodic sampling for medium, none required for low. See operationalizing-audit-trail-review and audit-trail-design-and-review.

Step 6: Document, approve, and feed downstream

Record the classification in a controlled assessment, signed by QA and the system or process owner. Feed the output into:

• The validation scope and rigour for the system (csv-risk-assessment-methodology, gamp5-csv-framework).
• The data governance plan and audit-trail review SOPs.
• The periodic review schedule.

Step 7: Review on a trigger and on a cycle

Re-assess when the system changes, the process changes, a deviation or inspection finding implicates the data, or on a defined periodic cycle (commonly annually for high-criticality streams, longer for low). Classification is not a one-time exercise.

A worked example: classifying an HPLC release assay

Take a single concrete stream and run it through end to end. The instrument is a chromatography data system (a category that includes commercial products such as Empower, OpenLab, or Chromeleon) generating the assay result that releases a finished drug product.

Stream: Finished-product HPLC assay result, used for batch disposition and printed on the certificate of analysis.

Step 2, criticality: High. It directly drives release and goes onto the CoA. A wrong result releases out-of-specification product or rejects good product.

Step 3, inherent risk: High. A chromatography data system is highly configurable. Integration parameters can be changed, injections can be deleted or reprocessed, the system clock can in principle be altered, and historically this class of system is the single most cited in data integrity findings. The data is electronic, which removes transcription risk, but manipulability is high.

Step 4, controls and residual risk: Suppose the controls in place are: unique logins, role-based access with admin segregated from analysts, audit trail enabled and reviewed for every release batch, a locked validated integration method, processing limited so analysts cannot alter the method, time synchronised to a controlled source, and results transferred to the LIMS. With those, residual likelihood of an undetected error or manipulation drops to low-to-medium. Without audit-trail review or with shared logins, it stays high.

Step 5, priority and control depth: High criticality with low-to-medium residual likelihood places it in the Critical/High band. Expected controls: routine audit-trail review of every release run (not sampled), no shared accounts, no analyst-level reprocessing without justification and review, periodic data integrity self-audit of the system.

Filled-in assessment row:

Now contrast it with a low-criticality stream on the same kind of dial: a non-GMP stability chamber’s temperature trend used only for facility monitoring, electronic, fixed-output, behind access control. Criticality low, inherent risk low-medium, residual low. Priority band low. Required depth: backup and access control, no routine audit-trail review. Same rigour applied to both would be a misallocation; the model is what lets you defend treating them differently to an inspector.

See chromatography-data-system-integrity for the CDS-specific control detail behind this example.

A second worked example: a hybrid balance weighment

Contrast the high-criticality CDS with a hybrid stream that needs different controls, so the dial is visible.

Stream: Dispensing weighment for a GMP batch. The balance prints a paper ticket; the operator transcribes the weight onto the batch record by hand; a second person verifies.

Criticality: High. The weighment feeds the batch formulation and therefore release. A wrong weight is a wrong dose.

Inherent risk: Medium to high, but for a different reason than the CDS. The balance is a simple fixed-output instrument with low manipulability, so the configurable-software risk is low. The risk lives in the transcription: a hand-copied number is the classic point of error and of falsification, and a paper ticket can be discarded and re-printed.

Controls and residual risk: Controls are a calibrated balance with the printed ticket retained as raw data, independent second-person verification of the transcription against the ticket, the ticket attached to the batch record, and a controlled batch record. With the ticket retained and a real second check, residual likelihood drops to low-medium. Without ticket retention (transcription only) or without the second check, it stays high.

Priority and control depth: High criticality with low-medium residual likelihood puts it in the High band. Required depth: retain the printout as raw data, enforce contemporaneous second-person verification, and treat a missing or re-printed ticket as a deviation. There is no audit trail to review because the instrument has none; the control that matters is raw-data retention and the second check, not audit-trail review.

The lesson: two high-criticality streams can need completely different controls. The CDS needs audit-trail review because its risk is electronic manipulation; the balance needs raw-data retention and a second check because its risk is transcription. The classification model is what tells you which control to spend on. See hybrid-paper-electronic-records for the hybrid-record control detail.

An optional numeric scoring rubric

The three-tier qualitative scale is enough for most programs, but some quality systems prefer a number so classifications sort and trend. If you use one, anchor every level, or the number becomes the hand-wave the model is supposed to prevent.

A workable scheme scores two axes and multiplies them.

Criticality (consequence), C:

Residual likelihood of error or manipulation after controls, L:

Priority = C x L. Band the product: 12 to 16 critical, 6 to 9 high, 3 to 4 medium, 1 to 2 low. A score of 16 (critical consequence, high residual likelihood) is the case that gets attention first. The number is a sorting and communication aid only; the written rationale behind each axis score is still the substance an inspector reads. Never record a score without the narrative.

Roles and responsibilities

Classification is a cross-functional act, and inspectors will ask who decided. Define it clearly.

The pattern that holds up: the owner and SME propose, QA challenges and approves, and the rationale is written down. A classification signed only by IT, or only by the lab, with no QA challenge, is a finding waiting to happen.

See gxp-roles-responsibilities and data-governance-roles-and-careers.

Acceptance criteria: what good looks like

You know the classification is done well when:

• Every GxP record stream is covered. There is a complete inventory, and nothing is “unclassified.” Gaps are the most common audit hit.
• Criticality and risk are scored separately, with documented rationale for each, not collapsed into a single hand-wave.
• Residual risk, not inherent risk, drives the decision. The assessment shows the controls and the post-control conclusion.
• The scale is anchored. Two independent assessors classifying the same stream reach the same tier because the procedure gives concrete examples per tier.
• The output is traceable to controls. You can point from a high-priority classification to the specific audit-trail review frequency, access model, and validation rigour it triggered.
• It is approved by QA and the owner, dated, and under change control.
• It is reviewed on triggers and on a cycle, with evidence of re-assessment after changes.
• It is defensible verbally. The owner can stand in front of an inspector and explain why a given record is high or low and what that drove.

Common mistakes and real inspection-finding patterns

These are the patterns regulators cite, described generically.

Treating every record the same. Either everything is “critical” (so the program is unaffordable and nobody actually does the daily audit-trail review they promised) or nothing is differentiated (so high-consequence records get baseline-only control). Both are misallocations the model exists to prevent.

Confusing criticality with risk. Calling a record “low risk because it is low criticality” while ignoring that it sits on a wide-open spreadsheet with no audit trail. Criticality is an input to risk, not the whole of it.

Stopping at inherent risk. Scoring how scary the system is in the abstract and never evaluating the controls already in place, so the assessment over-states risk and effort lands in the wrong place. Or the reverse: claiming low risk based on controls that exist on paper but are not actually operating (audit trail “enabled” but never reviewed).

No data inventory. Classifying the systems you remembered and missing record streams entirely, especially hybrid paper-and-electronic flows, standalone instruments, and spreadsheets. Inspectors routinely find an uncontrolled balance printout or a local spreadsheet that never made the list. See hybrid-paper-electronic-records and infrastructure-qualification-and-spreadsheet-validation.

Audit-trail review applied flat or not at all. Either reviewing nothing (a classic finding) or promising 100% review of everything and not delivering. The model’s job is to tell you which audit trails get reviewed how often. A risk-based review program with no documented classification behind it is hard to defend.

Classification done by IT alone, with no quality challenge. The people who built the system are not best placed to judge consequence to product and patient. QA absence shows.

Static classification. Done once at validation, never revisited after the system was reconfigured, the process changed, or a deviation exposed a gap.

Over-reliance on a number. A risk score of “12” with no narrative. The number is a communication aid; the rationale is the substance. Inspectors read the rationale.

Interview-ready: questions and how to answer

“What is the difference between data criticality and data risk?” Criticality is the consequence of the record being wrong, driven by the decision it supports and its link to product quality and patient safety. Risk is the likelihood the record is wrong, missing, or manipulated, driven by how the data is generated, how manipulable the system is, and what controls exist. Per the MHRA 2018 guidance, integrity risk is a function of criticality plus inherent generation risk plus the controls applied. Criticality is an input to risk, not a synonym.

“Walk me through how you would classify the data from a chromatography data system.” Use the worked example above: high criticality (drives release, on the CoA), high inherent risk (configurable and manipulable, the most-cited system class), then evaluate controls (unique logins, segregated admin, locked method, audit trail reviewed per run, time sync, LIMS transfer) to land on residual risk, conclude Critical/High band, and require 100% audit-trail review per release run plus periodic DI self-audit.

“Why do we bother classifying? Why not control everything to the highest standard?” Because control budget is finite. Flat maximum control either is not delivered (paper promises) or dilutes attention away from the records that actually move release and safety decisions. Right-sizing is the whole point of risk-based quality, and Annex 11 paragraph 1 and the MHRA guidance both require effort commensurate with risk to patient safety, data integrity, and product quality.

“Where does this fit with ALCOA+ and CSV?” Classification tells you where to spend the ALCOA+ effort and how rigorous the computerised system validation needs to be. A high-criticality, high-risk system gets the deepest validation and the most intensive integrity controls; a low one gets baseline. It is the bridge from principles (ALCOA+ in detail) to proportionate execution (csv-risk-assessment-methodology).

“What is the most common thing that goes wrong?” Either treating everything as critical so the program is not actually executed, or missing record streams entirely because there was no inventory, especially hybrid and spreadsheet data. And reviewing no audit trails, which is one of the most-cited data integrity findings overall.

“How do you keep two assessors from scoring the same record differently?” Anchor the scale with concrete examples per tier in the procedure, require a written rationale, and have QA review for consistency across streams. Calibration, not just a number.

“How often do you re-assess?” On triggers (system change, process change, deviation or inspection finding implicating the data) and on a periodic cycle scaled to criticality, commonly annual for high-criticality streams.

Practical tips

• Classify at the record-stream level, never the individual value. A handful of well-defined streams per system is manageable; thousands of values is not.
• Build the data flow map first. Most classification errors are inventory errors. You cannot right-size controls for data you have not mapped. The map also surfaces the hybrid and transcription touchpoints that carry the highest error risk.
• Make residual risk explicit. Always show inherent risk, the controls, and the post-control conclusion. An assessment that jumps straight to a final score with no control narrative does not survive an inspection.
• Anchor your scales. The single biggest driver of a defensible program is concrete examples per tier so the scoring is repeatable.
• Tie the output to a specific control. “High priority” means nothing unless it maps to “audit trail reviewed every release run, no shared logins, validated locked method.” Write that mapping into the procedure.
• Use the model to defend doing less, not just more. Its real value in an inspection is justifying why a low-criticality stream gets baseline control. That is a legitimate, expected use of risk-based thinking.
• Keep QA in the approval loop and keep it dated and under change control. An unapproved or undated classification is treated as no classification.

Related reading

• data-integrity-foundations and ALCOA+ in detail for the principles this model implements.
• quality-risk-management for the underlying risk methodology.
• csv-risk-assessment-methodology and gamp5-csv-framework for how classification drives validation rigour.
• audit-trail-design-and-review and operationalizing-audit-trail-review for the control most directly right-sized by classification.
• data-governance-framework and data-lifecycle-and-metadata for where classification lives in the program.
• di-gap-assessment-methodology for assessing existing systems against this model.
• chromatography-data-system-integrity and hybrid-paper-electronic-records for the system types most often classified as high.