Log: Complaint Trending, Rate Normalisation, and Threshold Review

This is a ready-to-use trending record for product quality complaints. It defines what is trended, how rates are normalised to distribution volume, how thresholds are set before the data arrives, what happens when one is crossed, and how the output reaches the annual product review and management review. Replace every <<FILL: ...>> placeholder, set your own document numbers and dates, and route it through your normal document control before use. A worked specimen with real arithmetic follows.

This document is educational and is written to be adapted. Confirm every cited regulation against the current published source, and set your own thresholds against your own product history rather than adopting the numbers used in the specimen. Maintaining this log does not by itself create compliance; it only structures analysis your quality system still has to perform and act on.

Individual complaints often look benign. Trending is how a stream of separate anecdotes becomes a quality signal, and it is the mechanism by which a defect that no single file could justify investigating gets found. Use it with the complaint handling SOP and the complaint investigation report form.

Document control header

1. Dimensions trended

Total complaint count is close to useless on its own. It rises with sales, falls with supply interruption, and tells you nothing about where a defect lives. Trend across the dimensions below, and record the analysis for each on every review, including the dimensions that produced nothing.

Cross the dimensions, do not just list them. A defect type that is flat at product level and concentrated in one lot is a manufacturing excursion. The same defect type flat at product level and spread evenly across every lot is a design, formulation, or specification problem. Those two conclusions need different actions, and only the crossed view distinguishes them. The specimen in section 6 turns on exactly this point.

2. Rate normalisation

2.1 Why raw counts fail

Raw counts move with volume. A product whose distribution doubles will roughly double its complaint count with no change whatever in quality, and a company reading raw counts will investigate a phantom. Worse in the other direction: a product whose distribution triples while its complaint count only doubles has a falling rate that a count-based review will read as a worsening trend, and a real rate increase during a supply shortage can be entirely masked. Normalisation is not a refinement; without it the trend is not interpretable.

2.2 The calculation

Complaint rate = (number of complaints of the defined type in the period / units distributed of the defined denominator in the period) x multiplier

In plain arithmetic: divide the complaint count by the units distributed, then multiply by 10,000 or 100,000 to get a number that is readable rather than a long string of zeros.

2.3 The rules that stop the arithmetic from lying

1. Match the numerator to the denominator. Complaints on one lot are divided by units distributed of that lot, not by total product distribution. Getting this wrong makes every lot-level rate look tiny and hides exactly the signal you are looking for.
2. Set a minimum denominator. Below <<FILL: e.g. 5,000 units>> a single complaint produces a rate so large it swamps every comparison. Where the denominator is below the minimum, do not calculate a rate; apply the count-based rule in section 3 instead and record that you did.
3. Account for the lag. A lot distributed last month has had one month of exposure; a lot distributed nine months ago has had nine. Comparing their cumulative rates directly is comparing different exposure windows. Either compare lots at a matched age (<<FILL: e.g. cumulative rate at 90 days after first distribution>>), or record the exposure period alongside the rate so the comparison is visible. Late-appearing defects such as stability-related failures are invisible in a young lot.
4. Fix the denominator source and do not change it silently. If the distribution report changes definition, the entire historical series changes with it. Record any change to the denominator definition in the revision history and restate the baseline.
5. Do not annualise a partial period. A quarter with one complaint does not support a projection.
6. Record zero. A defect type with no complaints this period is recorded as zero, not omitted. An omitted row is indistinguishable from an analysis that was never run.

2.4 Worked calculation, shown longhand

Product <<FILL>>, defect type <<FILL>>, lot <<FILL>>:

3. Threshold definitions and how they were set

3.1 The principle

Thresholds are set before the data arrives and are written down. “We will review the trends and use judgement” is not a threshold, and a review that decides after the fact whether a number was concerning will always find a reason not to escalate. Pre-defining the trigger removes that discretion at the moment it is least reliable.

A threshold is a detection device, not a verdict. Crossing one obliges you to investigate, not to conclude that a defect exists. Set them at a level where you accept some investigations will find nothing, because a threshold tuned so that every crossing is a real defect is tuned far too high to catch anything early.

3.2 The thresholds

3.3 How the baselines were set

On statistical limits. Where volumes support it, an attribute control chart gives defensible limits and detects a shift earlier than a fixed multiple. Where volumes are low, complaint counts are sparse, or the process has changed within the window, a stated multiple of a stated baseline is more honest than a control limit calculated on data that does not meet the chart’s assumptions. Pick one, state which, and state why. A control chart applied to eight data points is decoration. See statistics in quality, Cpk and control charts.

Do not move a threshold because it fired. If the review concludes a threshold is set too low, that is a legitimate finding, but it is changed through document control with a recorded rationale and it takes effect prospectively. Raising a threshold inside the review that it just triggered, so that the crossing disappears, is the single most damaging thing that can happen to a trending system, and it is visible in the revision history.

4. Periodic review record

Complete one record per review period. Every field is part of the record.

4.1 Summary by defect type

4.2 Summary by lot

4.3 Summary by presentation, market, severity, and conclusion

4.4 Review conclusions

5. Threshold-crossing escalation

5.1 What a crossing obliges

A crossing is not a conclusion. It obliges you to do the following, and to record each:

1. Verify the data before acting. Confirm the numerator (are these genuinely the same defect type, or has a classification drift merged two different things), confirm the denominator (is the distribution figure for the right lot, the right unit, the right period), and confirm no duplicate complaint numbers are inflating the count. A crossing driven by a data error is a data-integrity finding, not a quality signal, and it is found in minutes.
2. Open a trend investigation within <<FILL: e.g. 5 business days>>, or link the crossing to an investigation already open on the same facts rather than duplicating it.
3. Link every contributing complaint to that investigation, including the ones closed as unconfirmed. The population is the evidence.
4. Reassess the reportability of the contributing complaints in light of the aggregate. A pattern can make reportable what a single complaint did not. Use the reportability decision matrix.
5. Assess product impact and field action across the affected population, not just the lot that crossed.
6. Decide CAPA, and where a CAPA is opened, link the whole contributing population to it so effectiveness can be measured against that population. See what is a CAPA and CAPA effectiveness verification.
7. Define the effectiveness measure in the same terms as the threshold: the same defect type, the same normalised rate, against the same baseline, over a stated number of periods. An effectiveness check written in different units from the trigger cannot demonstrate the trigger was resolved.

5.2 Where the review decides not to escalate

Permitted, and it has to be reasoned in the record. State the specific facts: a data error identified and corrected, a denominator below the minimum with the count rule applied instead, a known and already-investigated cause with the existing investigation referenced. “Reviewed, no action” is not a reason. Record who decided and the date, because an unattributed decision not to escalate is the finding an inspector writes.

5.3 Escalation record

5.4 Classification consistency check

Trending is only as good as the classification underneath it. If two coordinators code the same defect differently, the trend is diluted across two categories and may never cross a threshold. Perform this check every <<FILL: e.g. quarter>>:

6. Feed into annual product review and management review

The flow that has to be demonstrable end to end is: complaint recorded, aggregated, normalised, threshold crossed, investigation opened, CAPA raised, effectiveness verified against the same rate, and the whole sequence visible in the annual product review and management review inputs. An inspector will pick a threshold crossing from your log and ask you to walk that chain. Where the chain breaks is almost always between “threshold crossed” and “investigation opened”, because nobody was accountable for the step. Name the owner and the timeline in section 5.1 and the break does not happen.

See annual product review and PQR, management review under Q10, and quality metrics and KPIs.

7. Data integrity expectations

Where the analysis is performed in a spreadsheet rather than in the validated system, the spreadsheet is itself a GxP tool and needs the appropriate level of control and verification. See data integrity foundations.

Filled specimen

The following is a completed monthly review in which a genuine rate signal emerges on one lot. Company, product, lots, and numbers are illustrative. The arithmetic is shown in full.

Review header

Baselines in force

Summary by defect type, rolling 12 months to 30 June 2026

Units distributed in the rolling window: 108,600 vials.

Appearance rate calculation, longhand: 4 complaints divided by 108,600 vials distributed = 0.0000368. Multiplied by 100,000 = 3.68 per 100,000. Against a baseline of 1.80, that is 2.05 times baseline, below the T1 trigger of 3 times baseline.

On the product-level view alone, nothing escalates. That is the point of the next table.

Summary by lot, appearance defects, cumulative to 30 June 2026

Lot LB-4417 calculation, longhand:

Both complaints on LB-4417 were closed as unconfirmed, because in each case the complainant had discarded the vial and no sample was returned. They are included in the numerator, per the numerator definition in section 2.2, and their unconfirmed status did not remove them from the trend. Had unconfirmed complaints been excluded from the count, the lot rate would have been 0.00 and this crossing would not have occurred.

Other views

Review conclusions

Escalation record

Action raised against the trending system itself

Approval

What this specimen is meant to teach. The product-level view showed 2.05 times baseline and would not have escalated anything. The lot-level view on the same data showed 12.10 times baseline and crossed. Two of the three complaints were honestly unconfirmed and would have vanished from the numerator under a rule that counted confirmed complaints only. And the crossing was visible five weeks after the second complaint arrived, purely because the review ran monthly. Each of those three points is a design decision made before the data existed, and each of them determined whether the signal was found. The threshold did its job; the cadence did not, and the review said so in its own record rather than waiting for an auditor to say it.

Common inspection findings this log prevents

• Complaint trending is described in the SOP but no trending records exist for the period under inspection.
• Trending is by raw count, so a real rate increase is masked by growth in distribution volume.
• Lot-level complaints are divided by total product distribution rather than by that lot’s distribution, making every lot signal disappear.
• Thresholds are not defined in advance, so escalation is a matter of judgement exercised after seeing the number.
• A threshold was crossed and nothing happened, because no owner and no timeline were attached to the step between crossing and investigation.
• A threshold was raised or a definition changed in the same review that the crossing occurred, so the crossing disappeared from the record.
• Unconfirmed complaints are excluded from the trend, so a defect that cannot be confirmed at unit level is invisible at population level.
• Zero rows are omitted rather than recorded, so it cannot be shown that a defect type was analysed at all.
• The numerator or denominator definition changed between periods with no restatement, so the series is not comparable and the trend is not interpretable.
• Classification is inconsistent between handlers, so one defect is spread across two categories and never crosses a threshold.
• The “other” defect category has grown to a large share of complaints, meaning the category list no longer matches what is being reported.
• Complaint trends are not visibly carried into the annual product review or management review, so the quality system cannot show it acted on its own data.
• A CAPA raised from a trend has an effectiveness check written in different units from the threshold that triggered it, so it cannot demonstrate resolution.
• The trending analysis is performed in an uncontrolled spreadsheet with no verification of the calculation and no record of the extract used.

How to adapt this log

1. Set your document number, owner, cadence, and data sources in the header, and name the validated report you extract from.
2. Fix the numerator and denominator definitions in section 2.2 first, before anything else. Every number in this log depends on them, and changing them later invalidates the historical series.
3. Calculate your own baselines from your own history using section 3.3. Do not carry over the 1.80 per 100,000 in the specimen; it is arithmetic on invented data.
4. Set the threshold multiples in section 3.2 against your own observed variation. If your baseline window shows appearance-defect rates varying between 1.2 and 2.4 per 100,000 across periods, a trigger at 1.5 times baseline will fire constantly and be ignored within two quarters. Set them where you will actually act.
5. Set the minimum denominator in section 2.3 from your typical lot size. For small-volume products, including many cell and gene therapy products, the count-based rules T3 and T4 may be the only workable triggers, and a rate rule may never be meaningful. Say so in the document rather than carrying a rate rule that never applies.
6. Match the review cadence to the severity of what you make. Monthly is a common default; for a product where a defect carries serious consequences, add the event-driven trigger from action 1 in the specimen rather than relying on the calendar.
7. Align the defect categories here with the categories on the complaint intake form and the cluster queries in the investigation report form, so the three documents cannot disagree about what a defect type is.
8. Name the person accountable for the step between “threshold crossed” and “investigation opened”, with a timeline. That step is where trending systems fail.
9. If the analysis runs in a spreadsheet, bring it under the appropriate control: fixed formulas, verified calculation, version control, and a retained copy of each period’s extract.
10. Confirm every regulation and guidance referenced against the current published version before issue.