Unmix the data before blaming a shift

Unmix the data before blaming a shift

Stratification keeps meaningful source labels attached to the evidence. Pooling machines, shifts, products or suppliers can conceal patterns and can also create misleading comparisons. Define comparable opportunities before dividing the data into groups. In this example the smaller count belongs to the line with the higher observed nonconforming percentage. That difference still does not prove a line effect: the groups may have different products, inspection methods or random uncertainty. Compare relevant strata, examine the underlying process and retain the pooled view for context. Do not use a convenient grouping as a shortcut for blaming people.

Follow the method

1. Rates with declared denominators
2. Nonconforming units (%)
3. Source group

Unmix counts, rates and source explanations

Fictional case: Brook Assembly reports eight nonconforming units from 100 inspected on line A and two from 20 on line B. Those values remain unchanged in the core exhibit. Supervisor Mara wants to congratulate line B for having fewer failures and assign corrective training to line A. Analyst Jo must reconstruct a fair comparison before the meeting.

Follow the method

1. Line A
2. Line B
3. Pooled
4. Cause

Fewer failures can still mean a higher observed rate

Compare meaningful groups with their denominators and context, and distinguish an observed difference from a causal explanation.

Fictional case: Brook Assembly reports eight nonconforming units from 100 inspected on line A and two from 20 on line B. Those values remain unchanged in the core exhibit. Supervisor Mara wants to congratulate line B for having fewer failures and assign corrective training to line A. Analyst Jo must reconstruct a fair comparison before the meeting.

Role: Quality analyst with both line leaders

Normal condition

The response definition, observation period and inspection method are comparable. Each source retains numerator, denominator and product/operating context.

The gap

Raw counts hide unequal inspection amounts. Product mix, inspection differences and small-sample uncertainty remain unresolved.

• The groups are observational and small.
• The exercise compares observed rates; it does not estimate a causal line effect or prove statistical significance.

1. Retain the source labels

   Jo preserves which line produced each inspected unit and confirms the observation window and classification rule. He does not strip source fields after calculating one pooled total.

   Why: Stratification is possible only when the collection record carries meaningful group identity. Reconstructing a source from memory would introduce another uncertainty.

   Evidence: Each observation retains line, product, time and inspection status where available.

2. Calculate rates with the actual denominators

   He reports A as 8/100=8% and B as 2/20=10%. He shows the counts next to the percentages rather than presenting equally precise-looking bars without sample sizes.

   Why: The line with fewer recorded failures can have the larger observed fraction. Denominator visibility prevents the visual from implying equal evidence amounts.

   Evidence: Two percentages with their corresponding inspected populations.

3. Calculate the weighted pooled result

   He sums failures and inspected units:10/120=8.33%. He rejects the unweighted average of 8% and 10%, which would be 9%.

   Why: Pooling must weight by the underlying population counts. A mean of percentages answers a different question when denominators differ.

   Evidence: The pooled numerator and denominator reconcile to both groups.

4. Investigate comparability before attribution

   Jo requests product family, inspection method, sampling timing and relevant operating conditions. He proposes within-product comparisons if both lines make the same products.

   Why: A source label locates a pattern but does not explain it. Product mix or measurement differences can account for an apparent source difference.

   Evidence: The decision record labels line effect as unverified and names missing stratifiers.

5. Choose a proportionate response

   Mara pauses the blame-based training assignment and commissions a comparable observation window while responding to actual nonconforming product under its own procedure.

   Why: Better comparison does not mean ignoring known product problems. It means keeping product response separate from an unsupported causal claim about a team.

   Evidence: The action names collection improvements and responsible owners rather than declaring a worse shift.

The apparent winner changes within product

A later record shows the two lines made very different product mixes.

Separate the products where enough comparable observations exist, retain pooled and stratified views, and report gaps where one line has no matching product.

A convenient aggregate can hide or reverse relationships. Do not manufacture comparable groups by discarding inconvenient units without an explicit rule.

Each comparison declares its product scope and actual exposure.

Run a practice session

Materials

• Source-labelled unit records
• Calculator
• Blank comparison table

1. Challenge the count-only story · 4 minutes

   Which line looks better before dividing?

2. Reconcile the pooled value · 8 minutes

   Why is 8.33% nearer 8%?

3. Complete the changed comparison · 10 minutes

   What makes a fair product comparison?

4. Debrief attribution · 5 minutes

   Which sentence would unfairly blame a team?

Debrief

• Ask learners to explain unequal weighting with units rather than a formula alone.
• Separate containment of known product from investigation of the source pattern.

Write a careful meeting statement containing both rates, sample sizes and one remaining uncertainty.

Transfer into the work

Owner: Quality data owner with process leaders

Record: Source-stratified comparison and collection plan

Review: At the next comparable observation window

Evidence: Consistent classifications, denominators and product context

Improve source capture or sampling before assigning a cause-specific action.