02 / Data preparation
Missing is not zero
An absent observation and an observed zero carry different information. Treating them as interchangeable can quietly change your conclusion.
A three-day example
Suppose an illustrative sensor log contains the following values. The middle day was not recorded:
| Day | Observed value | Status |
|---|---|---|
| Monday | 10 | Observed |
| Tuesday | — | Missing |
| Wednesday | 20 | Observed |
The average of the two observed values is 15. Replacing the missing observation with zero gives a three-day average of 10. Neither calculation recovers Tuesday's unknown value.
Ask why the value is absent
A blank cell might mean a failed collection, an unanswered question, a field that does not apply, or a deliberately withheld value. Those cases should not automatically receive the same treatment.
- Retain the original missing marker and document its meaning.
- Report the number of observed values with the statistic.
- Check whether missingness differs across dates or groups.
- If you fill values, keep an explicit flag identifying the filled records.
Make the denominator visible
“Average: 15 across 2 observed days; 1 day missing” is more informative than an unexplained 15. For percentages, explain whether the denominator contains all eligible records or only completed observations.
In the sample dataset, missing_labels is a count of absent labels inside that day's extract. It is not a missing value for the entire day. The name and definition of the field matter as much as its numeric type.