Data Engineering & Analytics

A warehouse can contain newly loaded records while a dashboard continues to show an older extract, cached result, or delayed semantic model refresh. What to check Compare source load time, transformation completion, dashboard cache, scheduled refresh, and user filters. Check whether the dashboard reads the same dataset being inspected. Recommended...

Removing a filter can change totals in unexpected ways if the dashboard combines different grains, deduplicates records, or recalculates a ratio. What to check Compare row-level records, aggregation level, join paths, filter scope, and numerator and denominator definitions. Test with a small known data sample. Recommended next step Document filter...

A pipeline may repeatedly process old records when its watermark does not advance, source timestamps change, or retries are not recorded consistently. This can inflate totals and increase cost. What to check Compare source update time, ingestion time, watermark state, replay behavior, and job history. Identify whether records are intentionally restated...

An event can appear in two reporting weeks if one report uses event time and another uses processing time, or if timezone conversion crosses the week boundary. What to check Compare timestamp fields, timezone conversions, week-start definitions, late-arriving updates, and deduplication rules. Trace several affected events through the transformation....

Product availability feeds can differ because they use separate inventory snapshots, reserved stock rules, or timing. A downstream dashboard may not reflect the store’s sellable quantity. What to check Compare source timestamps, warehouse and storefront rules, reservations, backorders, location, and synchronization delay. Trace a specific SKU across...

A quality check can pass if it validates only rows that arrived, while an entire source or date range is missing. Completeness requires expected-volume and freshness checks. What to check Compare expected source feeds, row counts by period, arrival timestamps, partition coverage, and upstream job status. Check whether the quality report excludes failed...