Extraction strategies: Is a static extract (full snapshot) the method typically used for ongoing, routine data warehouse maintenance, or is it mainly for initial loads/occasional snapshots?

Difficulty: Easy

Correct Answer: Incorrect

Explanation:


Introduction / Context:
Data warehouses are maintained via periodic loads. Two broad extraction styles are common: static (full snapshot) and incremental (delta/change data capture). This question checks whether static extracts are the norm for ongoing maintenance.



Given Data / Assumptions:

  • Ongoing loads must be efficient and minimize source/ETL impact.
  • Large tables make repeated full snapshots expensive.
  • Change data capture (CDC) or timestamp-based deltas are widely used for maintenance.


Concept / Approach:
Static extracts export all rows regardless of change, making them suitable for initial population or rare rebuilds. For routine cycles, incremental extracts move only new or changed rows using last_update timestamps, log-based CDC, or merge patterns. This reduces load times, storage, and contention on source systems.



Step-by-Step Solution:

Classify load: initial vs. maintenance.For maintenance, design delta logic (timestamps, sequence numbers, or logs).Implement upserts/soft-deletes in the warehouse according to change type.Reserve static extracts for first loads or special reconciliations.


Verification / Alternative check:
Measure daily ETL runtime. If full snapshots are used for large fact tables, run times spike and windows lengthen. Switching to deltas typically shortens the window and reduces resource use.



Why Other Options Are Wrong:

  • Calling the statement “Correct” conflates initial load with steady-state maintenance.
  • Bandwidth alone does not determine the method; source impact and SLAs matter.
  • CDC reference is misapplied; CDC is itself an incremental technique.


Common Pitfalls:
Trusting soft deletes without audit trails, or forgetting late-arriving facts and out-of-order updates. Always design idempotent, restartable delta loads with reconciliation checks.



Final Answer:
Incorrect

Discussion & Comments

No comments yet. Be the first to comment!
Join Discussion