Skip to main content
For Enterprise Data License (EDL) customers, Shovels employs an overwrite approach when pushing data to your data warehouse.

​
How It Works

Each delivery is treated as a fresh table, completely replacing the previous delivery’s data with the updated dataset.

​
Benefits

This approach simplifies data management by:
  • Eliminating the need to merge incremental updates
  • Providing a complete, consistent dataset each delivery
  • Avoiding complex change-tracking logic

​
ID Stability

Critically important: Our IDs remain static and don’t change from release to release:
  • Permit IDs
  • Address IDs
  • Contractor IDs
This consistency ensures that any scripts, analyses, or processes relying on these IDs won’t be impacted by a refresh.
The stability of our ID system allows customers to build reliable data pipelines and maintain referential integrity.

​
What’s Included

Each delivery includes:
  • All historical data
  • New permits added since last delivery
  • Updates to existing records (status changes, etc.)
This provides a complete and current view of the construction permitting landscape.

​
Tracking Changes

For customers who need to track changes between deliveries, we recommend:
  • Using the first_seen_date field to identify new records
  • Comparing status change indicators between deliveries
  • Maintaining a copy of previous deliveries for comparison

​
Delivery Schedule

Standard delivery is twice a month, on the same schedule as our main database. Custom delivery cadences are available for enterprise contracts with specific SLA requirements — contact sales@shovels.ai to discuss one.