Discussion about this post

User's avatar
Elias DeFaria's avatar

Hi David - appreciate you diving in on what we built at SDF.

Two quick clarifications:

1. Our Write Audit Publish implementation did not need to understand breaking downstream changes, since these were all caught at compile-time. Local compiles were run before sdf run and sdf build, thereby catching all downstream breaking changes in a matter of seconds as opposed to waiting for queries or audits to run against the remote database.

2. We actually did store state. We leveraged the local file system for this (particularly a `sdftarget` directory), and would incrementally rebuild the DAG based on changes to both models (code) and source data.

Expand full comment
2 more comments...

No posts