2 Comments

While I'm a huge fan of CDC from internal production databases to copy to the warehouse, I have a lot of skepticism around CDC from 3rd parties. In particular, a few questions I don't know the answer to:

Are we sure the vendor source data is actually structured how it's written to the warehouse, or are there transformations that happen in batch? If the latter, CDC itself won't be an end-to-end solution.

If providing CDC as a solution, how will vendors deal with heavier demand and the stress of supporting real-time systems? First, vendors will have to keep an eye on source databases and everything that comes with scaling CDC. This also has multi- vs single-tenancy implications. Second, anytime you say "realtime" there's an SLA around it. With CDC, if the SLA doesn't decrease, then there's no point on doing CDC as opposed to batch. If it does decrease, this will put a lot of stress on vendors to actually deliver which has potential financial ramifications.

Expand full comment