Spot on: 'Data Contracts is that they are a way for us to go down the stack and change our foundations to be on rock' & 'companies that have “won with data” have been using the concept of Data Contracts'. I totally aree.
I would propose these additions to the contents of a data contract:
1) Access: The tech details of where to consume the data and pointers to the process of getting access. These technical aspects like the storgage system do matter as they often imply contraints and capabilities of the underlying system and hence of the data.
2) Monitoring: Many (but not all) parts of the data contract can be monitored. That is the big advantage of going down the stack. Executable data contracts imply that monitoring for it can be automated. The data contract should include which aspects are already monitored by the producer. Monitoring may also include a producer vs consumer view. As a producer, you want to monitor what you produce. As you mention correct, the responsibility of the quality of the beer is with the producer, not the consumer. But the consumer may have specific usage requirements that go beyond the default producer checks. It should be possible to also add usage data contracts that expresses the specific usage requirements. This also enables better impact analysis.
Great work, David! Love it.
Spot on: 'Data Contracts is that they are a way for us to go down the stack and change our foundations to be on rock' & 'companies that have “won with data” have been using the concept of Data Contracts'. I totally aree.
I would propose these additions to the contents of a data contract:
1) Access: The tech details of where to consume the data and pointers to the process of getting access. These technical aspects like the storgage system do matter as they often imply contraints and capabilities of the underlying system and hence of the data.
2) Monitoring: Many (but not all) parts of the data contract can be monitored. That is the big advantage of going down the stack. Executable data contracts imply that monitoring for it can be automated. The data contract should include which aspects are already monitored by the producer. Monitoring may also include a producer vs consumer view. As a producer, you want to monitor what you produce. As you mention correct, the responsibility of the quality of the beer is with the producer, not the consumer. But the consumer may have specific usage requirements that go beyond the default producer checks. It should be possible to also add usage data contracts that expresses the specific usage requirements. This also enables better impact analysis.
You mentioned not having seen anything open-sourced. Check out https://docs.soda.io/soda-cl/soda-cl-overview.html It's a Apache licensed and big step towards data contracts imo.
Thanks again for sharing your thorough research!