Yes yes! It's not "either / or" it's about staging. Currently the idea of headless BI is confined to, mostly, Substack posts. If there is a single thing that Drew and I have really internalized about how to create standards over the past six years, it's that you have to start with simple, highly useful constructs and get more sophisticated from there as you train your user community to think about their work using the constructs you've already given them.
I absolutely will never advocate that a semantic layer should not have joins *ever*. But I really think a tremendous amount of the value can happen without them to start out.
I agree with approach #1 to start, but would argue that you don't need a "very high level of maturity" to approach #2. Also, the "cons" of #1 quickly lead to an unmanageable environment, not much different from the old paradigm of using views as single reports.
If you're going to take approach #1 so the broader analyst community is able to engage without needing to know how to do joins, then perhaps build these wide one-big-table's from re-usable entities so that you're not constantly re-defining business logic in every model. For example, this single-table metric model is built from a re-usable dimensional model:
Another advantage of this approach is that, as analysts gain sql skills, they can easily create models on their own because all the joins will be simple inner joins on surrogate keys with the same name. Thus, rather than hindering progress, this approach actually enables progress.
Lastly, taking this approach allows the best of both worlds. If you want to use a tool such as Looker, you can do that too. One thing about the BI tools that I'm sure we can agree on is that complex business model logic does not belong in their proprietary metadata modeling components. I get the need for LookML, but let's start to push that logic to dbt, when possible.
We had Looker at Lyst, the LookML and dashboards built over a number of years had become like a monolithic piece of software. This is bad enough but it being locked into a proprietary platform which consistently increased in price over time is terrible.
I will only choose BI tools which integrate with open-source metrics layers in the future.
Yes yes! It's not "either / or" it's about staging. Currently the idea of headless BI is confined to, mostly, Substack posts. If there is a single thing that Drew and I have really internalized about how to create standards over the past six years, it's that you have to start with simple, highly useful constructs and get more sophisticated from there as you train your user community to think about their work using the constructs you've already given them.
I absolutely will never advocate that a semantic layer should not have joins *ever*. But I really think a tremendous amount of the value can happen without them to start out.
I agree with approach #1 to start, but would argue that you don't need a "very high level of maturity" to approach #2. Also, the "cons" of #1 quickly lead to an unmanageable environment, not much different from the old paradigm of using views as single reports.
If you're going to take approach #1 so the broader analyst community is able to engage without needing to know how to do joins, then perhaps build these wide one-big-table's from re-usable entities so that you're not constantly re-defining business logic in every model. For example, this single-table metric model is built from a re-usable dimensional model:
https://github.com/flexanalytics/dbt-business-intelligence/blob/main/models/metrics/product_monthly.sql
Another advantage of this approach is that, as analysts gain sql skills, they can easily create models on their own because all the joins will be simple inner joins on surrogate keys with the same name. Thus, rather than hindering progress, this approach actually enables progress.
Lastly, taking this approach allows the best of both worlds. If you want to use a tool such as Looker, you can do that too. One thing about the BI tools that I'm sure we can agree on is that complex business model logic does not belong in their proprietary metadata modeling components. I get the need for LookML, but let's start to push that logic to dbt, when possible.
We had Looker at Lyst, the LookML and dashboards built over a number of years had become like a monolithic piece of software. This is bad enough but it being locked into a proprietary platform which consistently increased in price over time is terrible.
I will only choose BI tools which integrate with open-source metrics layers in the future.