The greatest BI tool ever

Is not in this post

I recently watched

Benn Stancil

’s Small Data SF talk and one stat he quoted stuck out for me: that BI tools are swapped out every 2 to 3 years on average:

Benn didn’t address why, as it wasn’t really the point of his talk, but my thinking is that it is related to my post from Monday - Risk.

Why the stat above stuck out to me was that it hasn’t been my experience. I’ve ripped Redshift out for Snowflake, I’ve ripped custom Python out for dbt, I’ve ripped Airflow out for Dagster… but I’ve never actually replaced an existing BI tool. I’ve added one to accommodate a different purpose before, but never replaced the old one.

Two of the orgs I worked for chose Tableau and stuck with it for well over 5 years¹. At the first org it was fairly early and there weren’t loads of other choices out there yet. By the time I left, they had hired a whole dedicated Data Viz team who just built Tableau dashboards all day². All of these folks became, or were already, Tableau Certified. Then I joined another company which had Looker, and had had it since the early days - being one of the first UK Looker Customers. They had Looker for 4 years before I joined, then still have it today - so now 9 years going on 10, with no sign of replacement.

I always found it much easier to swap out a data warehouse or transformation tool - yes, code change isn’t easy, but it’s finite and knowable. It’s a GitHub repo or a stored procedure - you can see the scale of what you have to change, you can see where things are fairly standard and where things are custom and will need extra care. You know what code actually runs because it’s scheduled. Everything else can be chucked before a migration.

Migrating a BI tool is a much more horrible affair. Most BI tools don’t have dashboards as code. You have to use the GUI to see what exists. Most BI tools do tell you when an asset has been used, how much and by whom… to an extent - the metadata is often limited. Where the BI tool doesn’t define semantics separately to dashboards and charts, everyone who has made a dashboard can define business entities, dimensions and measures however they like. Every dashboard could have some “business critical” definition.

Even with a tool like Looker, which has semantic code to define entities, dimensions and measures, it’s a very difficult task. If you’ve had it for a while and haven’t been extremely disciplined about pruning your LookML repo (I’ve never seen this happen in the wild), you end up with tens of thousands of lines of LookML code. At the time I was using Looker, I hadn’t heard of Cube yet and didn’t know it was possible to migrate LookML code somewhere else automatically.

Migrating a BI tool is like a slow rehab process. You need to run the new tool concurrently and take strategies like:

• Moving existing content to the new tool whenever it is worked on

• Locking down any development on the old tool

• Archiving old content in the old tool and seeing who shouts, even if they don’t use it

• Only creating new content in the new tool

Then over a year or so (longer at large enterprises), you may be able to shut down the old tool and stop paying for it. BI tool migration is expensive!

So why are most teams swapping them out every 2 to 3 years? I have some theories:

• The greatest BI tool for any executive is the one they had at the last company. It’s not definitely a wrong move to choose this tool based on this kind of whim - they could be right, but often it’s just because of familiarity. They don’t want to have to learn a new tool or even see something that looks aesthetically different. The number of times I’ve heard: “oh, at my last place we had X BI tool, and it was really good because…” and then they go on to describe things which are actually delivered by good data modeling and infrastructure.

• The greatest BI tool for finance people is Power BI. Power BI is like Powerpivot, Powerpivot is like pivot tables, which is Excel! Excel is the greatest tool of all time and of all future times for finance people. It’s also bundled with your Office 365 sub, so finance can argue that you shouldn’t pay for another BI tool when you already have one³.

• The greatest BI tool for an Azure shop is Power BI, for the same reasons as above.⁴

• The greatest BI tool for a risk-averse incoming data lead is probably the one they’ve invested time to be certified on. They want to eliminate failure from tool choice (and this is the shopfront of tools for them). They can blame an existing unwanted BI tool for any upstream data problems while they ramp up, and when time comes for renewal they can switch to their choice and blame benefits unrelated to the new tool on their great choice and expertise on tooling. The BI tool is the only data some will ever see.

• The greatest BI tool for an adventurous incoming data lead is the next one they want to try. When I joined the company that had Looker, it was actually the next tool I wanted to try and that’s a big part of why I didn’t rip it out. I had started to consider ripping it out by the time I was leaving, due to the negative effects of the GCP acquisition.

• The greatest BI tool for a company that has been acquired is the one their acquirer has, as then the new dominant data lead from the acquirer can get a better deal for their choice of BI tool and score an easy win. The acquired data lead, who may be trying to survive while they look for a new job, just has to suck it up and tell their disgruntled team that this is the new BI tool because they got acquired.

• The greatest BI tool where you have one that is too difficult to rip out and you want something for ad-hoc analysis is a notebook or canvas style one. At the company where I had Looker, we realised that LookML had been abused for ad-hoc analysis for many years, causing bloat. We started off experimenting with Hex for this reason and then they also used Count after I left.

• The greatest BI tool for “self-serve” uses a semantic layer. It doesn’t try to solve for all of analytics, but for simple questions of your silver/gold medallion datasets. Data users need a menu to order from, it’s not reasonable to give them raw ingredients to cook for themselves. If you want to know about other things not served by this semantic layer, this is work that requires an analyst to explore unknown territory.

• The greatest BI tool for embedded analytics or a data product, is not a BI tool. Take the time to build something that is right for your customer, not just easy for you to deploy. Testing the waters with i-framing can make sense though.

• The greatest BI tool of the future will be AGI, where analyst and stakeholder are also replaced by AGI and the AGIs exchange information over some interface and make decisions in milliseconds. The only humans left are the investors. 🙊

The real greatest BI tool helps someone using data to orient it, so that it reflects the control levers they have to enact change. Then, showing them what happens if they pull the lever up, down, left, right or rip off the control panel altogether. Explaining how confident you are, how it compares to anecdotal information, whether they should use the data at all for the decision and what the data person thinks based on the evidence in front of them. The decision is still theirs, but you’ve been their data business partner. You’ve helped them understand their risks, rewards and possible optimal balance. It’s not a tool at all, the tool was just a prop - the greatest BI is you.

1. The first are still probably at least partially on it after nearly 10 years, and the second went under.

2. You could literally walk behind their desks and see Tableau Desktop or random stuff they were shopping for all day. Just the thought of it made me despair 😱.

3. This is hands down the biggest reason Power BI is the biggest BI tool in the world right now - bundling. I’m surprised Tableau didn’t sue like Slack did for Teams having the same effect. Someone will probably reply to this post saying they actually did. Although, Power BI are increasing prices for the first time at the end of this year to take advantage of cornering the market and to reflect “enhanced value from AI” - this may shake the tree and big enterprise commits to Power BI may evaluate other tools in the next two to three years.

4. Which is why I would never work for a company that chooses Azure as its cloud provider again. I turned down a role a few years ago for this reason - I just know I’ll be depressed and blocked in short order. I’ve muted the #MicrosoftFabric tag on BlueSky… I’m not offended, I’m just not interested.