Life finds a way...

... and "rebundling" in the data stack can look different to how we imagined

A personal announcement

As you may have read, I started this blog when I knew I was moving role from Senior Director of Data at Lyst to VP of Data & Analytics at Ruby Labs. If any of you are connected with me on LinkedIn, you will have seen that I have moved role to be Chief Product & Strategy Officer and Cofounder at Avora.com and this may have been a bit unexpected!

Ruby Labs have chosen to go through a product pivot and in the meantime need the team focused on building MVP; I will continue to help them occasionally as an advisor whilst in my new full-time role. I jumped in at the deep end upon joining Ruby Labs, producing data in Hex notebooks for data rooms, refactoring our data model from scheduled scripts on BigQuery to dbt along the way. It was a short but fun ride, and it made me realise that I want to operate in a strategic way for a company as I was indeed allowed to do there. Good luck Ruby Labs!

Purpleglots and Unicorns

As you can tell from the post above, being a cofounder is something I’ve thought about a lot, and at the time of writing it, I thought the opportunity would be a few years away after an exit for Ruby Labs, providing the capital I needed to start a company. Sometimes when your plan is ripped up, it can allow you to take a chance you didn’t think possible before…

I have worked with Avora for a number of years now as a customer and a product advisor - some of the key new features in their roadmap over the last couple of years have been based on my advice. I had talked with Ricky Thomas, founder of Avora (and now my cofounder), about joining in some capacity or another for some time, but the timings have never worked out. Either I was happy at Lyst or Avora weren’t in the right place to take someone like me on. Even before I joined Ruby Labs, I discussed the move with Ricky and we agreed it looked like a good move with plenty of other options in the market should things not work out; at that point in time it wasn’t quite the right time for me to join Avora yet.

However, three months have passed, and Avora are coming into a good place with what looks like product market fit, customers organically signing up and working on a dbt metrics layer integration I had assisted with as an advisor. There probably won’t ever be a more perfect time for me to join and lead the company headfirst into the Modern Data Stack!

Having a cofounder in Ricky is of tremendous value as we have years of trust, openness and candour built into our relationship already; it’s pre-prepared for cofounding together. As someone who is new at leading a company, having someone I can trust who has many years of experience running startups is invaluable, as I have unknown unknowns of an unknown quantity! This is an opportunity I couldn’t turn down, I may never get another chance to lead a Modern Data Stack company, let alone in such a big year for data, where I really know what I want to build already.

Avora is a business observability platform - it automatically tracks key business metrics for anomalies, taking into account seasonality, growth trends and statistical significance. It also explains these anomalies detected through root cause analysis - eg Why did our orders decline compared to yesterday? Your paid marketing channel had fewer than expected orders. Having these insights with reasons, plus alerting in Slack, makes them more actionable.

I remember being a young analyst and continually being asked to explain why a metric had changed - you simply don’t have the time to explore every permutation of dimensions, take into account growth and seasonality and test for statistical significance. You end up on a hamster wheel, churning out reactive work; there was a time when this was the norm, but analysts today expect a higher level of impact than we did back then. Getting them to repeatedly do this kind of work is good way to get great analysts to work somewhere else.

I have designed a root cause analysis system before, during my time at Worldpay, but it was specific to payments processing and not generic at all. Having a system that can generically compare two datasets and explain the differences is incredibly useful.

We have seen a big trend towards the proliferation and uptake of data SaaS as we exit the pandemic - businesses have taken up cloud faster than expected and are fighting to hire the best people. Therefore, SaaS tools are appealing to them as a way to get more output from the people they have, by automating tasks best done by a machine.

I really believe we’re at a place on the line just leading into the Product/Market Fit point, and with our upcoming integration with the dbt metrics layer, server and marketplace (no one has announced this last piece but I believe it’s inevitable and entirely logical), we plan to scale, which is why we’re raising a seed round at the moment. We have a lean but great team, and believe with the right colleagues joining us we can make good progress in growing our business and product.

Could “rebundling” look different

A few weeks ago, before my change above was even conceived, I posted the blog post below about dbt being the data interoperability layer. I think their Series D announcement has this position in the market - and its value - baked in.

Highly Opinionated Integrations

Now that I’m unexpectedly leading an MDS SaaS vendor, I realise I have unintentionally constrained my future self in writing this! However, I still really believe in the principles I laid out in this, and it has actually prepared a big part of my roadmap in advance.

I think the “rebundling” of the data stack has been mentioned a lot recently, with industry thought leaders saying that having many different products in a data stack is not sustainable in the long term, therefore, suggesting that some big fish will swallow some smaller fish in the pond. This is definitely going to be true to an extent (as we saw with Streamlit and Snowflake just recently), but does it have to be? What if with a common interoperability layer, app marketplace, and data “roads” built to connect the network we have a different type of bundling.

With dbt providing an interoperability layer and probably app marketplace on top of this in the future, customers can have a great choice of trusted data SaaS that plugs straight into this securely, without needing to necessarily share credentials or set much up. Customers of Salesforce don’t complain too much about the added complexity of the thousands of apps in the app exchange, as they can securely and conveniently use them.

Apache Arrow is fast becoming the bitumen the data “roads” will be built with, allowing data to flow around different apps like Hex - but what if it enabled data to flow directly between SaaS products themselves? I’m not talking about the volumes of data handled by data warehouses in transformation and data engineering; this is the size of data needed for analysis and graphing, the output of a query that aggregates or a metrics layer. Could this level of integration and partnership afforded by common standards and a shared interoperability layer be the “rebundling” we need, thus allowing for the complexity in the current data stack to be transformed into choice? This is the “unbundled rebundle”: I see it as being like bonds in a large molecular structure.

From a technical point of view, it’s very similar to how it would work with acquisition: the acquiring company needs to integrate the acquired tech into their stack through APIs and standardisation across both existing and acquired tech… sound familiar? From a commercial point of view, it’s also very similar: the acquisition happens because the synergies mean that the value of the company, post-acquisition, is greater than the sum of the parts. The value of integrated, partnered companies is also higher than if they weren’t so. I’m not against bigger data companies buying or merging with others, but I don’t think it has to be the only form of “rebundling”.