This week, I travelled to Berlin to speak at Data Days Berlin. It was organised by a data consultancy called A11, which has recently spun out of a VC fund called Project A, who had originally built this consultancy for portfolio companies. This is the first year that the conference has been open to people outside of Project A portcos, and it was really great to attend and speak. Everything from the venue to the social events was really enjoyable, too!
The conference audience was curated by the organising team to filter to people accomplished in the data industry, which made it a really great bunch of people to meet and to present to. When I asked: “Who knows what a semantic layer in data is?”, more than half the audience put their hands up, which is unusual. I have spoken to audiences at data events where less than half know what SQL is.
The talks I managed to catch were high quality. They seemed to be split between impact focus and how to operate data in organisations to drive value, alongside technical excellence and innovation. This is logical, because data leaders need to be able to excel both as technologists and business people. Strangely, I feel this AI era means we actually need to be better at both at the same time.
Tools like Claude Code, Cursor etc make us more capable of doing our technical work and allow us to be more ambitious. Vendors we use are rapidly releasing new features for us to, in turn, share with our stakeholders.
Our stakeholders are clear about what they want. They want AI access to accurate, consistent data. They think this will be by MCP in ChatGPT Enterprise/MS Copilot/Claude Desktop. They still want us to be responsible and accountable for the provision of this access and the quality of the outputs. Maybe this means dashboards are dead, but BI just entered beast mode.
There are more semantic layers being heavily invested in by huge tech companies, being built by startups or simply being deployed by startups into other AI applications than ever before. I think even by a factor of 10x than at any time in the past.
Why is this? For the reasons above… Stakeholders want AI access to data, they have made this our top priority and told us to get it done1. I’m working with an organisation that, through their existing vendor relationships, have access to four2 semantic layers and are in the process of trying out three of them, with the aim of making a champion available via MCP in ChatGPT Enterprise as soon as possible. The stakeholders want AI access, the vendors and data teams now both know3 they need a semantic layer to make this work - money, attention, product-focus directs to semantic layers.
With this increased attention on semantic layers and the focus on AI, I feel the era of humans having to create and maintain semantic layers is coming to an end. This was always one of the challenges for semantic layer proliferation. Why do I have to learn another tool? Why do I have to complicate my stack? I don’t have time right now to invest in building the semantic layer. Why can’t we just have a good data model?
If AI looks after the semantic layer, then all of these questions and concerns disappear. The semantic layer is something that AI can create from scratch, from existing metadata or with some limited input from a human in natural language. It can then be maintained through human use and human requests for extension with human approval of changes. All without humans looking after more software, or learning another domain specific language. This isn’t science-fiction - I have already seen tools doing this today, and I predict most vendors who provide a semantic layer will offer this by the end of next year.
Well, I hope to be back in Berlin soon to stay in touch with the friends I made at Data Days and to spend time with even more people from the data community here!
I am co-hosting London Analytics Meetup X with DeepMind on 24th September - if you’re in town for Big Data London, it would be great to see you! Check out the details here.
I also asked this question at the conference and it was a similar number of people who put up their hands compared to who knew about a semantic layer… is it correlation or causation? We can’t tell without a better experiment, but I know that the drivers for it to be causation exist.
Only one of the four vendors had a semantic layer at the point the organisation engaged with them to begin with. Yes, you would be right in that it was the BI tool. However, with the addition of a semantic layer and AI querying to the other three tools, aren’t they all BI tools?
was right… everything in the data stack becomes a BI tool, even my counterargument to him about orchestrators.All my preaching has paid off! I am a prophet brought in from the wilderness. Now I can have a Patagonia gilet instead of a burlap sack.