The Lindy effect (also known as Lindy's Law[1]) is a theorized phenomenon by which the future life expectancy of some non-perishable things, like a technology or an idea, is proportional to their current age. Thus, the Lindy effect proposes the longer a period something has survived to exist or be used in the present, the longer its remaining life expectancy.1
SQL has not been overthrown as the primary way of accessing data, even though there have been attempts - which have had huge popularity (pandas, nosql). Nonetheless, SQL keeps going from strength to strength, with no sign of abating - but perhaps signs of increasing abstraction.
If you are to apply the Lindy effect here, you would anticipate that SQL will still be with us well past 2060. I think one of the assumptions in applying the law is that new entrants into the space will have the uphill climb to gain traction (basecamp), another to gain popularity (plateau), and then a further climb to become ubiquitous (summit).
Many ideas and technologies fall off the mountain and die before reaching the summit. Some get to a plateau and stay there, or climb to a slightly higher plateau over time. There are many reasons for this, starting from early to late:
It never really gains any following or popularity - doesn’t make it to basecamp.
It gains a niche following and walks round the mountain, perhaps to some success.
It makes it to basecamp, but then things like funding, growth rate and competition stifle it or push it off the mountain.
It makes it to a plateau, but then things like preparing for exit, community development struggles, OSS leadership, acquisition, life as a public entity… prevent it from trying to reach for the summit.
It’s survival of the fittest and you can probably count those who’ve arrived at the summit on two hands. However, as I just described, the assumption here is that the new technology or idea has to gain popularity - it doesn’t already have it.
I wrote about self-serve and NLP last week and my own dog in the race:
Part of the success of existing self-serve BI tools like Lightdash, Looker, Thoughtspot, Metabase… are that they are like using a pivot table, whether through clicking around or writing a request with specific structure and terms. Many more people are able to use a pivot table than can write SQL, probably to at least one order of magnitude more. There are probably 100 to 1000 times as many people who speak English as can write SQL.
The vast majority of people who know SQL also speak English - if English became the language to interface with data, they won’t have to learn anything new. Some infrequent SQL users would revert to the ease of using English to access data: these users often feel forced to use SQL to get what they need directly and quickly.
Abstraction(Abstraction(Abstraction(…
If English could become a reliable interface for data, then it easily has the potential to overtake both semantic layers and SQL. This is not to say that the latter two will cease to exist - English will be an abstraction over a semantic layer (which itself is an abstraction over SQL) or an abstraction over SQL (which itself is an abstraction - nothing compiles SQL directly to machine code).
Language is an abstraction on meaning2. Meaning is universal to all human beings and perhaps animals, too. Language is to meaning what a higher level program language is to assembly. We even have a wonderful compiler built into our brain for interpreting meaning from language. Meaning is a primitive and linked to our senses and feelings - I sometimes think of the middle layers of a neural net as levels of meaning that aren’t able to be expressed in language. However, these primitives are much closer to the final inputs or, indeed, are the final inputs to the decision layer.
I can’t express raw meaning in language to you, but if you think about what the words ‘teddy’, ‘plane’, ‘clock tower’ and ‘steam train’ mean to you, I’m sure you would come up with images based on your experiences - perhaps even feelings and sound, too (for me it feels like playing around with Encarta, if you remember this, and being able to click on the sound and video snippets!).
Researchers have managed to reconstruct images shown to human subjects from fMRI signals of the brain. It’s possible to think of these signals as a physical manifestation of meaning in the brain. Now these entities are simple and clear, physical objects - we’re probably some way off interpreting a more abstract concept like a measure, dimension or filter from these signals. However, we’re probably a lot closer than we thought we were a month ago.
A meaning-to-data interface is clearly the final interface, but we can still confine this to science fiction for now. An English-to-data interface is right around the corner.
https://en.wikipedia.org/wiki/Lindy_effect
Yes, I have recently read Neal Stephenson’s Snow Crash for the first time, but, nonetheless, I feel this is clearly true.