I’m heading to New York City early next week and will be there for the whole week. Ping me if you’d like to meet up!
I’m going to the new Data Universe conference, organised by some of the folks behind Big Data London. I feel like this conference could be the first of an important new annual series. As you know if you read this substack regularly, I’ve attended Snowflake Summit and dbt Coalesce for the last two years and had a good time of it. However, both of these conferences are far from vendor-neutral, being one vendor’s main event for the year.
London has the Big Data London (BDL) conference as a vendor-neutral conference, and it’s felt weird to me for some time that New York didn’t have one. New York and London are probably the biggest tech hubs, after SFBA. The large tech companies in SFBA are proportionally more focused on infrastructure over application - London and New York probably have a bigger share of data practitioners. By this, I mean data folks working in industries outside of tech infrastructure such as airlines, payments, financial services, travel, e-commerce, insurance… The big hubs for these other industries are not SFBA, more often than not they are cities like New York.
By this logic, New York should have the most dense population of data practitioners in the world. Yet they are being asked to travel to California, Las Vegas or outside of the US for a data conference. They have been underserved! As someone who has worked for data vendors for the last couple of years or so, it makes sense for me to travel, even to another continent, for a conference. It is part of the job for me and for other folks who work for vendors. The chance to talk about our product, meet investors, meet customers, meet collaborators, meet partners, demo our product in partner booths… only happens at conferences.
For data folks who are working in industry, data conferences can be valuable for different reasons - but there is a trade-off between doing their actual job and going to the conference, along with its associated costs. This trade-off is often too high for their employer, especially if the conference is on the other side of a continent or in a different one altogether. I expect that many NYC data folks will be able to attend Data Universe, who would never otherwise go to a data conference.
Normally, the big cost of data conferences for me is time away from my family, but this time I’m bringing them with me! I’m actually looking forward to having my kids on the plane with me, keeping me company. I’ll let you know if the theory doesn’t match the practice if I see you at the conference or elsewhere in the city!
Flux continuous
I’ve spoken to many data teams recently, as part of my GTM efforts at Delphi, and almost all of them are going through some reorganisation or another. Whether it is specific to their team, being moved to another part of the org, having lay-offs or being part of an org-wide change.
Part of me thinks that this is just a natural part of life and change is to be expected. At the same time, I feel like in previous years this wasn’t the case - there were long periods where businesses made plans and tried to execute, with only a minority needing restructuring at a time. The level of change I see now really feels much higher and it’s felt like this has been happening since 2022.
I know data team professionals are supposed to be able to have the resilience to deliver in difficult circumstances where their org may be shifting around, but I wonder if the reality isn’t so clear. After all, we’re humans and not machines. We’re concerned with our basic hierarchy of needs - if they aren’t secure, then our ability to achieve higher order goals is diminished.
If I were a data team member who was at a company experiencing mass layoffs, in all honesty, I would start proactively applying for roles. It’s much easier to find a role when you are in one and you aren’t currently being laid off. It’s hard to have the swagger to impress in interviews when you’re feeling more desperate, because you really need the job. Unless you were really happy in your role and company, and felt your job was unassailable (which would be false), it would make sense to have a look around. If you could find a well-funded or high performing company willing to hire you for the same or more… it would simply make sense to move, rather than play the Hunger Games at your current company.
If I look back at the times I’ve been most productive in my career as a practitioner, circumstances weren’t perfect at the company (they never are), but at the same time our problems were in front of us, not within. We weren’t deciding whether we should have the level of staff we did, we were already relatively lean. We weren’t deciding whether we should move people around - where teams were positioned had been decided long ago and wasn’t up for debate. I was able to focus on the clear and present problems and try to solve them or help others solve them by applying my skillset, or leading my teams to do so.
I think data teams, and all teams really, need this stable platform to deliver from. Otherwise, the best they’ll do is keep the lights on and maintain what already exists, while they surreptitiously click “Easy Apply” on LinkedIn. The last time I was at a company going through significant reorganisation, the company had started to lay people off and reduce spend, and I had started to apply for new roles just in case. One of the interview processes had progressed quite fast. The week before my final interview day, my company asked me to lay off half my team. I asked if I could be laid off instead of one or two of my team members, much to the shock of my manager and company. I had a week to change my mind, during which I secured my next role…
My point in all of this is - company leadership needs to aim for some kind of stability. They may well need to go through a round of lay-offs, but try to make it just one, and do it in a humane way - your remaining staff are watching and thinking: “it could have been me, and it may well yet be”. They may need to do a logical reorg to make their company better adapted for the future, but do it once and and do it well.
Company leadership can’t keep their company in perpetual flux - there is no stability or platform for high performance or growth like this. Although I also hear of company leaders who seem to relish reorg after reorg… Humans who are worried about where they may end up (perhaps with that CXO member who doesn’t value them), or being laid off, don’t usually get a hell of a lot done - they don’t go above and beyond. They sharpen their CVs with GPT-4, they get good at writing cover letters, they write about what they’ve built in the “modern data stack”, rather than build anything new with it. They do tidying up work that saved “X” in Snowflake credits, that reduced dbt models from Z to Y, that made dashboards load 10 seconds faster on average. They do work with marketable, tangible short-term benefits. They do work well within their capability, leaving them time for LinkedIn and to get interview fit. They don’t do high risk work that may lead to nothing to put on a CV - but which is probably the highest value work to the company.