Guardians of the Data
Learnings from building data teams
Hiring
Let’s start with the most important part of building any team: hiring. I can’t stress how important this part is: if you get this right everything else follows more easily, if it’s done wrong then everything else is like uphill skiing. I usually expect to spend 25% of my time on hiring when I have open roles, and if you have a few open at the same time that can easily be 50%.
Hiring is a funnel and you need to take ownership of the whole thing - your recruitment team can only do so much for you. I can usually screen a CV in a minute to see if I’m interested in a candidate. This is actually much slower for your recruitment team, who aren’t data specialists. They can end up screening out good candidates and letting ones through to interview that you would have screened out at CV stage. I know it’s commonplace to allow your recruitment team to provide you with screened candidates to interview, but you lose a huge part of the funnel doing this; I find the few minutes it takes to screen CVs (inappropriate CVs can be screened out in seconds) are so worth it. You’re more excited with the candidates that make it through recruiter screen and save your recruiter’s time, which is a scarce resource (you can save them from screening an inappropriate candidate for 30 minutes with a 30 second read of a CV).
You will also find that in doing screening, you will guide your recruiters to screen for specific things related to your team and roles and therefore filter out inappropriate candidates earlier in the process too. Your relationship with your internal recruitment team should be like a partnership - not where you are a client - in order to get the most value out of it. They aren’t meant to be at arm’s length like an agency, that’s the whole point of them being internal - after time invested in this relationship they learn to look for the things that are important to you in candidates, adding further efficiency to the process.
Culture Capital
When you’re building a team, if it’s to be successful during your time there and beyond, you need to build a culture: a team that feels safe with each other, supports each other and works together to achieve common aims and ways of working. The things I have always looked for are the personal traits of integrity, openness, honesty and kindness (4PT). If a candidate has these personal traits, along with some drive and technical ability, their contribution to the culture of your team will almost certainly be positive. In combination these traits are a superpower! Understanding whether someone will contribute positively to your culture before joining is almost entirely about learning about their personal traits and ways of working.
I would trade 4PT over higher technical ability in pretty much every situation (exceptions being when you definitely need high technical ability as a founding member of a team… then I would just not compromise on either and this is why you find these founding members are often prior contacts as they are high risk hires), you can teach and train higher technical ability into a member of staff but it’s nearly impossible to get them to have 4PT once they’re well into adulthood. 4PT also allows them to learn and grow much faster than staff who don’t have it. It allows them to take and act on constructive feedback faster and without friction. It allows them the ability to admit mistakes and learn from them, and put their hand up when they don’t understand and need support.
Technical ability is important, but this is where having a good technical test is much more valuable than having your primary method of understanding technical ability be a verbal interview. Lots of technically brilliant people don’t fare well when asked technical questions in an interview scenario - this situation favours extroverted people over introverted.
It’s not straightforward to find 4PT in a candidate when you interview them. It requires you to dig deep into why they did things in situations they discuss and how they felt about certain things. Some candidates will feel uncomfortable showing you this much about the inner workings of their mind, but this is telling in itself as a way of assessing openness. There is no interview bank of questions to find this information in an interview, it requires practice and the ability of an interviewer to adapt to a candidate. It’s better to find this information in the stories a candidate goes through of their own accord, rather than to try to pull it out by going through a contrived scenario.
I’ve met senior people in orgs who don’t really understand the concepts of unselfishness and truth, they’re so far gone they only understand what they want and what’s convenient; in larger orgs these staff are just accepted as a natural state of things and even respected as effective political operators. However, they destroy good culture and encourage your best staff to leave.
I agree that a team at work is more like a sports team than a family, but what do you notice about sports teams that have big egos, selfish players, lack of respect, lack of trust, lack of care for each other..? Even with modest levels of any of these, they fail. Being unselfish and being able to self-sacrifice for others to win is a key part of serving your team and your org - there is real opportunity cost in not having this in your team.
This may seem very warm and fuzzy, but I believe it's the bedrock of an org being able to aggressively win in the market. Having a great culture is worth pounds and pence, dollars and cents; it can be the thing that determines survival and success. We’re still in an era where people build products and companies - having those people work even marginally better together provides a big operating multiplier. Great people find it hard to leave great culture, leading to a sustained level of talent density.
There have been a couple of occasions where I’ve had to fail probations of staff, both times where they had been hired prior to me joining - the things that let them down was a lack of one or more of these core personal traits which made them a destructive member of the team and less effective and able to reach the level I needed for them to pass their probation. When this happens to you as a line manager, it takes up a huge amount of time, it wastes money in paying staff to join and then produce very little and there’s also the potential loss of better candidates through closing the role. It can cause you to lose sleep and experience a great deal of stress too, which is why I would advise avoiding this at all costs by hiring well! The contrary to this is true too: when you hire well, you often know within weeks that the candidate has passed their probation and you start focusing on their future in your team and org. It’s an absolute joy as a hiring manager, it relieves your burden and enhances the culture and fun of being in your team.
Speed & Offer
One of the benefits of owning the whole recruitment funnel is speed - in my last three roles as an employee, before becoming a cofounder at Avora, the thing that made me feel wanted as a candidate was the responsiveness and speed of the recruitment team engaging with me:
Ruby Labs - 11 days from application to offer, 7 days from initial contact with recruiter to offer. Offered more than I asked for.
Lyst - 17 days from application to offer with a technical test completed in the process. Offered more than I asked for.
Elevate - roughly 3 weeks (I can’t find all the emails!) from application to offer.
In the current market only FAANG companies can afford to make good candidates wait and jump through the hoops of a lengthy recruitment process.
Offering a candidate more than they’ve asked for, even if it’s a modest amount extra, goes a long way to getting them on board. It shows the generosity of your org and willingness to pay someone what they’re worth. Another way to describe paying a competitive wage is to pay as little as possible to be in line with the market. I really think we need to move towards having (at the least) pay ranges on job descriptions, as so much time is wasted with different expectations. I remember speaking to different companies about VP of Data roles with as much as a $100k disparity in pay! It’s not hard to describe what combination of skills and experience you would pay more for, of which some aren’t necessary to do the role but are beneficial, and also where you would be willing to compromise but therefore would pay less. This is a 4 by 5 grid in Excel at the most.
Recycle good candidates - if you’re lucky enough to find multiple great candidates, take as many of them on as you possibly can. Having a really good headcount plan for the next few quarters allows you to do this - usually your org will allow you to hire early if it saves another costly recruitment cycle. Understand that a candidate may be interested in a similar but adjacent role (especially if it isn’t explicitly open): an analytics engineer applicant may also be interested in a role as a data engineer or as an analyst.
I respect candidates who are a bit creative in reaching out - I feel it shows proactivity, but I know many other hiring managers who want all candidates to follow the same process and I respect that too. This candidate saw a role I was advertising and managed to find my contact details to email - we ended up hiring them at Lyst:
Dear David,
Hope this email finds you well. My name is XXX and I am writing to express my interest in working for Lyst. I found your contact from LinkedIn, hope you don't find it rude.
I am particularly applying to Lyst because it is a company with the culture and value that I really admire. What I'm looking for now is a role where I get the chance to be creative and innovative and because I am passionate about my work, this is going to be somewhere I will be pushed and challenged. I would also like to work in a company where I can work at a fast pace. I’ve been in jobs before where it takes a long time to get things done, and I believe Lyst will be different. Finally, being a commercially-driven person, I want to work as part of a positive team of people who are all working towards the same goal. Lyst is exactly the place that I'm looking for.
I am able to demonstrate my skills and competencies through my previous work experiences.
As a data science consultant at XXX, I've played various technical roles in different projects including data migration, machine learning solution, web development, and ad-hoc data analysis. I specialise in using data to bring impact to the business for our clients.
I also used to work in an e-commerce startup company, where I analysed customer journey and provided actionable insights to cross-departments. I also carried out A/B testings with statistical methodologies to optimise user experience, which led to an increase in conversion rate, CTR, and most importantly revenue. I successfully increased our revenue by 50% in 3 months.
I am proficient in SQL and Python, with experience in a number of BI tools like Tableau and Power BI. I also have knowledge in machine learning, statistics, cloud computing, and dashboard development. With my relevant experience and the technical skills I have developed from my current role, I believe I could be a great addition to you and help Lyst to grow to another level.
I have enclosed my CV for your perusal. Please don't hesitate to get in touch should you need anything else. I look forward to hearing from you.
Yours sincerely,
XXX
Alternatives to Hiring
If you notice I haven’t spoken much about data yet, most of building a great data team has little to do with data… that’s more to do with the engineering and execution itself and having a great team in place takes care of this. You could argue that what I’ve talked about in hiring, above, is a huge exercise in metadata gathering, curation and modelling.
Finding these candidates in this market is very difficult and you should consider whether you definitely need to hire. Do consider whether you could buy a tool or automate a workflow instead of throwing humans at the problem. I know there are some orgs that will never want to buy a tool and that’s fine for the top 20 tech companies in the world, but for most companies this isn’t pragmatic. If a tool can have a good Value of Time Saved to TCO ratio, it’s worth considering. You might say this is biased as I’m a cofounder at a MDS company focused on automating analytics!
However, I’ve only recently become a cofounder and have been a practitioner for a long time, having bought augmented analytics and ML products at multiple orgs including Avora, DataRobot and Continual.ai
Tools are getting cheaper because of product-led companies, and they are better integrated with the rest of the stack because of interoperability layers. At the same time, staff are getting more expensive and hard to find. This is probably a factor towards driving unbundling - some of the people you could have hired to lead or work in an industry standard data team are founding these startups.
Post-Hiring
Now that you’ve hired as well as you can, guiding your team to success and maintaining a great culture is a mixture of strategy, coaching and mentoring. There are some basic operational elements to consider that will help your team succeed.
Fail fast and safe
Things will go wrong, it’s just a matter of when and how badly. Failure must be embraced with debriefs and incident reports to codify the learnings and actions taken, with no stigma attached. The natural decisions that come after a few failures are things like:
Version control
CI/CD
Dev/Test/Prod environments
Unit Testing
Integration Testing
Implementing monitoring such as with a data observability tool
Stakeholder communications protocols
Limitation of the scope of “golden” data
Data domain owners
Processes to deal with product changes that lead to data change proactively
Even with all of these things in place, things will go wrong and it has to be OK, as long as you learn and adapt afterwards. Often, data teams have to deal with changes out of their control which break the data downstream - they can end up being the lightning rod for non-technical stakeholders who don’t understand why something has gone wrong. Clear ownership of each stage of the data creation chain has to be understood and documented. This is one of the benefits of a data mesh operating model.
Product mindset
I won’t say too much on this topic as I feel this article covers the topic well:
Your data is a collection of products, some internal and possibly some external to your organisation
Adopting a product mindset in how you think about customers, service and SLAs can make you much more sympathetic to your stakeholders needs
You aren’t here to provide data, you’re here to solve problems
A product can be a clean and well defined dataset, a graph, a dashboard, a piece of research, an ML model deployed… data teams can provide a wide array of products!
Allow space for your team to innovate on their own
Automate reactive work as much as possible as it stands in the way of building your products out and therefore allowing for its own reduction
Having a dedicated data product manager, once you reach a certain size of team and level of specialisation, is helpful as product management is its own discipline that data people aren’t necessarily good at. It is helpful to have a data product manager who has a data background.
Context Hierarchy
Data teams don’t do well being asked to pull data without any context - it’s the least fulfilling kind of work. With ML engineering this doesn’t seem to be as much of a issue, as usually there is a well-defined problem to work on ahead of hiring which is then executed and iterated on. The structure of the workflow is much more similar to engineering although, unlike with engineering, there is the possibility that the endeavour is not possible at all, and this can’t be determined until it is attempted.
The best situation for data researchers (can be analysts or data scientists depending on the org) is that they are asked for a recommendation or explanation, with clear context of its value and how it will be used. This way they can leverage everything they know about their data and org in the work.
At the very least, clear specifications of what is required need to be provided, but this is much harder without context. It often ends up being iterative even with good specifications.
The worst case is when a request with little thought is thrown at the data team, who have to work to fill in the blanks and go back and forth to build the specifications and context themselves. They may never know what it was used for.
The efficiency, impact and fulfilment of the data team is highest at the top of the hierachy and declines as you go further down. Sometimes you have to work at the bottom, but you don’t want to stay there for long. With good communication, stakeholder management, asking the right questions and trust built you can move up quickly.
Enabling self-serve metrics is a great way to avoid working at the bottom of the hierarchy too much. At Lyst we were fairly successful in this. Often when we were having to work lower down, it was to deal with slowly changing dimensions which are hard to offer in an easy-to-use way for self-serve.
It really helps to have a data executive, as then context flows really well to the data team or the work can be done in the most strategically impactful setting.
Standardised tooling
It’s OK to have many tools in the stack with clear and separate purposes, it’s chaos to have many tools that do the same thing. Thoughtful, architectural design of a stack is needed. It doesn’t need to be overly technical or detailed, but it needs to be purposeful and clear.
Team members having the same operating system, IDE, shared repositories, hosted notebooks… all generates efficiency and shorter onboarding for new starters. It allows anyone to pick up a piece of work should a team member be absent or leave, too. I remember there being an R script at Lyst that was occasionally mentioned by the engineering team - noone wanted to touch it as they didn’t use R! There are limits to how far you want to push conformity but don’t undervalue it either - tools like Hex allow a level of safe non-conformity not previously available.
Think two moves ahead
Promote proactively - you need to be thinking a move or two ahead for your team members’ career pathways. It's not entirely your job - they need to do their bit in deciding a direction and owning their development, but you have to triage the opportunities appropriately for them, as you have the oversight of everything coming in (which they won't), plus knowledge about how a piece of work will develop them. When you're planning on team expansion, think about how to promote high performers in the process. Whilst they are responsible for their own development, team members aren’t responsible for strategy (apart from team leads or where they are invited to have input) and you are ultimately responsible for strategy.
It needs to be possible for people to progress to high levels as individual contributors and as managers, and for hybrids of the two.
I’ve previously talked about abstraction of line management, which enables this.
Debrief
Even with all this in place, it’s unlikely you will be 100% towards where you want to be. Often, getting data a seat at the top table of an org is hard, but it is getting increasingly easier. There are a lot of spinning plates mentioned above to balance at the same time, and you won’t consistently. This is also OK and gives you an opportunity to trust in your team more and develop them as the next generation of data leaders.