Failing upwards

risk-aversion = value-avoidance

Painting by Willem de Poorter entitled The Parable of The Talents or Minas.

#datasky is now definitely the home of great data discussions. This thread I had with Katie Bauer and Robert Yi is the inspiration for this post.

If you think about what the private sector is, it is like a lifecycle of money. The private sector is composed of companies of all types and sizes. The inception of a company is investment of capital to start a company. Investors in businesses have very wide ranges of risk appetite. With the lowest risk appetite we have banks, who usually provide business loans secured against the assets of the business - roughly speaking, 25% of businesses they lend to fail in their first year but the bank can recoup some of that loss in liquidation and initial repayments. With the highest risk¹ appetite you have venture capitalists, where 75% of businesses they invest in fail completely.

So from the outset, the private sector has risk baked in - the investors invest with risk to get a potential reward. The businesses take risks to try to win reward - they are themselves investors, but they can invest in a more nuanced way. Businesses can convert capital to different resources in order to invest differently. They most often do this by converting capital into labour, to offer products and services. There is no way of doing business without taking this risk.

Inside the business, this risk is divided amongst divisions and employees. Usually, the largest departments shoulder the biggest risk, like marketing/sales and product. Every employee has risk assigned to them whether they know it or not. The company has taken a risk in paying their salary, and will incur an opportunity cost in terms of what else they could have done with that capital.

Even employees in teams which sometimes feel they are support teams like legal, finance, HR and data do bear this risk. These support teams are there to help the direct risk-taking teams optimise the reward they obtain from their risk. If they don’t do this, their only value can be to meet regulatory requirements. Legal, finance and HR obviously have regulatory requirements to fulfil for a company. Data almost always doesn’t have these - therefore data needs to help the business optimise their risk and reward, or they don’t need to exist in the business at all. You don’t get to just present the data without a recommendation, you don’t get to be afraid of stepping on people’s toes, you can’t even just hold on to the money invested in you and hand it back again.

It’s why it’s so dangerous for data teams to spend lots of time building infrastructure² and little time working with the business and the management of risk. The only reason to build data infrastructure is if it increases the data team’s ability to help with the management of risk in the future. If it’s not clear that your re-platform will do this, you’re better off limping along with Redshift and focusing on the business. Do less data work and more business work - take more risk by getting involved with your business’s risk-taking more often. The data team that does this for three years leading into bad times usually won’t get axed - it has broadened the shoulders to bear risk. The data team that re-platformed for three years and increased operating expense whilst not helping the business bear risk, should get axed.

None of this probably is a surprise to anyone who has read this substack for a while, but why do some go down the bad path? There are a subset of humans who can’t cope with risk - they agonise over the smallest decisions. It may be genetic, it may be environmental, it’s almost certainly both. My point is that the private sector always involves some amount of risk-taking. To be effective in the private sector, you need to be comfortable with this and in fact become good at it. The way to manage risk in the private sector is not to try to eliminate it, it’s about taking enough diversified risk often and in small enough doses - it’s portfolio theory.

What I think happens is that people come into the private sector who shouldn’t be there. They excel at politics instead of helping their business take risk, they bring in other people who are likeminded, they help those people get promoted, they build up their CV… When it comes to a point where they haven’t really delivered much, years have passed but they can apply for a new job having done A years at B company, and X years at Y company and so on. Their CV looks great, they’ve sat on committees, they’ve been on more Zoom calls than most, they went on the management offsite, they didn’t make enemies³…. but no-one can point to what exactly they did do.

So they move on to another role, at another company. A higher-ranking role at a bigger company - and the loop increments again. Then they re-hire their acolytes, to shore up their position. It’s like a tumour growing and increasing its blood supply. This is a real problem with recruitment at companies - HR don’t really know to test for what someone actually did and what risks they took. HR may not even think in this way. Perhaps a new way to get people to fill in applications could be: provide a list of risks you took, or helped your business take, in each role you’ve held and what the outcomes of them were. Many flashy-looking applications today would cease to look so shiny in this light. Many won’t even be able to link their risks taken with real business outcomes.

1. Well maybe people who invest in crypto have a higher risk tolerance, but these are often the same people.

2. Don’t take this to mean you shouldn’t bother with infrastructure. I’ve built plenty of it as you can see if you read some of my earlier posts. If self-serve analytics is possible in your org, as it has been for some of mine, having better infrastructure allows your stakeholders to use data for more decisions, quicker - this is an indirect way for data to help bear risk.

It is inferior to the more consultative direct ways for data to work with stakeholders, but oftentimes you aren’t trusted to do this without getting the basics right.

My point is: don’t spend all your time on the basics and infrastructure.

3. At least not overtly, you’ve probably been one of their enemies without knowing. You usually find this out after you leave the company and you go back for someone’s leaving drinks.