The Ample Sample
How much information do you need in order to make a decision?
This sounds like a pretty simple question to answer, but there’s surprising complexity embedded in this riddle.
Maybe a more pertinent way to ask this might be: what’s the least amount you need to know to make a good decision?
US Grappling was very, very data-driven. I was very proud of this aspect of our little company: we really cared about studying what had worked in the past and what hadn’t worked, so we could navigate toward a better outcome.
My business partner, Chrissy, had a steel trap of a mind. She loved to process data and could think very, very quickly on her feet. This was a real superpower: she could hold a ridiculous amount of information inside her brain at once, access it in useful ways, and ultimately get things done in a way I had never before encountered.
Her husband and my other business partner in USG, Brian, is much more of an analyst, able to process information carefully and deeply. While Chrissy’s mind could process things in an instant, Brian’s superpower was zooming out and slowing down to think about the big picture.
I think I was somewhere in between the two of them, and all three of us loved gathering and analyzing data. I don’t think I had a data analysis superpower, but I do think me being in the middle helped all three of our minds when considering what to do next.
The only problem was that we were kind of creating the data as we went along, so we only had a limited set of information to work with. Much of the time, we needed to make an educated guess as to what turnout would be like, so we could hire enough referees, book a venue that would be large enough, and so on.
Some folks will take a look at an incomplete data set, and think: no way, there’s not enough information to make a decision here.
If you’re a business owner, you don’t always have the opportunity not to decide.
Ultimately, we did the best we could with what limited data we had, and it turned out to be really useful… and maybe enough to make a decision.
With our friend Jeff Shaw, we drew conclusions based on over 4000 matches to find out what the most common types of submissions were. We also got a good idea of average match lengths, which was incredibly important for a new concept we introduced to the BJJ world at a large scale for the first time: submission only, no time limits jiu jitsu tournaments.
You can probably imagine why knowing match lengths was very important. The only way a competitor could win was by catching their opponent with a joint lock or choke, we would need to understand how long the day might be. We knew we had to eliminate bias, open our minds, and decide based on what the limited data actually said.
While having more data is generally a good thing when you’re making a big decision, there are sometimes choices that need to be made without having much data at all. The good news is that we have a lot of great examples of folks making bold, yet accurate predictions based on very, very limited data.
Pierre-Simon Laplace was a brilliant polymath who is best known today for his contributions to math and physics, especially his Celestial Mechanics, which picked up the torch from Newton and provided a robust mathematical framework that changed astronomy forever.
During the late 18th century, Laplace became interested in understanding how waves worked, especially tides. His idea was that the gravitational pull of the Moon and Sun influenced the Earth’s oceans, inducing them to slosh around at regular intervals.
By thinking carefully and deeply, Laplace reasoned that the speed of tidal waves depended on how deep the water was, and he knew that he could simply invert his equation to figure out the ocean’s average depth.
His equations told him that the average depth was around 4 KM, and it turns out to be about 3.7 KM. Laplace’s way of thinking—of inferring natural phenomena by inverting equations—is very common in lots of scientific fields today.
Laplace’s estimation of the ocean’s depth is impressive, but it relied on another brilliant calculation that was already well known in Europe by this time: the circumference of the Earth. Laplace needed to stand on the shoulders of another giant.
That giant was Eratosthenes, a Greek philosopher (polymath, really) who took great pains to study shadows in two faraway Egyptian cities. He wanted to figure out the length of the shadows that vertical poles cast at the same time. By measuring the angles of these shadows and knowing the distance between the two cities, basic trigonometry told him the circumference of the Earth.
If this tale sounds familiar, it’s probably because Carl Sagan told you this as a bedtime story:
Eratosthenes knew full well that the Earth was round, laying down the foundation for what should be common knowledge today. With shockingly little data, he was able to use reason and inference to get incredibly close to the actual circumference.
writes about jumping into the unknown, and how it’s necessary at times. Like us with US Grappling, you’re not always going to have a ton of information, but you might need to make a decision. Instead of falling into analysis paralysis, you need to assess the situation and get something done.What you end up doing might not be the best decision, but it’s the best decision you could make under the circumstances. Whenever I have to do this, I am completely at peace with the consequences, because I know that when I look back on the decision, I’d make the same choice, given the limited information I had at the time.
How about you—have you ever had to make an important decision with incomplete data? What are some examples, if you don’t mind sharing?
Too much data can also pollute decision making. You need enough of the right data. I've got an essay coming out shortly on Infobesity talking about this.
I want to do a TV show about the history of animation that would be like what Sagan did with "Cosmos".