Design effective experiments

3 copia 3.png

If you’re in need of investment capital, whether it's through banks or angel investors, you will probably need to prove whether your business is going to work or not. This is where designing effective experiments is key. A common misconception about startups is that you need massive capital outlays prior to starting to cover the costs of development or manufacturing. I'm going to walk you through the steps you can take to test and validate your business without actually starting it. 

So, let's run through an example of designing effective experiments. 

We’ll use an example problem that we’ve identified in the market: International Students often have trouble finding the right jobs in New Zealand because of their lack of work experience. 

My potential solution would be to create a website that matches businesses with students. This would work with businesses actively seeking students with minimal experience and students who are looking to gain more experience. To form an effective experiment for testing prior to actually building a website I need to follow these steps:

3 copia 2.png

First, you need to create a hypothesis

This is a clear true/false or yes/no statement. There's no ambiguity around it, which means there's no ‘maybe’, ‘potentially’ or ‘perhaps this could work’. It can only ever be correct or incorrect. 

4 copia 3.png

Next, you want to identify your assumptions based around around idea. These are uncertainties that you have that could potentially flaw and foul your business. 

Thirdly, you want to choose a method to test this. There are a few different techniques you could use: observation, surveys, fake ads, landing pages. Basically any technique that you can use to measure and validate against your hypothesis is perfect. 

Lastly, you want to choose a minimum criteria for success. This is the minimum number of our method results that would lead us consider our hypothesis as true or false. In any experiment it’s important to identify what numbers we would consider a successful test before we actually test. If the number is too low it might not represent enough of a demand. If I set my success criteria at 1 of 10 people interviewed agree with my hypothesis it doesn’t mean a successful test. 6 or 7 out of 10 people might better represent my users.

That’s the basic framework required to create effective experiments, now we can look at applying this to our example problem:

A hypothesis should generally start off with “I believe customer X has a problem achieving goal Y”. In my example I would phrase this as:

“I believe that international students have a problem finding relevant job experience before getting a job in New Zealand”

Notice that in the way we’ve written it, the answer can only be true or false - students either do or don’t have a problem getting job experience. 

Next, I would list my riskiest assumptions. These are the uncertainties that I identified around my business model. These are behavior, mental or action based and these assumptions need to be true in order for me to be able to say that my hypothesis is true.

5 copia.png

A few assumptions I identified for students are:

  • Students actually want to find relevant job experience

  • Students are already looking for experience on various sites and social media platforms

  • Students are looking for easier ways to show interest to employers

  • Students will generally talk to about 10 employees but after 10 will stop looking and give up

6 copia.png

Now, with these assumptions, I can a few methods to test whether this is true or false. The methods that I have considered to test my assumptions and to test my hypothesis are:

  • Sponsored Social Media Posts to gauge interest

  • Fake Ad for my platform to see if people are interested enough to register

  • Flyers in supermarkets and universities asking for their participation in surveys and interviews

  • Start a Facebook group offering work experience to see if students join

Lastly, I want to choose my minimum success criteria. This is the minimum number of students out of the total number involved in my tests that have agreed with my hypothesis for me to consider it true.

For my social media posts and fake ad I might consider that if at least 10% of people who see them click through and sign up then there is enough people for me to say this is a problem that the students face. If I had created a Facebook group and had 200 students join who are actively seeking work experience I could then say that there is demand for my solution.

From here you can apply your findings to develop an initial Minimum Viable Product or if you found your hypothesis to be false you can use your findings as a start point for some additional research around your idea before you start the experiment process again.

Experiments, if designed well can be monumental in giving you insights into your problem and their users. It will allow you to see if you’re heading in the right direction or if maybe you have some extra questions/research that needs to be conducted before you progress. The beauty of these experiments is that they can be cheap and save you time and resources that you might waste on developing a solution that doesn’t meet the needs of the market.