Before you start experimenting, you need to have a clear and specific goal that you want to achieve. This will help you focus your efforts and measure your progress. For example, your goal could be to increase your sign-ups, conversions, retention, referrals, or revenue. Make sure your goal is SMART: Specific, Measurable, Achievable, Relevant, and Time-bound.
My approach to growth hacking experiments involves: 1. Hypothesis: Define a clear growth challenge and hypothesis. 2. Metrics: Develop measurable KPIs. 3. Audience: Segment users for targeted impact. 4. Experiment: A/B test changes with one variable. 5. Timeline: Set a data collection timeframe. 6. Data: Analyze user interactions meticulously. 7. Evaluation: Compare results with the hypothesis. 8. Iterate: Adjust or pivot based on outcomes. 9. Document: Keep records for future reference. 10. Scale: Apply successful changes widely. Continuous testing and learning will drive startup growth.
For example: Your goal could be to get more organic YouTube views on a video. You could use the features YouTube offers in a way that will amplify the success of a specific video by creating a "growth hack."
Three things to consider when defining the goal for your growth hacking experiment: 1) Specificity: Clearly define what you want to achieve, ensuring the goal is specific and measurable 2) Relevance: Align the goal with your broader business objectives to ensure it contributes meaningfully to overall growth. 3) Timeframe: Set a realistic deadline for achieving the goal, allowing for proper evaluation and iteration based on the results.
When you're designing a growth hacking experiment, start with a clear goal, like increasing website sign-ups. Your hypothesis should be specific about what you think will boost sign-ups, like changing the color of the sign-up button. Next, choose metrics to track, such as the number of new sign-ups per day. Run the experiment, analyze the data, and see if your hypothesis was right or if you need to try something different. Keep testing and tweaking until you find what works best to help your business grow!
Une autre approche consiste à définir les KPI à améliorer, puis à fixer un objectif. Par exemple, pour le taux de conversion, l'objectif pourrait être d'améliorer ce taux de 20%. Cela facilite la recherche d'hypothèses découlant directement des objectifs fixés qui elles même découlent d'un KPI
Once you have your goal, you need to identify the assumptions that underlie your growth strategy. These are the beliefs or hypotheses that you have about your customers, your product, your market, or your channels. For example, you might assume that your customers value a certain feature, that your product solves a specific problem, that your market is large enough, or that your channels are effective. Write down your assumptions and prioritize them based on their impact and uncertainty.
Keep in mind the following when identifying the assumptions that underlie your growth strategy: 1) Impact: Assess how critical each assumption is and focus on those with the highest potential impact. 2) Testability: Ensure the assumptions can be tested with available data within a reasonable timeframe. 3) Risk: Evaluate the level of risk each assumption carries.
MECE (Mutually Exclusive, Collectively Exhaustive) is a framework that promotes clear and comprehensive thinking when structuring information or identifying assumptions. In the context of assessing assumptions for a growth strategy, applying the MECE technique involves breaking down and categorizing assumptions to ensure a well-rounded evaluation. ✅️ Mutually Exclusive: 👉🏻 Ensure that assumptions don't overlap, preventing ambiguity in their interpretation. ✅️ Collectively Exhaustive: 👉🏻 Ensure that the set of assumptions covers all possible scenarios, leaving no gaps in understanding.
Continuing with my YouTube views example: You assume that putting YouTube cards feature on all of your videos will lead to more views because the person might be interested in the video promoted by the card and might click through. So it would be wise to know the average view duration and place the card at that point which people tend to be finished watching it.
A growth project should NOT be based loosely on assumptions, it should be based on data. Base your experiments on the real user behavior and insights you uncovered during your UX Research. Remember “I am not the user.”
Acknowledge the assumptions that underpin your growth strategy. These are the hypotheses you'll test. Whether it's user behavior, market trends, or product features, identifying assumptions sharpens your focus. It's the foundation upon which you build your experiment, transforming uncertainties into opportunities for data-driven insights.
Next, you need to turn your assumptions into testable hypotheses. A hypothesis is a statement that expresses a causal relationship between a variable and an outcome. For example, a hypothesis could be: "If we add a referral program, then we will increase our sign-ups by 10% in one month." A good hypothesis should be clear, concise, and falsifiable. It should also include the following elements: the action, the variable, the outcome, and the metric.
This is where the real brainstorming & ICE plays a vital role, We usually set a time of 30 - 45 mins on a daily basis where we bring all the brainstorming ideas and we calculate the score for ICE which looks like the Impact of this idea, Confidence in executing this idea, Ease is the executional difficulty we face behind this Brainstorming idea. Segmenting all the brainstorming ideas based on A3R3 funnel will help you to analyse to work towards the desired outcome.
For example: If we put YouTube cards on all of our videos. We will get more views. Therefore if we make every card point to our most recent video instead of random ones, it's going to maximize the usefulness of the feature and get us as many extra views to our most recent video as possible. That would be a "growth hack."
You can identify multiple different solutions. Then put them all into action on different segments. Run a t-test or ANOVA to determine any significant differences in outcomes based on your various solutions.
A hypothesis should be clear, grounded in data or insights, testable and falsifiable, measurable, time bound, and leave room to learn.
Formulating a testable hypothesis involves: Causal Relationship: Clearly define how one variable's change might affect a specific outcome. For example, "Implementing a referral program will boost sign-ups by 10% in a month." Clarity and Conciseness: Ensure the hypothesis is straightforward and focused. Incorporate Key Elements: Include the action (implementing a referral program), the variable (the referral program itself), the outcome (increase in sign-ups), and the metric (10% increase). A well-structured hypothesis sets the stage for effective growth hacking experiments.
Now that you have your hypothesis, you need to design an experiment to test it. An experiment is a method of collecting data and observing the effects of changing one or more variables. For example, an experiment could be: "We will launch a referral program for 50% of our users and compare their sign-up rate with the other 50% who don't have it." A good experiment should be valid, reliable, and ethical. It should also follow the following steps: define the sample, choose the method, set the duration, and plan the analysis.
For first tests, I always recommend starting with a clean 50-50% test. Design your treatment based on any research, insights, and data you have available to create the experience you feel most confident about. I don't recommend testing multiple variants at once until you've established an initial baseline of performance that you can optimize against.
For my YouTube card example you would want to use something like TubeBuddy to quickly add cards to all your content by doing batch editing instead of one by one.
Designing an effective experiment for growth hacking: Set Up Control and Test Groups: Split your user base, offering the referral program to 50% and withholding it from the other 50%. Ensure Validity and Reliability: Design the experiment to accurately test the hypothesis and yield consistent results. Define the Sample: Choose a representative sample of users for accurate insights. Select the Method: Determine the approach for introducing and measuring the variable (the referral program). Set Duration: Decide on a time frame for the experiment, sufficient to observe significant effects. Plan for Analysis: Outline how you'll analyze the data collected to assess the hypothesis. This helps in obtaining actionable insights from the experiment.
It's always a good idea to launch the experiment with a small number of users prior to known impact. You can slowly expand. For example 5%. All you need is the indication of how it lands. Be sure to measure the baseline BEFORE and after in order to understand your impact.
Once you have your experiment design, you need to run it and collect the data. This is where you execute your action and measure your variable and outcome. For example, you would launch your referral program and track how many users sign up through it and how many don't. Make sure you run your experiment long enough to get enough data and avoid external factors that could bias your results. Also, make sure you follow the ethical principles of informed consent, privacy, and transparency.
Execution is where the magic happens. Few tips based on my experience: 1. Experiment in sprints: I define 30-45 day experiments with weekly sprints 2. Always run split tests: I A/B test elements within each experiment all the time 3. Volume > Time: Sometimes you may have enough volume (traffic) to help reduce your overall experiment duration. The article suggests running the experiment long enough but what's more important is if you have a substantial volume, then collect data and move to the next step.
The duration of the experiment will depend on the project. It could be 1 week it could be 1 month. Considering it is growth, it'd be best to keep your initial project as small and modular as possible until you have indication that a greater investment is worthwhile.
Executing the growth hacking experiment: Implement the Action: Launch the referral program for the designated user group. Data Collection: Monitor and record sign-up rates among users with and without access to the referral program. Sufficient Duration: Run the experiment long enough to gather substantial data, ensuring results are statistically significant. Mitigate External Bias: Be aware of external factors that could skew results and try to control for them. Ethical Considerations: Uphold informed consent, privacy, and transparency throughout the experiment. This phase is critical in testing the hypothesis and gathering meaningful data for analysis.
Finally, you need to analyze your results and draw conclusions from your data. This is where you compare your actual outcome with your expected outcome and evaluate your hypothesis. For example, you would calculate the difference in sign-up rate between the two groups of users and see if it is statistically significant and meaningful. You would also look for any insights or learnings that could help you improve your product or strategy. Based on your analysis, you would decide whether to accept, reject, or modify your hypothesis and iterate on your experiment.
Detailed documentation is key! It's important not just to record the actual result, if you reached your expectations or not - but also any additional learnings and aspects that happened during the test. For example, were there any processes that could be improved next time? Could the collaboration with other stakeholders of the experiment have been better? Did your team encounter some unexpected, time-intensive parts of the task? These small details are easily forgotten after a few weeks - so it's crucial to hold regular retrospectives and document them. It'll build up your institutional knowledge and prevent knowledge loss, if team members leave your growth team. By documenting every experiment, you will benefit even from "failed" ones!
One of the most skipped steps in the experimentation process. Very difficult to increase confidence in future experiments if you skip this step. Get the team together and review team+experiment performance by asking the following Q's - Who was the target user? - What did we implement? What was the result? - Brainstorm - why do we think the experiment wasn't successful? Was it successful in non-business related areas? - What key insights can we take away to inform future experiments?