Seven Experiments : 3 validated hypotheses and 4 invalidated hypotheses

October 31, 2011 | christinetho

Experiment A (Growth Engine):

Assumption:

Foodblogger will invite their friends if they can create a guess game.

Prediction:

1 foodblogger will bring in 7 people

Result:

1 foodblogger brought in 2 people
2 facebook shares + 2 twitter shares for 9 foodbloggers
ran experiment for 12 days October 16 – 27

Conclusion:

Invalidated Hypothesis
2 people/blogger < 7 people/blogger

Follow up:

Don’t allow bloggers to create their own games
too time consuming
not enough value

Experiment B (Value Engine):

Assumption:

Adding skip button on guess game will result in more games played

Prediction:

5% in games played

Result:

very few people clicked on skip button
ran experiment for 7 days October 14 – 20

Conclusion:

Invalidated Hypothesis

Follow up:

Remove skip button from guess game

Experiment C (Value Engine):

Assumption:

Players will add comments to recipes

Prediction:

1.0 comment/player

Result:

23 comments / 140 user = 0.16 comments/user
ran experiment for 14 days October 18 – 31

Conclusion:

Invalidated Hypothesis
0.16 < 1.0 comment/player

Follow up:

Remove comment box or
Run another experiment (maybe list # of comment and redirect to blog post for comment) This may help us create more blogger value with increased outbound links.

Experiment D (Value Engine):

Assumption:

Adding blog pics will increase comments

Prediction:

5% increase

Result:

4 comments / 77 users = 5.2% (control group, with no pictures)
9 comments / 64 users = 14.0% (1 week experiment)
14%/5.2% = 170%
ran experiment for 12 days October 20 – 31

Conclusion:

Validated Hypothesis 🙂
170% > 5%

Follow up:

Add blog pictures for all

Experiment E (Value Engine):

Assumption:

Word cloud will increase the number of pageviews on popular page.
204 pages with “cat=” / 529 Unique visitor = 0.38 pages/ unique visitor (Oct 17 – 23)

Prediction:

50% increase
0.57 pages/unique visitor

Result:

515 pages with “cat=” / 601 Unique visitor = 0.86 pages/ unique visitor (Oct 24 – 30)
226%
ran experiment for 9 days October 23 – 31

Conclusion:

Validated Hypothesis 🙂
226% > 50%

Follow up:

none. Leave word cloud.

Experiment F (Value Engine):

Assumption:

Adding guess game buttons on popular page will increase average # of games played per player

Prediction:

Result:

769/92 = 8.35 games/player (previous 2 week average)
504/31 = 16.25 games/player (1 week experiment)
16.25/8.35 = 95%
ran experiment for 9 days October 23 – 31

Conclusion:

Validated Hypothesis 🙂
95% > 20%

Follow up:

Add guess game buttons for all recipes on popular page

Experiment G (Value Engine):

Assumption:

Guess games are too hard.

Prediction:

0% too easy
50% just right
50% too hard

Result:

19% too easy
58% just right
23% too hard
ran experiment for 8 days October 24 – 31

Conclusion:

Invalidated Hypothesis
23% < 50%

Follow up:

What should the balance of difficulty be?
Experiment: Showing the difficulty of the game will increase the number of games played
Experiment: Call to action “Feedback” vs. “Play another game”

Experiments on our Erase DryWall

(Thanks to the Work Republic for the idea of painting our wall)

Comments are in

October 19, 2011 | kevtsoi

No integration into emails or the messaging system yet. Just simple comments on the duel page for now.

Growth Engine Hypothesis

October 17, 2011 | kevtsoi

We started our experiment with having bloggers create guess games this week. Our assumption is that bloggers will not only help us generate content, but more importantly, bring their friends and fans to Knapkins to play their own games. This is the first step in building social right into our app. The next iteration would be to have people create games solely for the purpose of challenging their friends. Kind of like Zynga’s Hanging With Friends for recipe guessing. We also started being more rigorous in defining our experiments. So far we’ve just been shipping and looking at data after the fact to make sense of things. Now we are making sure that we have the right cause and effect metrics (or measuring of metrics) nailed before launch, along with our assumptions and predictions of course. Will go into more details on our process next week. For now, here are some screen shots of our guess game create process.

step #1: Select a recipe