Member profiles and follows

1. index page now link to a solo dish page around users

2. user profile pages added

3. users can now follow bloggers and categories

 

Corresponding experiments are as follows:

preferences (personal word cloud) (Due Dec 7 12am)

– vote 20 on a category/blogger

– prompt “do follow this category/blogger”

– add personal cloud “blogger, category” on popular

– add personal cloud “blogger, category on vote, remove main category

– add follow btn on profile page

– assumption: adding follow will increase 

– metric: #added into follow cloud, #clicks on follow cloud, # of votes, #login 

– prediction: #added into follow cloud 20% from prompt, 

#clicks on follow cloud (+20%) vs. #click on category

# of votes (10% increase)

# re-login (5% increase)

social  (Dec 5 12am, Dec8)

– blogger profile (name of blog, location, twitter, fb)

– add stats (#sweet # savoury #guess #votes #pt)

– You may also like (remove sweet, savoury, main, appetizer, dessert, side dish)

– assumption (people will visit profile page)

– metric: outbounds, page view, # of login, # of message

– prediction (a) outbound: from dish page (35% decrease)

 pageview: additional clicks on left hand canvas and profile page (increase)

(b) outbound from popular page 

 

Experiment O, P, I part 2 completed

Experiment O: 

 Assumption: Players will share if they got a perfect score.

Prediction: 15% conversion

Result: 7.5%

Decision: Keep anyways as the prompt is not intrusive.

 

Experiment P: 

 Assumption: Players will play more guess games if they are shown more details on how they are doing.

Prediction: 10% increase in games played

Result: 74% decrease!  People get discouraged seeing a bad record

Decision: Removed and may try to only show record for people who are doing well

 

Experiment I part 2: 

Assumption: Trying to optimize top 10 message

Prediction: 3% conversion

Result: 20% on prompt with date. Not that significant (7/35) but is quite better than the other 2 prompts.

Decision: Keeping prompt with date.

Facebook open graph

We’re now setup for facebook open graph that’s supposed to go live this wednesday. So far, there are two actions that would publish to our users ticker and timeline. On complete guess game, and on favouriting a recipe. We also added an individual recipe page to properly publish to the graph.

Experiment N, I, J completed

Experiment N: 

Assumption: share win on hard game

Prediction: 5% conversion rate

Results: 41 subjects, 10 shares, 24% conversion

Decision: Keep prompt and doing a follow up experiment that prompts users to share when there’s a 65% failure rate on guess game (currently set at 75%)

 

Experiment I:

Assumption: share top ten

Prediction: 3% conversion rate

Results: 524 subjects, 10 shares, 2% conversion (*due to bug, # of subjects that were actually shown the share prompt is likely less than the # reported)

Decision: Doing a follow up experiment to try to increase this percentage. 2 new versions of share prompt with updated text. 2% determined acceptable though.

 

Experiment J:

 Assumption: signup from duel 2

Prediction: 30% increase in signup Results: All versions had the same signup conversion

Decision: Reverting back to original call to action.

3 New Experiments

Experiment O:

Assumption: Players will share if they got a perfect score

Metrics: % of players who share

Prediction: 15% of players will share

 

Experiment P:

Assumption: Players will play more guess games if they are shown more details on how they are doing.

Summary of the player’s record on top

Details on games the player has played

Metrics:

# of guess games played per player

# of revisits to play more guess games per player

Prediction:

10% increase in # of guess games played

 

Experiment Q:

Assumption: Players will vote more if the pictures are matched on more detailed categories (currently matching on sweet & savoury, testing on more detailed categories such as chicken, beef, pie, etc)

Metric: # of votes per player

Prediction: 7.5% increase in votes per player

 

And that is all. Now onto open graph…

Increasing sharing

Added 2 new experiments to tries to increase sharing on our site.

 

Experiment N:

Assumption: Players will share a guess game if it was a hard game (75% incorrect) and they got it right.

Metrics:

# of tweets and likes

Prediction:

5% of displayed will share

 

Experiment I:

Assumption: Players will share if their vote got a recipe into the top ten (max 2 prompts/session)

Metrics:

# of tweets and fb shares

Prediction:

3% of players will share

 

Also, we finally added an about us page after many requests from our users.  Those pics we took at Google were perfect for this!

 

 

Experiments K & J completed

Assumption J:

Player signup will increase if shown a better signup prompt page after first 25 votes.

Metric:

signup conversions on prompt

Prediction:

30% increase

Results:

Decision:

Revert back to controlled.  Going to follow up with another experiment this week with a different call to action though.

 

Assumption K:

Players will signup to play more games (prompted after 3 games played, and cannot play more until signed up)

Metric:

-signup conversion on prompt

-# of games played/non signed up player (watching for decrease here since we are putting up a wall on accessing more than 3 games)

Prediction:

same as controlled conversion % from J

Results:

Expected around 4.5% conversion. Result was around 35%.

Decision: 

Users need to signup before playing more than 3 games.

Guess Game gaining momentum

With all the excitement happening with Guess Game lately, we’ve added an auto email for new guess games whenever one gets created (similar to our auto email when new dish has been added).

 

Long overdue, but we’ve also added a quick communication settings to the app so our users can select what and when to receive emails from us.

 

We decided to drop our leaderboard hypothesis for now. The initial assumption was that having a leaderboard and displaying relevant parts of it in the vote page and guess game page will drive our users to perform more actions as they can see in real time what actions they need to do to get into the next spot. However, there were challenges with screen real estate and implementation complexities that we decided against the experiment for now. We definitely need to do more work on the community/gamification side of things as our system doesnt really make much sense right now without the community stuff in place.

Increasing conversion, experiment dashboard updated

Added two experiments that tries to increase signups.

Assumption J:

Player signup will increase if shown a better signup prompt page after first 25 votes.

Metric:

signup conversions on prompt

Prediction:

30% increase

 

Split testing the following:

Community Call to action

Discovery call to action (screen is pulled from localhost, hence the shortage of pics)

Controlled call to action

Assumption K:

Players will signup to play more games (prompted after 3 games played, and cannot play more until signed up)

Metric:

-signup conversion on prompt

-# of games played/non signed up player (watching for decrease here since we are putting up a wall on accessing more than 3 games)

Prediction:

same as controlled conversion % from J

 

Also updated our experiment dashboard. Added some key metrics data in addition to just having conversions. Hopefully this will reduce the amount of manual reporting needed to validate our experiments.

Seven Experiments : 3 validated hypotheses and 4 invalidated hypotheses

Experiment A (Growth Engine):

Assumption:

• Foodblogger will invite their friends if they can create a guess game.

Prediction: 

• 1 foodblogger will bring in 7 people

Result:

• 1 foodblogger brought in 2 people
• 2 facebook shares + 2 twitter shares for 9 foodbloggers
• ran experiment for 12 days October 16 – 27

Conclusion:

• Invalidated Hypothesis
• 2 people/blogger < 7 people/blogger

Follow up: 

• Don’t allow bloggers to create their own games
• too time consuming
• not enough value

 

 

Experiment B (Value Engine):

Assumption:

• Adding skip button on guess game will result in more games played

Prediction: 

• 5% in games played

Result:

• very few people clicked on skip button
• ran experiment for 7 days October 14 – 20

Conclusion:

• Invalidated Hypothesis

Follow up: 

• Remove skip button from guess game

 

 

Experiment C (Value Engine):

Assumption:

• Players will add comments to recipes

Prediction: 

• 1.0 comment/player

Result:

• 23 comments / 140 user = 0.16 comments/user
• ran experiment for 14 days October 18 – 31

Conclusion:

• Invalidated Hypothesis
• 0.16 < 1.0 comment/player

Follow up: 

• Remove comment box or
• Run another experiment (maybe list # of comment and redirect to blog post for comment) This may help us create more blogger value with increased outbound links.

 

 

Experiment D (Value Engine):

Assumption:

• Adding blog pics will increase comments

Prediction: 

• 5% increase

Result:

• 4 comments / 77 users = 5.2% (control group, with no pictures)
• 9 comments / 64 users = 14.0% (1 week experiment)
• 14%/5.2% = 170%
• ran experiment for 12 days October 20 – 31

Conclusion:

• Validated Hypothesis 🙂
• 170% > 5%

Follow up:

• Add blog pictures for all

 

 

Experiment E (Value Engine):

Assumption:

• Word cloud will increase the number of pageviews on popular page.
• 204 pages with “cat=” / 529 Unique visitor = 0.38 pages/ unique visitor (Oct 17 – 23)

Prediction:

• 50% increase
• 0.57 pages/unique visitor

Result:

• 515 pages with “cat=” / 601 Unique visitor = 0.86 pages/ unique visitor (Oct 24 – 30)
• 226%
• ran experiment for 9 days October 23 – 31

Conclusion:

• Validated Hypothesis 🙂
• 226% > 50%

Follow up:

• none. Leave word cloud. 

 

Experiment F (Value Engine):

Assumption:

• Adding guess game buttons on popular page will increase average # of games played per player

Prediction:

• 20%

Result:

• 769/92 = 8.35 games/player (previous 2 week average)
• 504/31 = 16.25 games/player (1 week experiment)
• 16.25/8.35 = 95% 
• ran experiment for 9 days October 23 – 31

Conclusion:

• Validated Hypothesis 🙂
• 95% > 20%

Follow up:

• Add guess game buttons for all recipes on popular page

 

 

Experiment G (Value Engine): 

Assumption: 

• Guess games are too hard.

Prediction:

• 0% too easy
• 50% just right
• 50% too hard

Result:

• 19% too easy
• 58% just right
• 23% too hard
• ran experiment for 8 days October 24 – 31

Conclusion:

• Invalidated Hypothesis 
• 23% < 50%

Follow up:

• What should the balance of difficulty be?
• Experiment: Showing the difficulty of the game will increase the number of games played
• Experiment: Call to action “Feedback” vs. “Play another game”

 

Experiments on our Erase DryWall

(Thanks to the Work Republic for the idea of painting our wall)