In part 5 of this 5-part blog series about ‘Building a Culture of Optimization’ I’m going to talk about the importance of sharing your wins and bringing your organization along with you. You can see part 1, part 2, part 3 & part 4 here.

Part 5: So you’ve found a big win. Now what?

Ensure you’ve double triple checked your results!

Are they statistically confident? Did you control for external variables?

Why is this important? A personal example…

I ran a test where we found significant uplift over our control from a couple of test variations, but one version stood out as the clear winner. After closing the test, reviewing and analyzing the data, I communicated the results and recommendation to launch the winner to the rest of my organization. Most people were very excited.

But given I work in a data hungry organization, I had an engineer who questioned my findings (love the cross-functional involvement!). After he did a big of digging into the logs, he noticed there was an exceptionally high amount of traffic from the Philippines going to the winning variation. When we went back and filtered out all non-US data we actually had a different winner (thankfully still a good story). Oops! The test was supposed to be English/US only, so we hadn’t gone the extra mile to be sure to exclude other geos. Lesson learned – I had to go back to my organization and let everyone know that I had been mistaken, and we in fact had a very different winner for this test.

Safe to say I only made that mistake once!

Next – Make a deck!

Your deck should include:

• Executive summary
• Test design: traffic %, confidence, time period, variation screenshots, description of what you did (KISS – keep it simple, stupid!)
• Results
• Recommendation & Next Steps

Then… get it in front of all relevant stakeholders. Schedule a review meeting, send a wrap up email, communicate findings and changes to your broader organization.

Keep your results in a widely accessible place (ex. tab within roadmap)

In part 4 of this blog series I talked about having one source of truth via a master test doc or spreadsheet. In my spreadsheet (example here), I keep all test designs and results in the roadmap tab so that everyone in the organization has a single place to go for info on current and past tests.

To wrap up this whole mini-series, Culture Shift is an ongoing process…

• It doesn’t happen overnight… but can start with 1 big win
• Keep the momentum
• Bring everyone along: Marketers, Sales, Devs, Eng, Execs
• Reach across groups/teams for test ideation & input
• Consider a ‘roadshow’ of big wins to different groups/stakeholders
• Have ‘one source of truth’ for your organization to reference for everything test related

There you have it, follow these 5 steps and you’ll be well on your way to educating your organization on the benefits of and methods for testing & optimizing your web properties and soon you should also start to see a changing tide within your group.

Happy testing!