SPRINGBOX / Insights

The State of Optimization at Springbox

by Melia Watkins, June 27, 2016

Q&A with our Director of Optimization, Hayley Wick

Springbox’s Director of Optimization, Hayley Wick, has accrued an impressive portfolio of credentials.

She has worked for a wide range of companies including the Chicago Tribune, iProspect, Performics and USAA. In her various positions, her knowledge of optimization has exponentially grown. Given her depth of knowledge, we sat down with  Hayley to discuss her vision for optimization at Springbox.

First, how would you define optimization?

analytics_copy.jpgOptimization is the art of improving a site with hypothesis testing; otherwise known as A/B, split, or bucket testing.  It’s as simple as comparing two versions of a current web page, testing them against cookied visitors at the same time and determining which version is the winner based on conversion findings. Optimization is an essential practice for all website maintenance and is always data-driven. Keep it simple and always be testing.      


What’s the difference between A/B and multivariate testing?

There’s a thin line between growth and insights, which Chris Goward of WiderFunnel describes very well. Hypothesis testing is definitely a significant contributor to growth; however, exploration and insights should never be overlooked. A/B testing is a great solution to move the growth needle, whereas multivariate testing should be considered an additional insights solution, assuming the client is open to the appropriate amount of time that is needed. Sacrificing a “quick win” or two is still considered a win because new insights are the final outcome.

Anyone who engages in a testing practice needs to understand that, depending on what they would like to achieve, time will always play a factor. Very few organizations with an online presence have the level of traffic that Amazon experiences on a daily basis; therefore, smaller organizations should be heavily focused on both growth and insights.  If fast growth is the primary goal, then additional user behavior insights might be lost, but that is not to say that growth is not an important conversion insight. Testing is a gamble and the more practice, the more insights gained and the better one performs; therefore, time is essential when optimizing. Allow time for both quick wins and long-term insights.


What companies do you think set a great example for a testing and optimization practice?

I look up to Expedia and Amazon as I believe they do a fantastic job in terms of personalization efforts. It is evident that they know what to do with the data they collect. These two also make optimization a primary discipline at their organizations, demonstrating their commitment to iterative improvement. Expedia and Amazon test thousands of hypotheses every year and have large teams dedicated to setting and executing a testing strategy. These organizations have the resources to support optimization because it has proven to be crucial to their business success.


What are key benefits of optimization? Why would a company need to A/B test or optimize their website?

Optimization or hypotheses testing is the practice of making your site better — it’s as simple as that. Data is king. If you have the data to support your hypotheses, you will have no issues implementing what were previously "hunches." It is a no-brainer to test any design before production to see how users react to ensure it is the most effective strategy.


Do you have a preferred platform? What platforms can Springbox use to A/B test?

optimizely.pngSpringbox can work with clients regardless of the testing tool platform, but we will always educate on the strengths and weaknesses of each testing tool before executing a testing plan. My optimization tool of choice is Optimizely because it has worked better than the several other platforms I have worked with. Optimizely’s customer service is the best I’ve ever seen; they are a well oiled machine that I trust in recommending to clients.


What is Springbox’s relationship with Optimizely? Why is Optimizely our testing platform of choice?

LogoMark_Color-2.jpgOptimizely is our preferred partner and Springbox is in the process of achieving the rank of a 2-star provider. Optimizely’s Optiverse is unlike any other education platform I have seen before. One of Optimizely's goals is to equip their partners with the tools they need to become masters at using the tool, empowering both clients and members. The simplicity of the tool also makes it superior to other tools I've used in the hypotheses testing arena.


What types of tests can Springbox perform for an optimization or testing practice? Are there any limitations in our testing capabilities or types of tests we are able to execute? Are there strengths of certain tests over other types (multivariate vs. A/B)?

We aren’t limited in our capabilities to test, for Springbox it simply depends on the time frame that the
client lends us.

Some limitations to take into consideration are as follows:

- The current framework of the site. If the site has a lot of dynamic content, it would immediately require development work to perform testing. This requires additional time to run and implement a test, but it would not be impossible by
any means!

- Fewer than 50K unique visitors (MUVs). The fewer the number of the unique visitors, the longer the period of time required for any given test to reach statistical significance.

- A traffic or lead boosting campaign. If a client is executing a campaign within a specific time frame, testing results may be skewed by this influx of traffic.

I try to educate toward A/B testing, because with multivariate testing you are diluting traffic, which extends the testing duration and makes it more difficult to tease out meaningful insights. The environment being tested needs to remain completely untouched in multivariate testing. In A/B testing, tests can be completed much more quickly with clear and significant results. In general, we don’t want to turn any organization away from conversion rate optimization opportunities.


Interested in seeing what other organizations have achieved with an optimization practice?

Check out "Top 10 Hypothesis Tests"




Topics: Optimization