SMX Advanced recap: Lies, Damn Lies, and Search Marketing Statistics
Are your marketing tests giving you valid results? Columnist Mark Traphagen summarizes insights from a presentation by Vistaprint's Adria Kyne at SMX Advanced.
At SMX Advanced 2016 in Seattle, Adria Kyne, SEO Manager (North America, Australia, and New Zealand) for Vistaprint, gave a presentation on doing search marketing testing — or any kind of marketing testing — that provides valid results.
Kyne made the point that by not understanding the importance of validity of sample size, we often end up unintentionally lying to ourselves, and we may actually “prove” the opposite of what we think our tests are showing.
Common problems with marketing testing
We need to start with understanding the point of our testing:
- We want to know what is really happening when people visit our site.
- We want to be sure we can use the results to predict likely future behavior.
We also need to understand the basics of hypothesis testing:
- We want to know if the variation we are testing is better, worse or the same as the original.
- We don’t want to see a positive outcome that isn’t really there (a false positive, or Type I error).
- We don’t want to miss a positive outcome (a Type II error).
Opinions expressed in this article are those of the guest author and not necessarily Search Engine Land. Staff authors are listed here.