Skip to content
Sign in

How I fake an A/B test when there's no real split testing

Tips & strategies 1,554
Mira K.

You can't split test organic pins properly, so stop trying to make it clean and make it honest instead.

I change one thing. Same URL, same board, same week, two designs, a day apart. Then I leave them alone for a month, because a two day read on a platform where pins take weeks to find an audience tells you nothing at all.

The hard part is not touching the headline and the image in the same test. I did that for a year and learned precisely nothing from it.

# analytics # testing # workflow

3 replies

Aisha B.

A day apart is enough to matter if one of them lands on a better day. I put both out in the same hour on different boards and accept the board difference as noise instead.

Claire Dubois

A month minimum matches what I see. I've had a pin look like an obvious loser at two weeks and turn out to be the best performer of the quarter by about day 50.

yuki.creates

Add impressions to whatever you're measuring. If design A got ten times the impressions of design B then comparing raw clicks is telling you about distribution, not about the design, and you'll draw the wrong conclusion confidently.

Join the conversation. Sign in to reply