Low-Traffic Amazon Image Tests: Set a Minimum Detectable Effect Before Declaring a Winner

*By Rahul Bisht · Growithamazon.com · November 14, 2026 · about 9 minutes*

The Short Answer

Before you launch an Amazon main image A/B test on a low-traffic ASIN, decide the smallest lift you would actually act on. That number is your minimum detectable effect (MDE). If your traffic cannot detect it inside the test window, the test will most likely finish "inconclusive" no matter how good your new image is. So either test a bolder image, pick a busier ASIN, or skip the test.

That is the whole idea. The rest of this post shows you how to do it with numbers you can check yourself.

Table of Contents

• ●The Short Answer

• ●Why Low-Traffic Image Tests Keep Lying to Sellers

• ●What "Minimum Detectable Effect" Means in Plain Words

• ●A Worked Example With Real Numbers

• ●How to Set Your MDE Before You Hit "Schedule Experiment"

• ●When Your ASIN Cannot Reach the MDE

• ●How to Read the Result Without Fooling Yourself

• ●Quick Checklist Before You Launch

• ●FAQ

Why Low-Traffic Image Tests Keep Lying to Sellers

Here is a story you may recognise. You swap in a new main image through Manage Your Experiments. Two weeks in, Version B is ahead by a few points. You get excited, end the test, and push the new image live. A month later sales look exactly like they did before.

Nothing broke. The "lead" was never real. When an ASIN gets only a trickle of sessions, a handful of orders either way can swing conversion rate by whole percentage points. That is random noise, and noise looks very convincing when you want a winner.

Manage Your Experiments (MYE) is Amazon's built-in A/B tool for brand-registered sellers, and main images have been testable in it since 2021, according to EcomCrew's walkthrough. Sellers can usually pick a duration between 4 and 10 weeks. The tool is good. The trap is how we use it when traffic is thin.

Not brand-registered yet? Brand Registry is the entry ticket, and our guide to Amazon Brand Registry requirements walks through eligibility, cost and timeline.

What "Minimum Detectable Effect" Means in Plain Words

MDE answers one question: how much better does the new image need to be before this test could reliably show it?

Think of a bathroom scale that only moves in 2 kg steps. If you lose 500 grams, the scale cannot tell you. A test with too little traffic works the same way. It has a smallest change it can see, and anything smaller disappears into the noise.

Statsig makes a useful point in its guide to MDE: MDE is as much a business decision as a statistical one. You are not asking what lift you hope for. You are asking what lift would change your decision.

Three things drive it, and you control all three:

Baseline Conversion Rate

Your current rate on that ASIN.

Sessions per Version

Half your traffic goes to each image.

The Lift You Care About

Smaller lifts need far more data. As a rule of thumb, halving the lift you want to detect needs roughly four times the sample.

If you want a deeper look at the maths, AB Tasty's sample-size guide lays out the formula and the usual pitfalls, including the habit of peeking at results before the test has gathered enough data.

A Worked Example With Real Numbers

Let's say one ASIN converts at 10% and you want to detect a 10% relative lift, meaning 10% becomes 11%. Using the standard two-proportion formula at 95% confidence and 80% power, you need about 14,750 sessions per version, so roughly 29,500 sessions in total.

Now compare that with a typical small brand. At 150 sessions a day on a 50/50 split, each version collects about 525 sessions a week. Reaching 14,750 would take around 28 weeks. Amazon's tool tops out at 10 weeks. That test cannot answer your question.

Turn it around and ask what a 10-week test *can* detect. Here are approximate figures for a 10% baseline, a 50/50 split, 95% confidence and 80% power:

Read the 150-session row slowly. On that ASIN, a new image has to lift conversion by roughly 17% before a 10-week test can be trusted to catch it. Many image changes do not do that. If yours is a subtle tweak, such as a slightly warmer angle or a cleaner shadow, the test was set up to fail before it started.

One honest caveat: these are planning numbers from a textbook formula. Amazon's engine reports results in its own way, and your real baseline will differ. Use the table to decide whether a test is worth running, not to predict Amazon's exact verdict.

If you are still deciding which ASINs deserve this kind of effort, our 5-signal framework for finding winning products is a good place to start, because the best test candidates are usually the ASINs with real demand behind them.

How to Set Your MDE Before You Hit "Schedule Experiment"

You only need three steps and a calculator.

Step 1: Write Down Your Baseline

Pull the last 30 to 60 days of sessions and conversion rate for the ASIN. Use a stable stretch, not a promotion week.

Step 2: Pick the Lift Worth Acting On

Ask what improvement would justify the work and the risk of changing a live listing. For many sellers that is somewhere between 5% and 15% relative. Pick a number and commit to it before you see any data.

Step 3: Check the Arithmetic

Use any sample-size calculator with your baseline, your MDE, 95% confidence and 80% power. Compare the sessions it asks for with what your ASIN can deliver in 10 weeks.

Two quick checks make step 1 easier. Your Search Query Performance report shows how shoppers move from impressions to clicks to purchases on your ASIN. A listing audit tells you whether the image is really your conversion problem, or whether price, reviews or content are.

If the required sessions fit, run the test. If they don't, you have a decision to make, and that is the useful part.

When Your ASIN Cannot Reach the MDE

You have four realistic options.

Test a Much Bolder Image

A bigger visual change creates a bigger real lift, and bigger lifts are easier to detect. Swapping a flat studio shot for a clearly different composition is a better use of a thin-traffic test than adjusting the background tint.

Move the Test to a Busier ASIN

If one SKU in the family gets triple the sessions, test there first. Then apply what you learn to the siblings, treating it as a strong hint rather than a guarantee.

Accept a Larger MDE

Be honest about it. Saying "we can only detect a 20% lift" is fine, as long as you also accept that smaller wins will stay invisible.

Skip the Live Test and Use Cheaper Evidence

Check that the image reads clearly at thumbnail size next to competitors, and ask real customers which one they prefer. This kind of evidence is directional, so treat it that way. It is far better than declaring a fake winner from a noisy A/B result.

There is one more thing to watch in Q4. Black Friday, Cyber Monday and holiday gifting all push conversion rates up and change who is browsing. A test that straddles a major promotion mixes the image effect with the season, so if your window overlaps one, expect messy results. Our Amazon peak season 2026 guide lists the dates that matter, and our post on what actually drives Amazon ranking in 2026 explains why click-through and conversion shifts like these ripple into visibility.

How to Read the Result Without Fooling Yourself

Decide your rules in advance and write them down. Here is a simple version:

Two Habits That Protect You

First, do not end a test early because it looks good on day nine. Early leads often shrink or reverse as data piles up.

Second, treat "inconclusive" as a real answer. It means the images performed too similarly for your traffic to separate them. That is information, and it is cheaper than a wrong decision.

If your images also carry product claims or specs, make sure those are accurate before you test them. Our guide on verifying supplier specs before publishing listings covers that process.

Quick Checklist Before You Launch

☐ Brand Registry is active and the ASIN is eligible in Manage Your Experiments. Check Amazon Brand Registry if you are unsure. ☐ Version B meets Amazon's main image rules, including a pure white background and no text overlays. SellerLabs lists the usual technical requirements. ☐ You changed one thing, not three. ☐ Your baseline and MDE are written down. ☐ The test window avoids major promotions and stock-outs. ☐ You have agreed the stop rule and will not peek and quit early.