Conversion Rate Optimization
A/B testing programme
This is the narrowest useful unit of Conversion Rate Optimization: run tests that produce decisions rather than ambiguity.
What happens in an A/B testing programme?
Run tests that produce decisions rather than ambiguity. In practice that means hypothesis formation tied to research findings and a stated expected mechanism, and then the harder half: sample size and duration calculated before the test starts.
Why this work exists
Left alone, this does not fail loudly. It degrades, and the first sign is a number moving the wrong way three months later.
Part of Conversion Rate Optimization, and the part where the diagnosis is usually most useful on its own.
The nearest piece of work to it is making the price easy to find because hiding it costs more than showing it. Where that is already handled, this moves faster than the estimate; where it is not, the two get done together or neither of them holds.
What good looks like
It is healthy when the people who disagreed with it still use it.
The gap between that and where most accounts start is why this is scoped as A/B testing programme rather than as a line inside something larger. Finding the step where people leave and asking them why is the usual second step once it is in place.
When this is the right work
It is worth doing when:
- Tests are stopped when the result looks good
- Nobody calculated whether the traffic could detect the effect
- The testing tool adds a visible flash of the original
If none of those is true, something else is probably the constraint, and the Conversion Rate Optimization overview maps the alternatives. The usual neighbours are testing the thing with enough traffic to produce an answer and removing a reassurance that was never reassuring anyone.
What the work covers
What the work actually involves:
- Hypothesis formation tied to research findings and a stated expected mechanism. This is the piece a client can verify themselves, which matters more than it sounds.
- Sample size and duration calculated before the test starts, usually the first thing to show a result, so it goes early.
- Implementation that does not itself slow the page or flicker. Straightforward in principle; the detail is where it stops being quick.
- Analysis including segment-level results and a documented decision, handled before the parts that depend on it rather than alongside them.
How it runs
The shape of the work, stage by stage.
Setting the review rhythm
How often this gets looked at again, and by whom, is agreed now rather than left to whoever notices first.
Costing the options
Where there is more than one route, each gets a rough cost and a rough timeline, so the choice is made on trade-offs rather than on whichever was described most confidently.
Removing before adding
The first pass takes things away. Adding to a setup nobody has pruned in two years makes the next person’s job harder, not easier.
What each part runs into
The specific resistance each part meets:
- Hypothesis formation tied to research findings and a stated expected mechanism – in Singapore, complicated by the fact that a competitive set that changes street by street
- Sample size and duration calculated before the test starts, which in Delhi NCR runs into a decision made by somebody who never visits the site
- Implementation that does not itself slow the page or flicker: the Bengaluru version of this has to deal with a procurement step nobody accounted for
- Analysis including segment-level results and a documented decision, where a Surat engagement contends with a measurement baseline that was never taken
The trap
Stopping a test early on a promising result. Peeking repeatedly at an uncontrolled significance threshold produces false positives at a rate that makes the programme worse than useless.
It is a cheap mistake to avoid and an expensive one to discover six months in.
How it is measured
Conclusive test rate, validated uplift, and cumulative effect on conversion. That is the number the work is judged on, reported alongside whatever the business already tracks rather than instead of it.
Movement smaller than the normal week-to-week swing gets reported as no movement.
What you end up with
What you can point at afterwards is a running programme with pre-registered hypotheses, correct sizing, and documented decisions either way.
Versioned, so the next pass starts from this one rather than from scratch.
How it differs by sector
How the sector changes it:
- SaaS and technology, constrained by the fact that product-led companies where the buyer self-educates and the trial is the sales call.
- Home and trade services: Emergency intent, service-area geography, and a phone that has to be answered. so this work has to account for that before anything else.
- Travel and hospitality, where aggregators hold the demand, so direct booking depends on being the specific named property. which moves this work earlier in the plan.
The industry pages carry the rest of that detail. SaaS and technology and Home and trade services are the two where this piece of work most often decides the outcome.
Common questions
How long should an A/B test run?
Until the pre-calculated sample size is reached, and at least one full business cycle so weekday and weekend behaviour are both represented. Stopping when the result looks good is the single most common way testing programmes generate false conclusions, because results fluctuate most when samples are small.
What traffic volume do you need to test?
Enough that the smallest effect worth acting on is detectable, which depends on baseline conversion rate and the effect size. Low-traffic sites can still test, but only for large changes; attempting to detect a two percent improvement on a hundred conversions a month is not a test, it is a wait.
What do you need from us?
Access to the measurement and to whoever knows the history. Most of the delay on work like this comes from waiting for access rather than from the work itself. Start at GEO or SEO if nothing has been built yet, Google Ads or Meta Ads if something has and it is not working.
Is the output a report or a change?
Both, in that order. A running programme with pre-registered hypotheses, correct sizing, and documented decisions either way. The report exists so the change is accountable, not instead of it. Two of these come up often enough to be worth reading first: How do we measure SEO for a long sales cycle? and Do we need separate pages for every city we serve?.
How do we know this is the right piece of work?
The signals above are the test. If none of them describes your situation, something else is the constraint, and saying so is more useful than selling this. Read Review schema for the standard approach and sameAs for when the standard approach does not fit.
Ask about a/b testing programme
Describe the gap between what the site does and what it should. We will name the likely cause.
Begin with the diagnosis, not the pitch
One call, no sequence of follow-up emails afterwards. If the timing is wrong we will leave you to it.