Cookie banner A/B testing without dark patterns

Cookie banner A/B testing compares how visitors respond to fair changes in a consent prompt. Measure who makes a choice, keep Accept and Reject equally accessible, and judge results without steering people into consent.

How many visitors make no choice?

Many visitors leave a cookie banner without making a recorded choice. No single industry average applies to every site.

In a field study by Utz and colleagues, researchers observed 82,890 unique visitors to one German e-commerce website from late November 2018 to mid-March 2019. Among visitors without an ad blocker, 15.8% interacted with any notice. Bottom-left notices attracted the most interaction, at 37.1%, while top bars attracted the least. The authors also reported that 84.81% of visitors spent less than 10 seconds on the site.

Interaction and consent are different measures. In a privacy-by-default (opt-in) setting, fewer than 0.1% of visitors allowed cookies for all purposes. A visitor might open settings and leave, or depart before reading the banner. The short visits in that study make its results a poor substitute for your own baseline.

An ICO-commissioned Savanta survey of 2,316 UK adults in March 2024 asked about usual behavior. Of respondents, 48% said they normally accept cookies, 21% reject and 24% decide by website. Separately, 46% said they accept to dismiss the notice quickly, and 40% said they never read cookie settings. Those figures describe self-reported attitudes rather than observed choices.

Measure your own no-choice rate using a defined visit window. Count unique eligible visitors shown a banner, then count those who completed no choice before leaving. Divide that second count by the first and multiply by 100. Exclude visitors with an existing valid choice from both counts. Apply consistent rules for repeat visits, devices and traffic sources so your baseline stays comparable.

What may you test, and what is a dark pattern?

Fair tests change how people see and understand the banner while preserving an easy, informed choice. Dark patterns make refusal harder or present consent as the path visitors are supposed to take.

Fair to test	Do not test
Clear, neutral wording for each action	Misleading labels that pressure people to accept
Banner position with both choices visible	Hiding or removing Reject from the first layer
Layout and spacing with equally accessible buttons	Burying Reject in paragraph text or placing it outside the banner without adequate visual support
Accurate translations and understandable descriptions	Presenting refusal in language visitors cannot reasonably understand
Readable colors and button treatments	Making Reject text unreadable through extremely low contrast
When the notice appears, while optional trackers remain blocked before consent	Pre-ticking opt-in boxes or treating silence as consent
Clear explanations of optional cookies	Using legitimate interest as the basis for placing or reading cookies under Article 5(3)

The EDPB Cookie Banner Taskforce report, adopted January 17, 2023, sets out positions agreed as a minimum threshold among authorities. The taskforce did not prescribe a universal color standard: it assessed contrast case by case. It did regard reject text that virtually nobody could read as manifestly misleading. It also rejected legitimate interest as a legal basis for placing or reading cookies under Article 5(3).

The temptation to remove Reject is measurable. In a field experiment reported by Nouwens and colleagues, involving 40 participants, removing the opt-out button from the first page increased consent by 22 to 23 percentage points. That result explains why teams focused only on acceptance might consider the change. A test that makes refusal harder cannot fairly measure whether visitors prefer the revised message.

Why must Reject sit on the first layer?

Reject belongs on the first layer so refusing cookies takes no more effort than accepting them. Regulators in Europe and the UK have expressed that principle through distinct guidance and enforcement actions.

In its January 2023 report, the EDPB taskforce said a vast majority of participating authorities considered a consent button without a reject option on the same layer inconsistent with valid consent and an infringement. The report records common minimum positions rather than creating a separate binding design code. The taskforce also said withdrawing consent must be as easy as giving it.

In France, CNIL's May 2021 announcement of formal notices states that refusing cookies must be as simple as accepting them. In its January 2022 newsletter, CNIL reported fines of 150 million euros for Google and 60 million euros for Facebook. Its reason was that visitors to google.fr, youtube.com and facebook.com could not refuse cookies as easily as they could accept them.

In the UK, the ICO's consent guidance says refusal should be as easy as acceptance. Its good-practice example offers equally prominent Accept All and Reject All choices. For a test, give those controls comparable visibility, wording clarity and effort in both variants.

Ask a qualified lawyer to check the requirements that apply to your site and its visitors.

How do you run a fair test?

A fair cookie banner A/B test changes one presentation detail while preserving the same choices in both versions. Decide who enters the test, what counts as a decision and when the test ends before assigning visitors.

1. Check the existing setup. Before changing the banner, check Consent Mode first and verify its defaults and updates. If tracking behavior differs between variants, the results will be difficult to interpret. The free checker also flags cookies set before consent on the page.

2. Pick one variable. Compare two neutral headings, two banner positions or two readable layouts. Keep all other text, controls, timing and targeting unchanged. Write down what improvement you expect.

3. Keep Reject equal. Put Accept All and Reject All on the first layer in both variants. Make them equally prominent and easy to use. Preserve access to settings and an easy way to withdraw consent.

4. Randomize visitors. Assign each eligible visitor randomly to A or B, and keep that visitor's variant stable throughout the test. Use identical eligibility rules. Review any storage or identifiers used to maintain the assignment under applicable privacy rules.

5. Define the metrics in advance. Your main metric is the share of eligible banner viewers who complete any choice: accept, reject or save custom preferences. A second metric is acceptance among those who choose, calculated as Accept All choices divided by all completed choices. Track refusal and no-choice rates too, using the same denominator rules in both variants. Make sure the measurement method can account for visitors who do not consent.

6. Check the arithmetic. For example, if 1,000 eligible visitors saw variant A, 600 made a choice and 240 accepted all cookies, the choice rate would be 600 / 1,000 = 60%. Acceptance among those who chose would be 240 / 600 = 40%. The no-choice rate would be (1,000 - 600) / 1,000 = 40%. These numbers are illustrative, not observed results.

7. Run full weeks. Set a sample-size target and an end date before launch, based on your baseline choice rate and the smallest difference you want to detect. Use complete weeks to capture changes in weekday traffic. Do not stop early just because one variant is ahead; a temporary lead can disappear as more visitors are counted.

8. Record the test. Save both versions, randomization method, eligibility rules, start and end dates, outcome definitions, sample counts and exclusions. Compare the choice rate and acceptance among choosers together. Check that any expected improvement did not come from making Reject less visible.

CookieConsentHQ offers consent-rate A/B testing on Growth and above. Use the test to choose clearer presentation while keeping refusal straightforward.