How to plan marketing creative tests that teach you something

A marketing creative test compares different ways of presenting an offer so you can make a better decision about the next campaign. Start with a hypothesis, choose a meaningful difference between the versions and agree how the result will be judged. Producing more assets is useful only when the variations help you learn something.

The first decision is what you are testing. A different promise, a different proof point and a different visual format are separate ideas. Changing all three can help compare complete concepts, but it does not tell you which ingredient caused the difference. Write down which kind of decision the test is intended to support.

1. Start with a customer question

Use what customers ask, misunderstand or hesitate about. For a hypothetical workflow product, one message might explain how setup works while another demonstrates the task the product helps complete. The test is about which explanation better supports a useful next action for that audience. It is not simply a competition between two colours.

Keep the hypothesis specific enough to guide the creative. “People do not realise this works with their existing process” suggests a demonstration. “They understand the product but do not trust the claim” suggests relevant evidence. A generic request for something more engaging leaves the designer guessing about the job the asset needs to do.

2. Build a small, clear test plan

• Audience: who will see the creative and in what context?
• Question: what do we want to learn about their response?
• Variants: what changes, and what remains consistent?
• Destination: where does each version send people?
• Outcome: which action matters, and how is it recorded?
• Decision: what would make us keep, revise or stop the direction?

Name the versions in a way that preserves this information. A folder full of “final” files makes it difficult to connect a result to the message, format and destination that produced it. Keep the production brief, asset and result together so someone else can understand the reasoning later.

3. How should you compare the versions?

Where a platform supports an appropriate controlled experiment, use it to make the comparison more interpretable. Google Ads describes experiments in terms of control and treatment groups receiving different traffic. Ordinary delivery across several ads is not automatically an equal or random comparison; the platform may allocate exposure differently.

Keep the question narrow enough that the campaign can produce useful evidence. Avoid splitting a small audience across so many variations that none receives enough activity to judge. There is no universal number of days or conversions that makes every test reliable. The required evidence depends on baseline behaviour, the difference you need to detect and the way the test is run.

4. What should you do with an apparent winner?

Check the business outcome before declaring victory. A striking image can earn attention without bringing suitable buyers. A quieter creative may attract fewer clicks but better conversations. Keep the meaning of the conversion event visible in the report, and distinguish a platform-attributed action from a confirmed customer outcome.

If the evidence is weak or the versions received very different exposure, record the result as inconclusive. That is still useful if it tells you how to improve the next test. Do not turn a small early difference into a permanent creative rule or a claim that the same idea will work for every audience.

5. How do you turn learning into a creative system?

Keep the messages and proof points that have earned a place, along with the context in which they worked. Build reusable layouts for practical production, but continue testing the underlying idea. A system should make it easier to create coherent variants without removing the judgement needed to decide what to say.

This is where Refract’s brand, design and marketing work connect. The brand gives the work a consistent voice and appearance; the campaign gives each asset a job; the results help decide what to make next. Bring the offer, the existing creative and what you currently measure. We can work from that to a more deliberate production and testing plan.

6. Questions

Should we test a whole concept or one small change?

Either can be useful. A concept test compares complete directions. A narrower test helps investigate a particular variable. State which decision you need to make so you do not overinterpret the result.

Is the ad with the highest click-through rate the winner?

Only if attention is the agreed outcome. For lead or sales campaigns, also examine the quality and downstream actions associated with those visits. A click is not a customer.

How often should we replace creative?

Review the evidence and production context rather than following an arbitrary calendar. Falling performance may reflect the audience, offer, destination or delivery as well as the creative itself.