A/B testing

A/B testing is a controlled comparison between an existing version and an alternative, ideally assigning visitors randomly and consistently to a version.[3][2] It lets a merchant investigate whether a change such as a different product presentation affects shopping behaviour without visitors selecting their own treatment. Required sample size depends on factors including the change to be detected, variability and acceptable error risks.[1]

A size-guide experiment may answer a more useful question than an arbitrary colour change: identify the customer problem and the outcome that would demonstrate improvement before starting. Consider returns and margin alongside purchases. Agree on an analysis method and stopping rule in advance rather than declaring a winner when an interim result happens to look favourable.