It occurred to me recently that I frequently write and speak about the concept of testing, but have not really shown the steps of how to create random groups. Establishing random groups is an essential component to an effective AB testing or multivariate testing strategy. Assigning test subjects, in this case, potential survey respondents, to groups in a random fashion minimizes the potential for bias in the test.
Why is this important? The scientific method relies on the ability to control for the variance. Randomization is the best tool we have for minimizing the effect of factors that are outside of our control. Employing randomization in AB testing allows us to keep our focus on the test elements and not worry about whether or not a survey participant has lost their job, as an example of an extraneous factor.
In creating test groups I employ both MS Excel and SPSS, although the work could be done entirely in SPSS or any other survey data analysis package. ‘Order’ is one of the main threats to creating valid test groups. Often we do not know how, or if a list is ordered. Examples might be alphabetical ordering by last name or account number, newest customers’ first, geographic region, etc. In this case creating a bit of randomness is a good thing. If your list is in Excel add a column and use the random function. This creates a random number for each record. This random number can then be sorted, either ascending or descending. If there was order present in the initial list it will now be randomized out.
This step is essential in ensuring that members of your survey group are allocated to test cells in a random fashion. SPSS can then be used to create randomly selected test groups. You as the survey author can specify group size as a fixed number or percentage of the total database.
How do we know if we are successful in or ambition to create randomness? If your randomization was done properly each test cell will mirror the entire database. This can be tested via crosstabulations or analysis of variance for metric variables. If the chi-square values for each crosstab (or f statistics for ANOVA tests) are not significant then we have succeeded in randomly allocating prospective respondents into our test cells.
AB testing (which involves two groups) or multivariate testing (3+ test groups) is critical to the process of effective allocation of marketing spend. With resources forever being stretched having the ability to estimate response to marketing programs helps us to reach that level of balanced allocation.
For more insight into the world of data analysis check out this Question Pro blog post – Data Analysis Simple and Complex or this article Dynamic Data Analysis
