A chi-square test is a statistical method researchers use to compare categorical data, such as survey responses or demographic groups, against what would be expected if no relationship existed. It works with counts rather than averages, which makes it a go-to tool for survey research, market research, and social science studies.
Unlike tests built around numeric averages, a chi-square test tells you whether the differences between your categories are meaningful or just noise. That distinction matters when you are deciding whether a campaign actually shifted preferences or whether device type genuinely affects survey completion.
In this blog, we’ll break down the types of chi-square tests, the formula behind them, and a step-by-step way to calculate and interpret one using real data.
What is a chi-square test?
A chi-square test measures whether observed frequencies in categorical data differ significantly from the frequencies expected under a null hypothesis, a starting assumption that no relationship exists between the variables being studied. Categorical data refers to information sorted into named groups, such as yes/no answers, regions, or product types, rather than measured on a numeric scale, and you can read more in this guide to categorical data.
Researchers run a chi-square test when they want to know if two categorical variables are connected, or if one variable’s distribution matches a known pattern. The test compares actual counts to calculated expected counts, then checks how large that gap is compared to what random chance would produce.
This differs from simply eyeballing percentages in a report. Two groups can look different on the surface and still fall well within normal sampling variation, which is exactly the gap a chi-square test is built to check.
This makes the chi-square test especially useful in survey and market research, where most responses are categorical rather than numeric. It also sits inside the broader idea of hypothesis testing, the general framework for deciding whether evidence supports or contradicts a starting assumption.
Chi-square test vs. t-test, ANOVA, and Fisher’s exact test
The chi-square test often gets confused with other statistical tests, especially by anyone searching for the fastest way to compare two groups. Each test below answers a different question, and picking the wrong one can produce an invalid conclusion.
| Test | Data type | What it compares | Typical use case |
|---|---|---|---|
| Chi-square test | Categorical | Observed vs. expected frequencies | Survey response patterns, demographic breakdowns |
| t-test | Continuous (numeric) | Means of two groups | Comparing average ratings or spend between two segments |
| ANOVA | Continuous (numeric) | Means of three or more groups | Comparing satisfaction scores across several segments |
| Fisher’s exact test | Categorical, small samples | Same relationship as chi-square, exact probability | 2×2 tables where expected cell counts fall below 5 |
The chi-square test and Fisher’s exact test are related but not interchangeable, even though some articles use the names loosely. Fisher’s exact test calculates an exact probability rather than an approximation, which makes it the more reliable choice once your sample gets small.
Types of chi-square test
Not every chi-square test asks the same question. Picking the right type depends on how many variables you are studying and whether you are comparing one sample or several.
- Goodness of fit test: Checks whether a single categorical variable matches an expected distribution, such as whether survey respondents are evenly split across four answer options.
- Chi-square test of independence: Checks whether two categorical variables within the same sample are related, such as whether region affects product preference. This is a common form of inferential statistics, since it draws conclusions about a population from sample data.
- Homogeneity test: Checks whether the distribution of one categorical variable stays consistent across two or more separate populations or groups.
The independence test and homogeneity test share the same formula and calculations. The difference is conceptual: independence asks about a relationship within one population, while homogeneity asks whether separate populations look the same on one variable.
Chi-square test formula and key terms
The chi-square formula compares what you actually observed to what you expected, then scales that difference by the expected value. The result is a single number called the chi-square statistic, shown below as one of the most widely used statistical methods for categorical data.
χ² = Σ (O − E)² / E
Where:
- O is the observed frequency in a category
- E is the expected frequency in that category, calculated under the null hypothesis
Degrees of freedom (df) represent the number of values in a calculation that are free to vary once the totals are fixed. For a chi-square test of independence, df equals (rows − 1) × (columns − 1). For a goodness-of-fit test, df equals the number of categories minus one.
The chi-square statistic and its degrees of freedom are then compared against a chi-square distribution to produce a p-value, the probability of seeing a difference this large if the null hypothesis were true. A small p-value, typically below 0.05, suggests the categories are genuinely related rather than different by chance.
How to calculate a chi-square test step by step
Calculating a chi-square test by hand follows the same sequence every time, whether you are testing one variable or two. Here is the process from start to finish.
- State the null and alternative hypotheses.
Define what “no relationship” looks like for your categories before you look at the data. - Choose a significance level.
Most researchers set alpha at 0.05, meaning a 5% risk of a false positive. - Check your assumptions.
Confirm your observations are independent, your sample is random, and most expected cell counts are at least 5. - Calculate the expected frequency for each cell.
Multiply the row total by the column total, then divide by the overall sample size. - Apply the chi-square formula.
Sum the squared difference between observed and expected values, divided by the expected value, across every cell. - Compare your result to a critical value or p-value.
If your chi-square statistic exceeds the critical value for your degrees of freedom, reject the null hypothesis.
Chi-square test example: Does device type affect survey completion?
A worked example makes the formula easier to apply. Here is a chi-square test of independence using survey completion data from 800 respondents.
| Completed | Abandoned | Total | |
|---|---|---|---|
| Mobile | 240 | 160 | 400 |
| Desktop | 300 | 100 | 400 |
| Total | 540 | 260 | 800 |
The expected count for each cell equals (row total × column total) divided by the overall total. For mobile respondents who completed the survey, that is (400 × 540) / 800 = 270. Repeating this across all four cells gives expected values of 270, 130, 270, and 130.
Applying the formula to each cell and summing the results gives a chi-square statistic of 20.51, with 1 degree of freedom: (2 rows − 1) × (2 columns − 1). At an alpha of 0.05, the critical value for 1 degree of freedom is 3.84, according to standard chi-square distribution tables published by the National Institute of Standards and Technology. Because 20.51 is far larger than 3.84, the result is statistically significant, and the null hypothesis of no relationship is rejected.
In plain terms, device type and survey completion are related in this sample. Mobile respondents abandoned the survey at a noticeably higher rate than expected, which points to a usability issue worth investigating on smaller screens.
When to use a chi-square test in survey and market research
Chi-square tests show up constantly in survey and market research because most survey questions produce categorical answers rather than numeric scores. A few common scenarios show where the test adds real value.
A retail brand might use a chi-square test to check whether product preference differs by age group, turning a simple cross-tab into a statistically defensible finding rather than a guess. CX teams often test whether NPS category, meaning promoter, passive, or detractor, varies by support channel, which can reveal where service breakdowns are concentrated.
Academic and social researchers commonly apply the test to demographic questions, checking whether opinions or behaviors differ across gender, region, or income bracket. In each case, the test turns a table of percentages into a defensible statement about whether a pattern is real.
Running these calculations by hand across large datasets is slow and easy to get wrong. QuestionPro’s Advanced Cross-Tabulation report calculates the chi-square statistic, p-value, and degrees of freedom automatically whenever you compare two survey questions, so researchers can test relationships without leaving their results dashboard.
Common mistakes to avoid when running a chi-square test
A chi-square test is simple to calculate, which makes it easy to misuse. These are the mistakes that show up most often in research reports and dashboards.
| Mistake | Why it matters |
|---|---|
| Running chi-square on numeric data without binning it | The test only works on categorical data, not raw averages or scores |
| Ignoring expected cell counts below 5 | Low expected counts make the test unreliable and inflate false positives |
| Treating a significant result as a strong relationship | Significance shows a relationship exists, not how strong it is; pair it with an effect size like Cramér’s V, a 0 to 1 measure of association strength |
| Testing non-random or dependent samples | The test assumes independent observations, so repeated responses from the same person can distort results |
Most of these mistakes are avoidable by checking assumptions before running the test, rather than after seeing a low p-value that looks convincing. It also helps to report the sample size, degrees of freedom, and p-value together, since a chi-square statistic on its own tells a reader very little without that context.
From raw counts to confident decisions
A chi-square test turns a table of percentages into a defensible answer about whether a pattern is real or coincidental. That distinction matters whenever a decision, a budget, or a product change rests on survey data.
The formula itself is straightforward once you know what observed and expected values represent. The harder part is checking assumptions and reading the result honestly, rather than stopping at whichever number happens to confirm what you already believed.
Businesses that run frequent studies often manage this kind of analysis inside a dedicated market research software platform, from data collection through statistical testing, instead of exporting everything to a separate stats tool.
Frequently Asked Questions (FAQs)
No. Both test relationships between categorical variables, but Fisher’s exact test calculates an exact probability and works better for small samples. The chi-square test relies on an approximation that needs larger expected cell counts to stay accurate.
Yes, but it treats ordinal categories as unordered, so it ignores the ranking between them. If the order of your categories matters, a trend test or ordinal regression captures more information than a standard chi-square test.
There is no fixed minimum, but most guidelines recommend an expected frequency of at least 5 in each cell of your table. Samples with several low expected counts should use Fisher’s exact test instead of the standard chi-square formula.
No, it only tells you whether a relationship is statistically significant. To measure strength, pair the test with an effect size such as Cramér’s V or the phi coefficient alongside the p-value you calculated.
Excel, SPSS, R, and Python all include built-in chi-square functions for quick calculations on any dataset. Many survey platforms with cross-tabulation reporting also calculate the statistic directly from response data, so you never have to build the formula by hand.
