{"id":1108660,"date":"2024-04-24T22:36:00","date_gmt":"2024-04-25T05:36:00","guid":{"rendered":"https:\/\/www.questionpro.com\/blog\/?p=1108660"},"modified":"2026-08-27T23:00:31","modified_gmt":"2026-08-28T06:00:31","slug":"propensity-score-matching","status":"publish","type":"post","link":"https:\/\/www.questionpro.com\/blog\/propensity-score-matching\/","title":{"rendered":"Propensity Score Matching: A Practical Guide for Researchers"},"content":{"rendered":"\n

Randomized experiments are not always possible. Ethics, cost, or timing can rule them out, leaving researchers to work with observational data instead. Propensity score matching is one of the most established ways to make treatment and comparison groups fair to compare when that happens.<\/p>\n\n\n\n

At its core, the method estimates each participant’s probability of receiving a treatment based on their observed traits, then pairs people with similar probabilities across groups. This probability is the propensity score itself, and it forms the backbone of the wider family of propensity score methods used across medicine, economics, and social research.<\/p>\n\n\n\n

In this blog, we’ll explore what propensity score matching is, how to build and evaluate a propensity score model, and the mistakes that undermine even well-designed studies.<\/p>\n\n\n\n\n\n

What is propensity score matching?<\/strong><\/h2>\n\n\n\n

Propensity score matching is a statistical technique that pairs participants from a treatment group with participants from a control group who share a similar likelihood of having received that treatment, based on their measured characteristics.<\/p>\n\n\n\n

That likelihood, the propensity score, was first formalized by statisticians Rosenbaum and Rubin in 1983 as a way to design non-randomized studies that behave more like randomized ones. The idea is straightforward: if two people had almost the same odds of ending up in the treatment group, but one did and one didn’t, comparing their outcomes gets closer to isolating the treatment’s true effect.<\/p>\n\n\n\n

This matters most when random assignment isn’t available and selection bias<\/a> is likely, meaning the people who received a treatment differ systematically from those who didn’t. Without correcting for that, an analyst risks confusing a confounding variable’s influence with the treatment’s actual effect. Propensity score methods are widely used in causal research, particularly in cardiovascular medicine, education policy, and program evaluation, where running a true experiment would be impractical or unethical.<\/p>\n\n\n\n

Propensity score matching vs. other bias-reduction methods<\/strong><\/h2>\n\n\n\n

People often use “propensity score matching” as a catch-all term, but matching is only one of four ways to apply a propensity score once it’s estimated. Confusing the four leads to picking a method that doesn’t fit the data.<\/p>\n\n\n\n

The table below outlines how each approach works and when it tends to fit best.<\/p>\n\n\n\n

\n \n \n \n \n \n \n \n \n
Method<\/th>\n How it works<\/th>\n Best used when<\/th>\n <\/tr>\n <\/thead>\n
Matching<\/td>\n Pairs each treated participant with an untreated participant who has a nearly identical propensity score<\/td>\n Sample size is moderate to large and close matches are available<\/td>\n <\/tr>\n
Stratification<\/td>\n Groups participants into blocks based on score ranges, then compares within each block<\/td>\n Researchers want a simpler, more transparent grouping method<\/td>\n <\/tr>\n
Inverse probability weighting<\/td>\n Weights each participant by the inverse of their probability of receiving the treatment they actually got<\/td>\n The full sample should be retained rather than discarded through matching<\/td>\n <\/tr>\n
Covariate adjustment<\/td>\n Adds the propensity score as a control variable inside a regression model<\/td>\n Researchers want to combine propensity scores with other statistical controls<\/td>\n <\/tr>\n <\/tbody>\n <\/table>\n<\/div>\n\n\n\n

Propensity score matching also gets confused with quasi-experimental research<\/a> design more broadly. Quasi-experimental design is the umbrella category for studies that compare groups without full randomization. Propensity score matching is one specific tool researchers use inside that broader design to make the comparison more credible.<\/p>\n\n\n\n

Real-world examples of propensity score matching<\/strong><\/h2>\n\n\n\n

Propensity score matching shows up anywhere researchers need to compare groups that weren’t randomly assigned. A few examples illustrate the range.<\/p>\n\n\n\n

Cardiovascular researchers have used propensity score matching to compare patients who received a particular medication against those who didn’t, since randomly withholding a potentially beneficial treatment from sick patients would be unethical. In education research, analysts have matched students who participated in an intervention program with similar students who didn’t, to estimate the program’s effect on outcomes like graduation rates.<\/p>\n\n\n\n

Market and customer research teams increasingly apply the same logic outside clinical settings. If a company rolled out a new onboarding flow to some customers but not others based on self-selection rather than random assignment, propensity score matching can help estimate the flow’s real effect on retention, controlling for the traits that made some customers more likely to opt in.<\/p>\n\n\n\n

How to build a propensity score model<\/strong><\/h2>\n\n\n\n

Building a usable propensity score model comes down to a handful of deliberate steps. Rushing variable selection is where most models go wrong.<\/p>\n\n\n\n

    \n
  1. List potential confounders.<\/strong> Gather the baseline characteristics that plausibly affect both who received the treatment and the outcome being studied.<\/li>\n\n\n\n
  2. Run a logistic or probit regression.<\/strong> Set treatment status as the outcome variable and the confounders as predictors. Logistic regression is a statistical method that estimates the probability of a binary outcome, in this case, treatment or no treatment.<\/li>\n\n\n\n
  3. Generate the predicted probability for each participant.<\/strong> That output is the propensity score.<\/li>\n\n\n\n
  4. Exclude variables that were affected by the treatment itself.<\/strong> Including a treatment-affected variable hides part of the treatment’s real impact instead of removing bias.<\/li>\n\n\n\n
  5. Drop covariates that perfectly predict treatment status.<\/strong> If a variable makes treatment assignment fully predictable, there’s no overlap left to compare, and the variable should be removed from the model.<\/li>\n<\/ol>\n\n\n\n

    Covariate selection is a balancing act. A variable tied to the outcome but not to treatment assignment generally reduces bias when included. A variable tied only to treatment assignment, with no real link to the outcome, mostly adds noise without improving the estimate.<\/p>\n\n\n\n

    How to evaluate and balance a propensity score<\/strong><\/h2>\n\n\n\n

    A propensity score is only useful if it actually balances the groups it was built to compare. Skipping this evaluation step is one of the most common reasons published results don’t replicate.<\/p>\n\n\n\n

    Start by checking common support, the range of propensity scores where both treated and untreated participants actually exist. A treated participant with no comparable untreated counterpart cannot be matched, and their outcome cannot be attributed to the treatment with confidence. Analysts typically judge this visually, by plotting the distribution of scores across both groups and looking for meaningful overlap.<\/p>\n\n\n\n

    Next, compare standardized mean differences for each covariate between the two groups. This is a common way to check covariate balance, since balance in the average value doesn’t guarantee balance across the full distribution. A large imbalance signals that the propensity score model needs to be respecified, often by adding or adjusting variables.<\/p>\n\n\n\n

    If the sample is split into strata or blocks, check covariate balance again within each block, not just across the whole sample. According to a widely cited propensity score guide published through the National Library of Medicine<\/a>, using around five strata typically removes most of the bias associated with a single confounder, while larger samples can support ten to twenty strata for finer balance.<\/p>\n\n\n\n

    One caution worth repeating: don’t lean on measures like the c-statistic or AUC to judge how well a propensity score model is performing. Those metrics are built to predict outcomes, not to confirm that confounding has been reduced, so a high score there says little about whether the groups are actually balanced.<\/p>\n\n\n\n

    Matching and weighting strategies: how to choose<\/strong><\/h2>\n\n\n\n

    Once the propensity score is estimated and reasonably balanced, the next decision is how to compare the groups. Matching and weighting are the two most common choices, and each comes with trade-offs.<\/p>\n\n\n\n

    Matching pros:<\/strong><\/p>\n\n\n\n