The Delphi method is a structured forecasting and consensus technique in which a panel of experts answers anonymous questionnaires over several rounds. After each round, a facilitator shares a summary of the group’s answers. Experts then revise their views until opinions converge or stabilize.
Teams turn to it when the data they need does not exist yet. Think of questions like “Which skills will our workforce need by 2030?” A public survey cannot answer that, and an expert meeting often ends with the loudest voice winning.
In this guide, we break down how the Delphi technique works, how to measure consensus, real examples, the main variants, and the mistakes that weaken a Delphi study.
What is the Delphi method?
The Delphi method is a group research technique that turns individual expert judgment into a reliable collective answer, without putting the experts in the same room. People also call it the Delphi technique or a Delphi study.
The method grew out of Project RAND, a US Air Force research program that later became the RAND Corporation. In the 1950s, Olaf Helmer, Norman Dalkey, and Nicholas Rescher developed it to forecast how technology would affect warfare. A 1964 report by Theodore Gordon and Olaf Helmer then applied it to long-range trends in science and technology.
From there, the approach spread to business strategy, public policy, education, and health care. In health sciences, it now helps teams set research priorities and develop clinical guidelines.
It sits alongside interviews and focus groups among qualitative research methods that rely on expert insight rather than large samples. What sets it apart is the structure: anonymous answers, repeated rounds, and feedback in between.
What are the key features of the Delphi technique?
Four features separate the Delphi technique from an ordinary expert survey or meeting. Remove any one of them and you no longer have a true Delphi study.
- Anonymity: Panelists do not know who said what, and often do not know who else is on the panel. This removes the pull of seniority, reputation, and dominant personalities.
- Iteration: Experts answer the same or refined questions over at least two rounds. Each round gives them a chance to reconsider.
- Controlled feedback: Between rounds, the facilitator shares a summary of group responses and the reasons behind them. Panelists react to arguments, not to people.
- Statistical group response: The final answer appears as a group statistic, such as a median, a percent agreement, or a spread of ratings. Minority views stay visible in the report.
Many practitioners add a fifth trait: a heterogeneous panel. Experts from different backgrounds and regions challenge each other’s assumptions, which makes the final consensus more robust.
How does the Delphi method work? Delphi method steps
The Delphi method works through a planned sequence: frame the question, recruit experts, run two or three anonymous rounds with feedback, and stop at a pre-set rule. Most of the study’s quality depends on decisions made before the first questionnaire goes out.
- Appoint a neutral facilitator.
The facilitator designs questionnaires, analyzes answers, writes feedback summaries, and keeps the panel engaged. They should have no stake in the outcome. - Define the objective and scope.
Write one clear research question, such as “Which factors will most affect US nurse retention through 2030?” A narrow scope keeps rounds short and answers comparable. - Select the expert panel.
Use purposive sampling to recruit people with direct, relevant expertise. Write down your inclusion criteria, such as years of experience, role, or publications, and aim for a mix of perspectives. - Set the consensus and stopping rules.
Decide before Round 1 what counts as agreement and when the study ends. For example: “Consensus means 75% of panelists rate an item 7 to 9 on a 9-point scale. We stop after three rounds.” - Run the rounds.
Send each questionnaire, analyze the responses, and share anonymized feedback before the next round. The table below shows what each round contains. - Analyze and report the results.
Report the items that reached consensus, the items that did not, and how ratings shifted across rounds. Then turn the findings into a forecast, guideline, priority list, or decision.
What happens in each Delphi round?
Each round has a distinct job. Round 1 gathers ideas, Round 2 rates them, and Round 3 checks whether experts hold their positions after seeing the group view.
| Round | Purpose | Typical question format | Feedback panelists get next |
|---|---|---|---|
| Round 1 | Generate ideas and arguments | Open-ended questions, such as “What barriers will slow AI adoption in US community banks?” | A clean, deduplicated list of all ideas |
| Round 2 | Rate or rank each idea | Rating scales, such as a 1 to 9 importance scale or a 5-point Likert scale | Group median, spread, and their own previous rating |
| Round 3 | Reconsider and confirm | The same rating items, with group results shown | Final consensus list and remaining disagreements |
| Optional Round 4 | Resolve items that are still split | Targeted questions, plus a request to explain outlier ratings | Final report |
Many teams use a modified Round 1, where the facilitator starts from a literature review instead of open questions. This saves a round, but it limits the ideas to what the facilitator already found.
How do you measure consensus in a Delphi study?
You measure consensus in a Delphi study with a pre-defined statistic, usually percent agreement, often combined with the median rating and the interquartile range. The key word is pre-defined: set the rule before any data arrive.
In practice, many studies skip this step. A systematic review of 100 Delphi studies found that 98 said they assessed consensus, but only 72 actually defined it. Percent agreement was the most common definition, with a median threshold of 75%.
Four measures do most of the work, usually in combination:
- Percent agreement: The share of panelists who rate an item in the agreement range. A common bar is 70% to 80% of the panel.
- Median: The central rating of the group. A typical rule is a median of 7 or higher on a 9-point scale.
- Interquartile range (IQR): How tightly the middle 50% of ratings cluster. An IQR of 1 or less on a 4- or 5-point scale signals close agreement.
- Stability: Whether ratings change between rounds. Little or no shift across two consecutive rounds means more rounds will not help.
These rules of thumb come from a narrative review of Delphi use in health sciences. It adds that high-stakes guidelines, such as end-of-life care, may justify a stricter bar of 90% to 100%. The IQR is simply the gap between the 25th and 75th percentile ratings, so a small IQR means experts agree closely.
How many experts and rounds does a Delphi study need?
No single panel size is correct, but most Delphi studies use between 8 and 20 experts and run two to three rounds. Treat these benchmarks from the same health sciences review as starting points:
- Panel size: Published panels range from as few as 4 experts to several thousand, but most sit between 8 and 20. The review suggests 8 to 23 participants as a practical target.
- Small-panel risk: In a 10-person panel, one expert equals 10% of the vote. A single dropout can push an item across your threshold.
- Rounds: Classic Delphi studies typically run 2 to 3 rounds, and many authors consider 3 ideal.
- Stopping rule: Most studies stop after a fixed number of rounds. In the systematic review above, 70 studies stopped after a set number of rounds rather than at consensus.
- Attrition: Dropout across rounds ranged from 0% to 92% in classic Delphi studies, so recruit more experts than you need.
These numbers also resolve a common confusion. A Delphi panel does not need hundreds of people, because it measures expert judgment, not population opinion.
Delphi method examples
Delphi method examples span defense, health, business, HR, and education. The table shows how the same process adapts to very different questions.
| Field | Example question | Typical panel | Output |
|---|---|---|---|
| Technology forecasting | Which scientific breakthroughs are likely over the coming decades? (the 1964 RAND study) | Scientists and engineers | Timeline of expected breakthroughs |
| Health care | Which criteria show a patient is ready for discharge after knee surgery? | Surgeons, nurses, physical therapists | Consensus checklist or clinical guideline |
| Public health | Which measures matter most during a fast-moving outbreak such as COVID-19? | Epidemiologists, clinicians, public health officials | Rapid priorities, often through a fast-track Delphi |
| HR and workforce planning | Which skills will our engineering team need by 2030? | Engineering leads, HR partners, outside advisors | Ranked skills list for hiring and training |
| Market research | How fast will US small businesses adopt a new payment technology? | Industry analysts, bankers, product leaders | Adoption forecast with ranges |
| Education | Which competencies should a new data science curriculum cover? | Faculty, employers, alumni | Prioritized competency framework |
Worked example: A Delphi study for a product roadmap
Here is how a B2B software company might run a three-round Delphi study. The numbers are illustrative, but the flow mirrors a typical study.
- Round 1 collects ideas.
The facilitator invites 15 customers, sales engineers, and support leads. Each answers one open question: “What capabilities will mid-market buyers expect from our product in two years?” The panel returns 64 ideas, which the facilitator merges into 22 distinct items. - Round 2 rates the list.
Panelists rate each item on a 1 to 9 importance scale. Nine items meet the pre-set rule of at least 75% rating 7 or higher. - Round 3 tests the result.
Panelists see the group median and their own earlier rating for the 13 unresolved items. Four more reach consensus, while two stay split between customers and sales engineers. - The team acts on the output.
Thirteen consensus items go into roadmap planning. The two split items become follow-up customer research rather than forced decisions.
Where is the Delphi method used?
Organizations apply the Delphi method wherever decisions depend on expert judgment about an uncertain future or an ill-defined problem. These are the most common applications:
- Strategic decision-making: Leaders test long-term bets, such as market entry or new business models, against independent expert views.
- Forecasting and trend analysis: Teams estimate when technologies, regulations, or consumer behaviors will shift.
- Market analysis: Researchers estimate demand for products that have no sales history yet.
- Policy development: Government agencies and nonprofits weigh policy options and their likely consequences.
- Risk assessment: Project and risk managers identify and rank risks that historical data cannot reveal.
- Health research: Clinicians build guidelines, define outcome sets, and set research priorities.
- Education: Faculty and employers agree on competencies and curriculum content.
- Environmental planning: Scientists and planners assess climate, land-use, and resource scenarios.
- Disaster preparedness: Emergency managers rank response capabilities before a crisis hits.
- Software estimation: Engineering teams use Wideband Delphi to estimate project effort.
What are the advantages and disadvantages of the Delphi method?
The main advantage of the Delphi method is independent, bias-resistant expert judgment. Its main disadvantage is time, so weigh both before you commit to a multi-round study.
Advantages
The benefits all trace back to anonymity and iteration.
- Less groupthink: Anonymity stops senior or vocal experts from steering the group.
- Remote by design: Experts in different cities or countries can join without travel.
- Room to rethink: Panelists can change their minds without losing face.
- Broad input: You can include more experts than any meeting room could seat.
- Transparent output: Group statistics show both where experts agree and where they do not.
- Buy-in: Experts who shaped a result are more likely to support putting it into practice.
Disadvantages
Most drawbacks come from the same multi-round structure that makes the method rigorous.
- Slow: Several rounds of questionnaires, analysis, and feedback take weeks or months.
- Dropout: Busy experts leave between rounds, which can distort the results.
- Facilitator influence: Whoever writes the summaries can frame the feedback, intentionally or not.
- False consensus: Agreement does not prove a correct answer. Panelists may converge on a shared blind spot.
- Selection bias: A panel that is too similar will agree quickly and miss risks.
- Cost: Recruiting and compensating qualified experts adds up.
How does the Delphi method compare with other research methods?
The Delphi method differs from other group methods mainly in its anonymity and iteration. Use this comparison to pick the right tool for your question.
| Method | How it works | Interaction between participants | Best for |
|---|---|---|---|
| Delphi method | Anonymous questionnaires over several rounds, with feedback | None | Consensus or forecasts from dispersed experts |
| Focus group | A moderated live discussion with a small group | Open, in person or as an online focus group | Exploring attitudes, language, and reactions |
| Survey | A one-time questionnaire sent to a large sample | None | Measuring how common a view is in a population |
| Expert panel meeting | Experts discuss and decide together | Open and identified | Fast decisions when hierarchy is not a concern |
| Brainstorming | Free-flowing idea generation in a group | Open and spontaneous | Generating many ideas quickly |
| Nominal group technique | Silent idea writing, round-robin sharing, then private voting in one session | Structured and in person | Prioritizing options within a few hours |
What is the difference between the Delphi method and the nominal group technique?
Both methods structure group judgment, but they trade speed against anonymity. The nominal group technique (NGT) brings participants together for one facilitated session. Each person writes ideas silently, shares them in turn, discusses them briefly, and then votes privately.
A nominal group session typically takes less than two hours per question. A Delphi study keeps experts apart and anonymous for weeks or months instead.
Choose NGT when you can gather people in one room and need an answer quickly. Choose Delphi when experts are dispersed, when status differences could silence some voices, or when you want time for considered judgment. The RAND/UCLA Appropriateness Method combines both, pairing Delphi-style rating rounds with a group discussion.
What are the main types of Delphi method?
The main types are the classic, modified, policy, e-Delphi, real-time, and Wideband Delphi. All of them keep anonymity and feedback, but they differ in their starting point, format, and goal.
- Classic Delphi: Starts with open-ended questions and runs anonymous rating rounds toward consensus.
- Modified Delphi method: Starts from a literature review or interviews, and may add a meeting. Designs vary widely from study to study.
- Policy Delphi: Aims to surface the full range of opposing views on a policy issue, not to force agreement.
- E-Delphi: Runs every round through online survey software, which speeds up distribution and analysis.
- Real-time Delphi: Replaces rounds with a continuous online process, where experts see group results update as they answer.
- Wideband Delphi: Adds short discussions between estimation rounds and suits software effort estimates.
When should you use the Delphi method?
Use the Delphi method when the question needs expert judgment, the experts are hard to gather, and you have weeks rather than days. If any of those conditions fail, another method usually works better.
Use the Delphi method when:
- Data is missing: Little historical data or published evidence exists on the question.
- Experts are dispersed: Panelists work in different organizations, cities, or time zones.
- Hierarchy could bias results: Junior experts might defer to senior ones in a meeting.
- You need defensible consensus: A guideline, standard, or forecast must show how the group reached agreement.
Choose another method when:
- You need population data: A representative survey measures what customers or employees think.
- You need an answer this week: A nominal group session or a structured workshop is faster.
- You want to explore feelings and language: Focus groups and interviews capture nuance better.
- The answer is already known: A literature review costs far less than convening experts.
What are common mistakes in a Delphi study?
Most weak Delphi studies fail in the design, not the analysis. Avoid these six mistakes.
- Defining consensus after the data arrive.
Setting the threshold late lets anyone tune the result. Write the rule into your protocol before Round 1. - Recruiting a narrow panel.
A panel from one profession will agree fast and miss blind spots. Mix roles, sectors, and regions. - Asking too many items.
Long rating lists cause survey fatigue and dropout. Cut the list to items that matter for the decision. - Writing biased feedback.
Summaries that highlight one side nudge panelists toward it. Report medians and spreads neutrally, and include minority arguments. - Ignoring attrition.
If a third of the panel leaves, the remaining “consensus” may reflect only the most committed experts. Report dropout for every round. - Treating consensus as truth.
Agreement shows shared judgment, not proof. Check high-stakes findings against real data once it becomes available.
Is the Delphi method qualitative or quantitative?
The Delphi method combines both: Round 1 is usually qualitative, and later rounds are quantitative. That mix is why researchers often describe it as a consensus method rather than placing it in one camp.
Open-ended answers in the first round produce qualitative data that the facilitator codes into themes. Rating rounds then produce numbers, such as medians and agreement percentages.
However, the method does not rely on statistical sampling. You choose a Delphi panel for expertise, so its results describe expert judgment, not a population. For that reason, many textbooks still group it with qualitative methods, even though its output looks statistical.
How can QuestionPro support a Delphi study?
QuestionPro supports a Delphi study by handling the parts that slow facilitators down: anonymous distribution, round-by-round questionnaires, reminders, and analysis. Here is how each capability maps to the process.
- Anonymous responses: Respondent Anonymity Assurance keeps email addresses and identifiers out of reports, while still showing who has responded. You can follow up with non-responders without exposing anyone’s answers.
- Round-specific questionnaires: Build Round 1 with open-ended questions, then copy the survey and swap in rating scales for Rounds 2 and 3.
- Feedback blocks: Add the group median and summary arguments at the top of each new round, so panelists see the group view before they rate again.
- Survey logic: Show a follow-up question only to panelists whose ratings fall outside the group range, and ask them to explain their position.
- Text AI: Group hundreds of Round 1 comments into themes faster, then review and merge the items by hand.
- Smart Dashboards: Track medians, agreement percentages, and response rates per round, then export the data for IQR calculations.
Teams running Delphi studies for academic, health, or market research projects can manage every round in QuestionPro Market Research Software.
Getting reliable answers from the Delphi method
The Delphi method remains one of the few research tools built for questions that data cannot answer yet. Its strength comes from discipline: a sharp question, a varied panel, a consensus rule written in advance, and honest feedback between rounds.
Skip any of those and you get a slow survey with a confident-looking result. Get them right, and a small group of experts can produce a forecast, guideline, or priority list that holds up to scrutiny.
Frequently Asked Questions (FAQs)
The name refers to the Oracle of Delphi, the ancient Greek shrine where people sought predictions about the future. RAND researchers chose it because the method was built for forecasting, although its developers reportedly disliked the name’s mystical overtones.
A two-round Delphi study commonly takes two to six months. The timeline depends on panel size, how quickly experts respond, and how long the facilitator needs between rounds. Online surveys and automated reminders can shorten each round considerably.
In a classic Delphi study, panelists never meet, and they often do not know who else is on the panel. Some modified designs add a final meeting to discuss unresolved items, but individual ratings stay anonymous to protect independent judgment.
A lack of consensus is a valid result, not a failure. Report the items that stayed split, the range of ratings, and the arguments on each side. Persistent disagreement often signals real uncertainty that needs more data before anyone decides.
The method began at RAND for the US Air Force. Today, US hospitals, universities, medical societies, government agencies, and corporate strategy teams use it to build clinical guidelines, set research agendas, and forecast workforce or technology needs.



