• Skip to main content
  • Skip to primary sidebar
  • Skip to footer
QuestionPro

QuestionPro

questionpro logo
  • Products
    survey software iconSurvey softwareEasy to use and accessible for everyone. Design, send and analyze online surveys.research edition iconResearch SuiteA suite of enterprise-grade research tools for market research professionals.CX iconCustomer ExperienceExperiences change the world. Deliver the best with our CX management software.WF iconEmployee ExperienceCreate the best employee experience and act on real-time data from end to end.
  • Solutions
    IndustriesGamingAutomotiveSports and eventsEducationGovernment
    Travel & HospitalityFinancial ServicesHealthcareCannabisTechnology
    Use CaseAskWhyCommunitiesAudienceContactless surveysMobile
    LivePollsMember ExperienceGDPRPositive People Science360 Feedback Surveys
  • Resources
    BlogeBooksSurvey TemplatesCase StudiesTrainingHelp center
  • Features
  • Pricing
Language
  • English
  • Español (Spanish)
  • Português (Portuguese (Brazil))
  • Nederlands (Dutch)
  • العربية (Arabic)
  • Français (French)
  • Italiano (Italian)
  • 日本語 (Japanese)
  • Türkçe (Turkish)
  • Svenska (Swedish)
  • Hebrew IL (Hebrew)
  • ไทย (Thai)
  • Deutsch (German)
  • Portuguese de Portugal (Portuguese (Portugal))
  • Español / España (Spanish / Spain)
Call Us
+1 800 531 0228 +1 (647) 956-1242 +55 9448 6154 +49 030 9173 9255 +44 01344 921310 +81-3-6869-1954 +61 (02) 6190 6592 +971 529 852 540
Log In Log In
SIGN UP FREE

Home Market Research

Delphi Method: Definition, Steps, Examples, Pros and Cons

delphi-method

The Delphi method is a structured forecasting and consensus technique in which a panel of experts answers anonymous questionnaires over several rounds. After each round, a facilitator shares a summary of the group’s answers. Experts then revise their views until opinions converge or stabilize.

Teams turn to it when the data they need does not exist yet. Think of questions like “Which skills will our workforce need by 2030?” A public survey cannot answer that, and an expert meeting often ends with the loudest voice winning.

In this guide, we break down how the Delphi technique works, how to measure consensus, real examples, the main variants, and the mistakes that weaken a Delphi study.

Content Index hide
1. What is the Delphi method?
2. What are the key features of the Delphi technique?
3. How does the Delphi method work? Delphi method steps
4. How do you measure consensus in a Delphi study?
5. Delphi method examples
6. Where is the Delphi method used?
7. What are the advantages and disadvantages of the Delphi method?
8. How does the Delphi method compare with other research methods?
9. What are the main types of Delphi method?
10. When should you use the Delphi method?
11. What are common mistakes in a Delphi study?
12. Is the Delphi method qualitative or quantitative?
13. How can QuestionPro support a Delphi study?
14. Getting reliable answers from the Delphi method
15. Frequently Asked Questions (FAQs)

What is the Delphi method?

The Delphi method is a group research technique that turns individual expert judgment into a reliable collective answer, without putting the experts in the same room. People also call it the Delphi technique or a Delphi study.

The method grew out of Project RAND, a US Air Force research program that later became the RAND Corporation. In the 1950s, Olaf Helmer, Norman Dalkey, and Nicholas Rescher developed it to forecast how technology would affect warfare. A 1964 report by Theodore Gordon and Olaf Helmer then applied it to long-range trends in science and technology.

From there, the approach spread to business strategy, public policy, education, and health care. In health sciences, it now helps teams set research priorities and develop clinical guidelines.

It sits alongside interviews and focus groups among qualitative research methods that rely on expert insight rather than large samples. What sets it apart is the structure: anonymous answers, repeated rounds, and feedback in between.

What are the key features of the Delphi technique?

Four features separate the Delphi technique from an ordinary expert survey or meeting. Remove any one of them and you no longer have a true Delphi study.

  • Anonymity: Panelists do not know who said what, and often do not know who else is on the panel. This removes the pull of seniority, reputation, and dominant personalities.
  • Iteration: Experts answer the same or refined questions over at least two rounds. Each round gives them a chance to reconsider.
  • Controlled feedback: Between rounds, the facilitator shares a summary of group responses and the reasons behind them. Panelists react to arguments, not to people.
  • Statistical group response: The final answer appears as a group statistic, such as a median, a percent agreement, or a spread of ratings. Minority views stay visible in the report.

Many practitioners add a fifth trait: a heterogeneous panel. Experts from different backgrounds and regions challenge each other’s assumptions, which makes the final consensus more robust.

How does the Delphi method work? Delphi method steps

The Delphi method works through a planned sequence: frame the question, recruit experts, run two or three anonymous rounds with feedback, and stop at a pre-set rule. Most of the study’s quality depends on decisions made before the first questionnaire goes out.

  1. Appoint a neutral facilitator.
    The facilitator designs questionnaires, analyzes answers, writes feedback summaries, and keeps the panel engaged. They should have no stake in the outcome.
  2. Define the objective and scope.
    Write one clear research question, such as “Which factors will most affect US nurse retention through 2030?” A narrow scope keeps rounds short and answers comparable.
  3. Select the expert panel.
    Use purposive sampling to recruit people with direct, relevant expertise. Write down your inclusion criteria, such as years of experience, role, or publications, and aim for a mix of perspectives.
  4. Set the consensus and stopping rules.
    Decide before Round 1 what counts as agreement and when the study ends. For example: “Consensus means 75% of panelists rate an item 7 to 9 on a 9-point scale. We stop after three rounds.”
  5. Run the rounds.
    Send each questionnaire, analyze the responses, and share anonymized feedback before the next round. The table below shows what each round contains.
  6. Analyze and report the results.
    Report the items that reached consensus, the items that did not, and how ratings shifted across rounds. Then turn the findings into a forecast, guideline, priority list, or decision.

What happens in each Delphi round?

Each round has a distinct job. Round 1 gathers ideas, Round 2 rates them, and Round 3 checks whether experts hold their positions after seeing the group view.

Round Purpose Typical question format Feedback panelists get next
Round 1 Generate ideas and arguments Open-ended questions, such as “What barriers will slow AI adoption in US community banks?” A clean, deduplicated list of all ideas
Round 2 Rate or rank each idea Rating scales, such as a 1 to 9 importance scale or a 5-point Likert scale Group median, spread, and their own previous rating
Round 3 Reconsider and confirm The same rating items, with group results shown Final consensus list and remaining disagreements
Optional Round 4 Resolve items that are still split Targeted questions, plus a request to explain outlier ratings Final report

Many teams use a modified Round 1, where the facilitator starts from a literature review instead of open questions. This saves a round, but it limits the ideas to what the facilitator already found.

How do you measure consensus in a Delphi study?

You measure consensus in a Delphi study with a pre-defined statistic, usually percent agreement, often combined with the median rating and the interquartile range. The key word is pre-defined: set the rule before any data arrive.

In practice, many studies skip this step. A systematic review of 100 Delphi studies found that 98 said they assessed consensus, but only 72 actually defined it. Percent agreement was the most common definition, with a median threshold of 75%.

Four measures do most of the work, usually in combination:

  • Percent agreement: The share of panelists who rate an item in the agreement range. A common bar is 70% to 80% of the panel.
  • Median: The central rating of the group. A typical rule is a median of 7 or higher on a 9-point scale.
  • Interquartile range (IQR): How tightly the middle 50% of ratings cluster. An IQR of 1 or less on a 4- or 5-point scale signals close agreement.
  • Stability: Whether ratings change between rounds. Little or no shift across two consecutive rounds means more rounds will not help.

These rules of thumb come from a narrative review of Delphi use in health sciences. It adds that high-stakes guidelines, such as end-of-life care, may justify a stricter bar of 90% to 100%. The IQR is simply the gap between the 25th and 75th percentile ratings, so a small IQR means experts agree closely.

How many experts and rounds does a Delphi study need?

No single panel size is correct, but most Delphi studies use between 8 and 20 experts and run two to three rounds. Treat these benchmarks from the same health sciences review as starting points:

  • Panel size: Published panels range from as few as 4 experts to several thousand, but most sit between 8 and 20. The review suggests 8 to 23 participants as a practical target.
  • Small-panel risk: In a 10-person panel, one expert equals 10% of the vote. A single dropout can push an item across your threshold.
  • Rounds: Classic Delphi studies typically run 2 to 3 rounds, and many authors consider 3 ideal.
  • Stopping rule: Most studies stop after a fixed number of rounds. In the systematic review above, 70 studies stopped after a set number of rounds rather than at consensus.
  • Attrition: Dropout across rounds ranged from 0% to 92% in classic Delphi studies, so recruit more experts than you need.

These numbers also resolve a common confusion. A Delphi panel does not need hundreds of people, because it measures expert judgment, not population opinion.

Delphi method examples

Delphi method examples span defense, health, business, HR, and education. The table shows how the same process adapts to very different questions.

Field Example question Typical panel Output
Technology forecasting Which scientific breakthroughs are likely over the coming decades? (the 1964 RAND study) Scientists and engineers Timeline of expected breakthroughs
Health care Which criteria show a patient is ready for discharge after knee surgery? Surgeons, nurses, physical therapists Consensus checklist or clinical guideline
Public health Which measures matter most during a fast-moving outbreak such as COVID-19? Epidemiologists, clinicians, public health officials Rapid priorities, often through a fast-track Delphi
HR and workforce planning Which skills will our engineering team need by 2030? Engineering leads, HR partners, outside advisors Ranked skills list for hiring and training
Market research How fast will US small businesses adopt a new payment technology? Industry analysts, bankers, product leaders Adoption forecast with ranges
Education Which competencies should a new data science curriculum cover? Faculty, employers, alumni Prioritized competency framework

Worked example: A Delphi study for a product roadmap

Here is how a B2B software company might run a three-round Delphi study. The numbers are illustrative, but the flow mirrors a typical study.

  1. Round 1 collects ideas.
    The facilitator invites 15 customers, sales engineers, and support leads. Each answers one open question: “What capabilities will mid-market buyers expect from our product in two years?” The panel returns 64 ideas, which the facilitator merges into 22 distinct items.
  2. Round 2 rates the list.
    Panelists rate each item on a 1 to 9 importance scale. Nine items meet the pre-set rule of at least 75% rating 7 or higher.
  3. Round 3 tests the result.
    Panelists see the group median and their own earlier rating for the 13 unresolved items. Four more reach consensus, while two stay split between customers and sales engineers.
  4. The team acts on the output.
    Thirteen consensus items go into roadmap planning. The two split items become follow-up customer research rather than forced decisions.

Where is the Delphi method used?

Organizations apply the Delphi method wherever decisions depend on expert judgment about an uncertain future or an ill-defined problem. These are the most common applications:

  • Strategic decision-making: Leaders test long-term bets, such as market entry or new business models, against independent expert views.
  • Forecasting and trend analysis: Teams estimate when technologies, regulations, or consumer behaviors will shift.
  • Market analysis: Researchers estimate demand for products that have no sales history yet.
  • Policy development: Government agencies and nonprofits weigh policy options and their likely consequences.
  • Risk assessment: Project and risk managers identify and rank risks that historical data cannot reveal.
  • Health research: Clinicians build guidelines, define outcome sets, and set research priorities.
  • Education: Faculty and employers agree on competencies and curriculum content.
  • Environmental planning: Scientists and planners assess climate, land-use, and resource scenarios.
  • Disaster preparedness: Emergency managers rank response capabilities before a crisis hits.
  • Software estimation: Engineering teams use Wideband Delphi to estimate project effort.

What are the advantages and disadvantages of the Delphi method?

The main advantage of the Delphi method is independent, bias-resistant expert judgment. Its main disadvantage is time, so weigh both before you commit to a multi-round study.

Advantages

The benefits all trace back to anonymity and iteration.

  • Less groupthink: Anonymity stops senior or vocal experts from steering the group.
  • Remote by design: Experts in different cities or countries can join without travel.
  • Room to rethink: Panelists can change their minds without losing face.
  • Broad input: You can include more experts than any meeting room could seat.
  • Transparent output: Group statistics show both where experts agree and where they do not.
  • Buy-in: Experts who shaped a result are more likely to support putting it into practice.

Disadvantages

Most drawbacks come from the same multi-round structure that makes the method rigorous.

  • Slow: Several rounds of questionnaires, analysis, and feedback take weeks or months.
  • Dropout: Busy experts leave between rounds, which can distort the results.
  • Facilitator influence: Whoever writes the summaries can frame the feedback, intentionally or not.
  • False consensus: Agreement does not prove a correct answer. Panelists may converge on a shared blind spot.
  • Selection bias: A panel that is too similar will agree quickly and miss risks.
  • Cost: Recruiting and compensating qualified experts adds up.

How does the Delphi method compare with other research methods?

The Delphi method differs from other group methods mainly in its anonymity and iteration. Use this comparison to pick the right tool for your question.

Method How it works Interaction between participants Best for
Delphi method Anonymous questionnaires over several rounds, with feedback None Consensus or forecasts from dispersed experts
Focus group A moderated live discussion with a small group Open, in person or as an online focus group Exploring attitudes, language, and reactions
Survey A one-time questionnaire sent to a large sample None Measuring how common a view is in a population
Expert panel meeting Experts discuss and decide together Open and identified Fast decisions when hierarchy is not a concern
Brainstorming Free-flowing idea generation in a group Open and spontaneous Generating many ideas quickly
Nominal group technique Silent idea writing, round-robin sharing, then private voting in one session Structured and in person Prioritizing options within a few hours

What is the difference between the Delphi method and the nominal group technique?

Both methods structure group judgment, but they trade speed against anonymity. The nominal group technique (NGT) brings participants together for one facilitated session. Each person writes ideas silently, shares them in turn, discusses them briefly, and then votes privately.

A nominal group session typically takes less than two hours per question. A Delphi study keeps experts apart and anonymous for weeks or months instead.

Choose NGT when you can gather people in one room and need an answer quickly. Choose Delphi when experts are dispersed, when status differences could silence some voices, or when you want time for considered judgment. The RAND/UCLA Appropriateness Method combines both, pairing Delphi-style rating rounds with a group discussion.

What are the main types of Delphi method?

The main types are the classic, modified, policy, e-Delphi, real-time, and Wideband Delphi. All of them keep anonymity and feedback, but they differ in their starting point, format, and goal.

  • Classic Delphi: Starts with open-ended questions and runs anonymous rating rounds toward consensus.
  • Modified Delphi method: Starts from a literature review or interviews, and may add a meeting. Designs vary widely from study to study.
  • Policy Delphi: Aims to surface the full range of opposing views on a policy issue, not to force agreement.
  • E-Delphi: Runs every round through online survey software, which speeds up distribution and analysis.
  • Real-time Delphi: Replaces rounds with a continuous online process, where experts see group results update as they answer.
  • Wideband Delphi: Adds short discussions between estimation rounds and suits software effort estimates.

When should you use the Delphi method?

Use the Delphi method when the question needs expert judgment, the experts are hard to gather, and you have weeks rather than days. If any of those conditions fail, another method usually works better.

Use the Delphi method when:

  • Data is missing: Little historical data or published evidence exists on the question.
  • Experts are dispersed: Panelists work in different organizations, cities, or time zones.
  • Hierarchy could bias results: Junior experts might defer to senior ones in a meeting.
  • You need defensible consensus: A guideline, standard, or forecast must show how the group reached agreement.

Choose another method when:

  • You need population data: A representative survey measures what customers or employees think.
  • You need an answer this week: A nominal group session or a structured workshop is faster.
  • You want to explore feelings and language: Focus groups and interviews capture nuance better.
  • The answer is already known: A literature review costs far less than convening experts.

What are common mistakes in a Delphi study?

Most weak Delphi studies fail in the design, not the analysis. Avoid these six mistakes.

  1. Defining consensus after the data arrive.
    Setting the threshold late lets anyone tune the result. Write the rule into your protocol before Round 1.
  2. Recruiting a narrow panel.
    A panel from one profession will agree fast and miss blind spots. Mix roles, sectors, and regions.
  3. Asking too many items.
    Long rating lists cause survey fatigue and dropout. Cut the list to items that matter for the decision.
  4. Writing biased feedback.
    Summaries that highlight one side nudge panelists toward it. Report medians and spreads neutrally, and include minority arguments.
  5. Ignoring attrition.
    If a third of the panel leaves, the remaining “consensus” may reflect only the most committed experts. Report dropout for every round.
  6. Treating consensus as truth.
    Agreement shows shared judgment, not proof. Check high-stakes findings against real data once it becomes available.

Is the Delphi method qualitative or quantitative?

The Delphi method combines both: Round 1 is usually qualitative, and later rounds are quantitative. That mix is why researchers often describe it as a consensus method rather than placing it in one camp.

Open-ended answers in the first round produce qualitative data that the facilitator codes into themes. Rating rounds then produce numbers, such as medians and agreement percentages.

However, the method does not rely on statistical sampling. You choose a Delphi panel for expertise, so its results describe expert judgment, not a population. For that reason, many textbooks still group it with qualitative methods, even though its output looks statistical.

How can QuestionPro support a Delphi study?

QuestionPro supports a Delphi study by handling the parts that slow facilitators down: anonymous distribution, round-by-round questionnaires, reminders, and analysis. Here is how each capability maps to the process.

  • Anonymous responses: Respondent Anonymity Assurance keeps email addresses and identifiers out of reports, while still showing who has responded. You can follow up with non-responders without exposing anyone’s answers.
  • Round-specific questionnaires: Build Round 1 with open-ended questions, then copy the survey and swap in rating scales for Rounds 2 and 3.
  • Feedback blocks: Add the group median and summary arguments at the top of each new round, so panelists see the group view before they rate again.
  • Survey logic: Show a follow-up question only to panelists whose ratings fall outside the group range, and ask them to explain their position.
  • Text AI: Group hundreds of Round 1 comments into themes faster, then review and merge the items by hand.
  • Smart Dashboards: Track medians, agreement percentages, and response rates per round, then export the data for IQR calculations.

Teams running Delphi studies for academic, health, or market research projects can manage every round in QuestionPro Market Research Software.

Getting reliable answers from the Delphi method

The Delphi method remains one of the few research tools built for questions that data cannot answer yet. Its strength comes from discipline: a sharp question, a varied panel, a consensus rule written in advance, and honest feedback between rounds.

Skip any of those and you get a slow survey with a confident-looking result. Get them right, and a small group of experts can produce a forecast, guideline, or priority list that holds up to scrutiny.

Create memorable experiences based on real-time data, insights and advanced analysis. Request Demo

Frequently Asked Questions (FAQs)

Why is it called the Delphi method?

The name refers to the Oracle of Delphi, the ancient Greek shrine where people sought predictions about the future. RAND researchers chose it because the method was built for forecasting, although its developers reportedly disliked the name’s mystical overtones.

How long does a Delphi study take?

A two-round Delphi study commonly takes two to six months. The timeline depends on panel size, how quickly experts respond, and how long the facilitator needs between rounds. Online surveys and automated reminders can shorten each round considerably.

Do Delphi panelists ever meet each other?

In a classic Delphi study, panelists never meet, and they often do not know who else is on the panel. Some modified designs add a final meeting to discuss unresolved items, but individual ratings stay anonymous to protect independent judgment.

What happens if experts never reach consensus?

A lack of consensus is a valid result, not a failure. Report the items that stayed split, the range of ratings, and the arguments on each side. Persistent disagreement often signals real uncertainty that needs more data before anyone decides.

Who uses the Delphi method in the United States?

The method began at RAND for the US Air Force. Today, US hospitals, universities, medical societies, government agencies, and corporate strategy teams use it to build clinical guidelines, set research agendas, and forecast workforce or technology needs.

SHARE THIS ARTICLE:

About the author
Anas Al Masud
Digital Marketing Lead at QuestionPro, working across SEO, AI Search, and content strategy. Specializes in survey and research content, with a focus on improving search visibility, creating useful content, and helping people find the information they need through traditional search engines and AI-powered platforms.
View all posts by Anas Al Masud

Primary Sidebar

Research what's on your mind. Find out what's on theirs!

A suite of tools to leverage research and transform insights.

Discover our insight platform

RELATED ARTICLES

HubSpot - QuestionPro Integration

Consortium Licensing for Universities: How Shared Procurement Models Cut Research Software Costs

Sep 18,2026

HubSpot - QuestionPro Integration

Costco NPS 2025: What the Score Says About Member Loyalty

Apr 11,2025

HubSpot - QuestionPro Integration

CAPI Survey in India: How It Works, What It Costs, and When to Use It

Apr 16,2026

BROWSE BY CATEGORY

Footer

MORE LIKE THIS

question-stacks

Question Stacks in QuestionPro BI for a shared view of related ratings

Nov 11, 2026

QuestionPro CX- AI widget

Beyond Dashboards: How Conversational AI Is Transforming Customer Experience Analytics

Nov 9, 2026

shared-workspace-in-questionpro-audience

Shared workspace in QuestionPro Audience: Share projects with your team without giving up ownership

Nov 8, 2026

test-survey-logic

Test survey logic without the click-through marathon: Meet the survey path simulator

Nov 5, 2026

Other categories

questionpro-logo-nw
Help center Live Chat SIGN UP FREE
  • Sample questions
  • Sample reports
  • Survey logic
  • Branding
  • Integrations
  • Professional services
  • Security
  • Survey Software
  • Customer Experience
  • Workforce
  • Communities
  • Audience
  • Polls Explore the QuestionPro Poll Software - The World's leading Online Poll Maker & Creator. Create online polls, distribute them using email and multiple other options and start analyzing poll results.
  • Research Edition
  • LivePolls
  • InsightsHub
  • Blog
  • Articles
  • eBooks
  • Survey Templates
  • Case Studies
  • Training
  • Webinars
  • All Plans
  • Nonprofit
  • Academic
  • Qualtrics Alternative Explore the list of features that QuestionPro has compared to Qualtrics and learn how you can get more, for less.
  • SurveyMonkey Alternative
  • VisionCritical Alternative
  • Medallia Alternative
  • Likert Scale Complete Likert Scale Questions, Examples and Surveys for 5, 7 and 9 point scales. Learn everything about Likert Scale with corresponding example for each question and survey demonstrations.
  • Conjoint Analysis
  • Net Promoter Score (NPS) Learn everything about Net Promoter Score (NPS) and the Net Promoter Question. Get a clear view on the universal Net Promoter Score Formula, how to undertake Net Promoter Score Calculation followed by a simple Net Promoter Score Example.
  • Offline Surveys
  • Customer Satisfaction Surveys
  • Employee Survey Software Employee survey software & tool to create, send and analyze employee surveys. Get real-time analysis for employee satisfaction, engagement, work culture and map your employee experience from onboarding to exit!
  • Market Research Survey Software Real-time, automated and advanced market research survey software & tool to create surveys, collect data and analyze results for actionable market insights.
  • GDPR & EU Compliance
  • Employee Experience
  • Customer Journey
  • Synthetic Data
  • About us
  • Executive Team
  • In the news
  • Testimonials
  • Advisory Board
  • Careers
  • Brand
  • Media Kit
  • Contact Us

QuestionPro in your language

  • English
  • Español (Spanish)
  • Português (Portuguese (Brazil))
  • Nederlands (Dutch)
  • العربية (Arabic)
  • Français (French)
  • Italiano (Italian)
  • 日本語 (Japanese)
  • Türkçe (Turkish)
  • Svenska (Swedish)
  • Hebrew IL (Hebrew)
  • ไทย (Thai)
  • Deutsch (German)
  • Portuguese de Portugal (Portuguese (Portugal))
  • Español / España (Spanish / Spain)

Awards & certificates

  • survey-leader-asia-leader-2023
  • survey-leader-asiapacific-leader-2023
  • survey-leader-enterprise-leader-2023
  • survey-leader-europe-leader-2023
  • survey-leader-latinamerica-leader-2023
  • survey-leader-leader-2023
  • survey-leader-middleeast-leader-2023
  • survey-leader-mid-market-leader-2023
  • survey-leader-small-business-leader-2023
  • survey-leader-unitedkingdom-leader-2023
  • survey-momentumleader-leader-2023
  • bbb-acredited
The Experience Journal

Find innovative ideas about Experience Management from the experts

  • © 2022 QuestionPro Survey Software | +1 (800) 531 0228
  • Sitemap
  • Privacy Statement
  • Terms of Use