Organizational Intelligence Surveys
The workforce survey is one of the most prevalent and widely used methods for collecting data and information about employee thoughts, feelings, and behaviors in organizational settings. Surveys in general are commonly used for varied purposes in the context of human capital management. Uses may include assessing learning needs, evaluating programs and solutions, measuring employee perceptions and attitudes, and conducting human capital and organizational research. But organizational intelligence surveys are an entirely different form of workforce surveys that account for the strategic and primary drivers that enable or inhibit employee engagement, and other important business outcomes.
Employee Engagement – A Desired State and Proxy Measure for Business Results
In 1990 William Kahn, a highly respected professor of organizational behavior, introduced the notion of engagement in the workplace. Arguably, however, it was the dot-com era, coupled with the war for talent (circa 1995-2000) that popularized the concept employee engagement. This led to the development and validation of a number of branded and competing definitions of engagement, survey instruments, and concomitant items and questions by consulting firms and research consortia and think tanks. Unfortunately, these varying definitions and measurement tools limited the extent to which research on employee engagement can be generalized beyond specific firms’ practices. Moreover, many of the survey instruments available comprise merely a few items related to employee satisfaction, motivation, commitment, and retention. They omit important factors and variables (i.e., the drivers) that ultimately affect employee engagement. Hence, the lack of a standard definition and reliable measurement tools has left practitioners dazed and confused as to what employee engagement actually is, and how to accurately measure it.
Organizational intelligence surveys measure employee engagement as a desired state in terms of employee motivation and performance (i.e., discretionary energy and effort), commitment and retention, and employee advocacy at the individual and organizational level. Therefore, employee engagement serves as a proxy for improving individual and organizational performance.
A Smarter Workforce Survey
Organizational intelligence surveys measure various aspects of organizational health and effectiveness – including leadership, employee strategy, and culture, and employee engagement. They are broader than employee engagement surveys, yet concise and more focused than traditional employee satisfaction surveys (i.e., those antiquated and ridiculously lengthy surveys with 100-150 items).
More importantly, organizational intelligence surveys measure and pinpoint the specific drivers of employee engagement including organizational factors (e.g., organizational policies and practices) that extends beyond the popular notion that it’s all about the behavior and effectiveness of individual leaders and managers. Although there are a common set of drivers for all employees as a whole (i.e., the usual suspects) that precede the state of engagement, the specific drivers and value propositions that matter most to employees vary significantly by workforce segment (e.g., A, B, C players). Yet more often than not, organizations tend to employ a “one-size-fits-all” approach when it comes to measuring and maximizing engagement in the workplace. By catering to your B-players, organizations run the risk of alienating their A-players and top talent, while providing no incentive for mediocre (i.e., C-players) to change, improve, or opt out all together (i.e., choose to leave the company). Therefore, it is critically important to identify the specific drivers of employee engagement by various talent segments including high-performers, high-potentials, aspiring leaders, employee with advanced degrees, and those with critical or scarce skills as well as your average Joe’s and Jane’s.
Three Tenets of Organizational Intelligence Surveys
In general, there are three tenets that underlie organizational intelligence surveys, making them distinct from traditional workforce surveys — they are:
Evidence-based. Organizational intelligence surveys are grounded in theory and empirical research and are tested for validity and reliability in different settings over time. Validity refers to the extent to which the survey items and questions truly represent the factor or variable of interest. In other words, they measure what they are supposed to measure. Reliability refers to the extent to which the survey instrument consistently measures the same characteristic or attributes over time.
Model-driven. Organizational intelligence surveys are based on a specific model or conceptual framework surrounding how people and organizations function. Model-driven survey efforts have been the mainstay in organization development circles for many years. Yet many of the models used today lack predictive utility in terms of measurement internal validity and reliability through which meaningful predictive assertions can be made. The Organizational Intelligence Model below serves as a useful framework to facilitate the design and interpretation of most employee and organizational survey efforts. The model includes 11 factors that affect employee engagement and performance. It depicts a top-down causal chain, making some tentative assertions with respect to cause and effect. In many ways, the organizational intelligence model can be thought of as a representation of an organization. The variables in the upper part of the model (such as environmental inputs) affect the organization from the outside. Within the organization, all strategic levers (leadership, strategy, and culture) affect the key indices of organization capability and execution (e.g., structure and decision rights, information and technology, management practices, and rewards and growth opportunities, among other factors). These latter internal factors in turn influence employee engagement and performance. In short, the organizational intelligence model defines important factors and relationships to consider in designing and developing survey instruments and items.
Focused on Action Planning and Change. Organizational intelligence surveys focus on action planning and change. The action planning process involves identifying important issues for the organization to address. Through it, ideas and solutions are generated and appropriate solutions and best approaches to implementation are selected. For lasting change to occur, all levels of the organization—corporate, geographic regions, business units, functions, teams, geographic segmentation and individual line managers—must participate in developing, implementing, and assuming ownership for continuous improvement.
LEARN ABOUT: Workforce Planning Model
Survey Design and Core Instrument
Again, the organizational model serves as the framework to guide the design and development of the survey. A survey based on the organizational intelligence model would be organized into 11 categories based on the factors in the model. The survey items representing each of these categories would be developed based on the definition of each and the specific needs of stakeholders. Three to five survey items are a good rule of thumb for adequately covering each factor, while minimizing item redundancy. Broader constructs such as leadership and culture may require more items for adequate measurement. The final step in designing an organizational intelligence survey is the selection of the item-response alternatives and scales. There are many scales from which to choose. Typically, the five-point Likert scale is used.
The HR Intelligence Organization has developed a 55 item core survey instrument based on the Organizational Intelligence Model. The model and survey have been used and tested at several high profile firms including Hitachi, Oracle, and FedEx to name a few.
Survey Deployment
Less than a 20 years ago, online survey technology was considered an alternative to the traditional paper and pencil survey. Today, online technology and SasS-based solutions (e.g., QuestionPro Workforce) are the norm, and provides a quick and efficient means for deploying organizational intelligence surveys. For some organizations and industries where employees do not have Internet access, such as retail and manufacturing environments, online survey technology may not be feasible. But where practical, there are important advantages to using online surveys:
- global reach—the ability to distribute surveys to a geographically dispersed workforce in any language worldwide
- real-time response tracking—the ability to monitor workforce survey response rates in real time
- instant feedback and reporting—the ability to provide immediate online feedback and reporting to stakeholders
- driver analysis and segmented insights —the ability to automatically identify the top drivers of employee engagement (i.e., what matters most), and glean actionable insights by various workforce segments
- high-quality open-ended responses due to typed, as opposed to handwritten, responses—ability to edit and spell check comments prior to survey submission
Survey Analytics
Organizational intelligence surveys can be analyzed through a number of techniques. These include item analysis, conceptual analysis, comparative analysis, and content analysis. Item, conceptual, and comparative analyses are all used when working with quantitative survey data; content analysis is used for qualitative, or open-ended, survey data.
- Item analysis involves describing data in terms of frequencies, means, standard deviations, ranges, and percentages, and is the simplest type of analysis. This method is an important first step in identifying relative highs and lows in the data set. Since item analysis is applicable to individual survey items only (the unit of analysis is at the item level alone), the more complex relationships evident in the data will not be uncovered without further analysis.
- Conceptual analysis is more advanced and involves testing relationships among the factors in the survey, as identified by the organizational model. One trend that is gaining popularity in all realms of survey practice is identifying key factors, variables, and indices that drive desired outcomes such as employee engagement. Advanced statistical techniques such as correlation and regression analysis, and causal and predictive modeling are used as part of item analysis as well as conceptual analysis. If done with the appropriate level of rigor, accounting for sufficient validity and reliability, the technique adds more power, utility, and credibility to any workforce survey process.
- Another more common type of analysis is comparative analysis. As the name suggests, it involves comparing the survey results of one group or organization to another, comparing the results over time, or comparing the results of one company to another or to the industry (e.g., benchmarking). These comparisons may involve item or construct comparisons, as well as demographic comparisons.
- The forth type of analysis is performed on open-ended or write-in survey questions. Content analysis involves categorizing open-ended responses into major themes. This data is a rich complement to quantitative data, and often helps to add context to the quantitative data because it is far more descriptive in nature. Some recent advancements have been made with sentiment analysis. For example, there are a handful of open source and proprietary solutions available that use machine learning and algorithms to analyzed written comments.
Moving Beyond the Proverbial Data Dump
Today, more effort and emphasis is placed on the development of an executive summary of the survey results. The executive summary is crafted to tell a compelling story about the organization. This summary brings interpretation and meaning to the large amounts of data found in standard survey reports. Such data include percent favorable, neutral, and unfavorable; high and low ratings; and diagnostic inferences. Whenever possible, the data-based diagnostic inferences and insights should include advanced analytical procedures such as driver analysis as mentioned before as well as linkage research, and causal research modeling procedures that demonstrate cause and effect. This is a powerful step in the overall organizational intelligence survey process. It differs significantly from traditional employee and organizational survey efforts that tend to focus exclusively on data and information rather than survey intelligence.
In summary, organizational intelligence surveys that are grounded in science and that are model-driven can greatly enhance the analysis and interpretation of the survey results, and can provide a valuable framework from which to act and make organizational changes.
If you have any questions or are interested in learning more about organizational intelligence surveys, please contact: Salvatore Falletta at sfalletta@hrintelligence.org
