Plenty of universities offer survey design and stat courses. If “stats” was a mandatory requirement for your post-secondary degree (and math’s not your thing) you probably fell into the big pool of students drowning in all the formula and despising the subject.
Far from throwing a few questions together and tossing them out for responses, surveying is a complex, artful science. Researchers and statisticians are forever questing for accuracy improvements vis à vis design and analysis methodologies. After all, defective data is risky data that can lead to perilous decisions – the last thing you want to happen in your organization.
These six tips for reliable survey data can help shape the trustworthiness of your employee survey results.
1. Know what you want to know
To get the right data and insights for action, determine:
- the purpose of your survey – set a goal
- the questions to be asked – what themes
- the sub-groups you want to be able to analyze (departments, functions, regions).
2. Know your respondents
To increase your employees’ chances of completing the survey in full and answering as accurately as possible:
- keep the process easy
- factor in education, reading levels and languages
- keep in mind the length of each question and the employee survey in its entirety
- phrase questions with brevity, simplicity and clarity
- make sure questions are relevant to everyone taking the survey
- ask only one question at a time (“My manager shares information openly and often” – is double barreled, the manager may share information openly but not often, or vice-versa).
3. Pre-test before rollout
- include respondents in the design of your survey to check for wording, comprehension, flow, survey length
- invite people who haven’t been involved in the design process to complete your proposed employee survey – find out if there are any problems that might lead to inaccurate responses and revise accordingly.
4. Create and track benchmarks
- establish benchmarks (baseline numbers) for monitoring, planning and decision-making purposes
- ask standard, consistently-phrased questions so you can compare and monitor data results from one survey to the next, apple-to-apple
- use external benchmarks from other organizations to see trends and measure your results against a bigger, broader perspective
- track how responses to specific questions change on an organization level and by sub-groups.
5. Make results easier to analyze
- use the Likert 5 or 7 point scale format for optimal accuracy – the standard of most researchers
- include filters to be able to look at sub-groups (regional office A-regional office B- headquarters; HR-finance-communications-operations-sales)
- build in cross-tabulation components to be able to compare sub-groups, but make sure results are statistically significant and retain confidentiality; breakdowns for departments of five employees or less are not supported by TalentMap or most other employee survey companies
- build in correlations where questions and responses have a relationship to other questions and responses.
6. Draw conclusions from different angles
- establish criteria for exclusion and discard suspect data (i.e. incomplete surveys)
- understand the impact of unanswered/neutral data and treat these responses separately
- avoid the knee-jerk reaction of heading straight for the lowest scoring issues
- look at the distribution of responses and the three different kinds of averages
- Mean: add up all the numbers then divide by the number of numbers
- Median: the middle value where 50% of responses fall to one side and 50% to the other side
- Mode: the most frequent response
- consider how one factor might impact another (i.e. my manager encourages me to offer my ideas and opinions; there is a culture of innovation at this organization)
- consider how two factors work together but don’t influence each other (i.e. my manager acts consistently; my manager seems to care about me as a person)
- look at the relationship between different factors and their combined impact (i.e. my manager encourages me to offer my ideas and opinions; there is a culture of innovation at this organization; my career aspirations can be achieved at this organization; my organization inspires me to do my best work).