Surveys were designed in the 1930s as a way to aggregate or count the opinions of a large number of people. Scale questions – the agree versus disagree, satisfied versus not satisfied and everything in between – are ways we derive statistics, turn attitudes into mathematical equations, compile and then extrapolate insights. The thing is responses to scale questions typically lead to more questions than answers. That’s why open-ended questions and verbatim comments are so critical.
- Open-ended questions provide a better understanding of why employees answer in a certain way.
- They allow people to freely express themselves.
- They provide more detail; give more insight, context and nuance.
- And importantly, they’re a springboard for meaningful conversations.
Verbatim comments alter the way we look at employee engagement results and have a direct bearing on what we’re going to do about it.
But sharing verbatim comments is a difficult and confusing ethical matter. When you’re dealing with scale questions and responses from groups of five, ten or more individuals, you can’t tell who rated what or how. With verbatim comments, you’re in different territory. And you’re dealing with two basic, fundamental truths.
Employees WILL unwittingly or not “out” themselves
In the act of expressing themselves, their concerns, their worries, their feelings, employees will refer to a situation or condition so specific that anybody who works anywhere near this person and reads that comment will be able to make a very good educated guess as to who that individual might be. Even though there’s no name, or sometimes no reference to other people, employees sabotage their own anonymity by being so explicit. Some comments will have a number of characterizations that identify the individual. Even writing styles can give someone away.
Managers WILL try to guess “who said what”
So how do we protect individuals against themselves when the second truth is that managers and other employees will always try and guess who-said-what. If you’re in an HR position you hear all kinds of arguments.
- “I really need to know what my people said….”
- “My group is different so I need to see their comments….”
- “I already know what they think. Nothing will surprise me….”
- “I can’t solve the problem unless I know what it is.…”
- “We need to see the comments otherwise we’re not being transparent….”
How then, do we reap the necessary insights from comments, while protecting employees’ anonymity (and any possible retribution)? This is the conundrum.
TalentMap doesn’t pretend to have all the answers, but over the years what we have developed is a way to walk this tightrope.
We review all verbatim responses; identify themes, commonalities or trends in the way people answer; provide percentages of common responses, and share anonymous illustrative examples.
When it comes to who sees comments, it’s highly recommended that distribution be restricted to two groups.
- The HR sponsors behind the survey
- Senior leadership – with caveats:
- Everybody gets everything because the sheer volume of comments forces a general look at themes; anonymity remains with safety in numbers
- NO breakdown of comments by sub-group (including departments)
- Absolutely NO distribution of comments in organizations with fewer than 200
Keep it simple with the following list of DOs and DON’Ts
DO:
- Step back and divorce your emotional self from this exercise, comments can be brutally harsh.
- Understand that comments are purposely skewed to the negative:
- Employees are asked what can be improved (rather than what is done well) because the whole purpose of the survey exercise is to measure their levels of engagement with an aim or objective of improving it. Verbatim comments give guidance.
- Scan and use a highlighter to identify behaviors.
- Ignore characterizations (“mean, abusive, acts like a bully”).
- Identify recurring issues and broad themes across many comments.
- Use comments as a starting point to deep-dive into further conversations.
DON’T:
- Read comments like a novel and paint a picture of the characters in your head.
- Attempt to identify individual comment authors (states the obvious, but everyone still does it).
- Interpret comments as a reflection of the prevailing view or perspective of the organization.
- It’s not a true picture. Statistics, the measurement of attitude, provides the picture. Verbatim comments provide the answer as to why the picture is what it is.
- EVER act on a single or isolated comment.
