Employee Engagement Best Practice Series Verbatim Comments – Walking the Tightrope

One of the most common questions we at TalentMap receive almost on a weekly basis is how to handle and distribute verbatim comments. On the one hand, comments submitted by individual employees provide essential insight into the issues and root causes underlying the different scale responses. They need to be read. However, unlike marketing or public opinion surveys, the users of the employee survey data (HR and management) often know many of the respondents (employees), and in many cases, work directly with some of them. Of course, this introduces a whole new dynamic with regard to the sharing of the verbatim comments portion of the survey data.

A well done survey conducted according to best practice and industry guidelines (MRIA in Canada, CASRO in the USA, ESOMAR elsewhere in the world) will guarantee respondent anonymity by only providing aggregate results, usually in groups no smaller than at least five respondents. But let’s be real here. In more than 30 years of conducting surveys, I have come to understand two basic truths about employee surveys:

1. Employees will, unwittingly or not, “out” themselves. Notwithstanding the various warnings and disclaimers to respondents, many employees will still relate situations and experiences which will provide enough information for many survey users (read: managers) to at least try to guess the identity of the commenter.
2. Managers will try and guess “who said what”. As much as we warn and coach management groups not to attempt to attribute comments to the individual, human nature is such that the first reaction of many managers, even the most well-meaning ones, is to attempt to identify an employee based on a comment.

How then do we derive maximum benefit and insight which is provided through verbatim comments, while also protecting the anonymity of the individual employee, especially when many employees and managers are subverting the process? Here’s my list of do’s and don’ts:

DO provide verbatim comments to HR and senior leadership teams. That being said, provide the entire list of verbatim comments to all senior leaders, regardless of their area of responsibility. In the large majority of cases, organizations are large enough so that the sheer number of comments will be enough to protect anonymity. It’s awfully difficult and time-consuming to try and guess who said what when you’re confronted by dozens, if not hundreds of pages of comments. The reaction is to treat the comments the way we would treat them here at TalentMap: scan for themes and patterns in the issues, not caring about who said it. For their part, HR is accustomed to dealing with sensitive employee-specific information, and these comments should be treated no differently than other private employee information.

DO use the comments to understand why employees responded to scale questions in the manner that they did – as opposed to serving as a reflection of the entire body of employee opinion. For example, if we note that scores on an important driver of engagement, say professional growth, are atypically low, the comments on this issue will shed a lot of insight as to why that’s the case, thereby providing greater understanding of the issue. Keep in mind, open-ended questions are specifically designed to elicit areas for improvement, which implies criticism and a more “negative” bias. Therefore, when reading them, we’ll get a much more negative impression than is actually the case. The scale questions tell us how many people feel a certain way. The comments tell us why.

DO ensure your survey provider is reading and incorporating comments into the analysis. Your external survey provider is (or should be) the best placed to handle verbatim comments. Respondents are truly anonymous in their eyes (we don’t know them or their situations), yet an experienced survey analyst has seen many situations before and can quickly identify issues and themes, and provide a much more compelling and objective “story” behind the data.

DON’T distribute comments to individual managers even when they ask, beg, cajole, and/or tell you that they can’t act on the survey results without this information. The greater the desire to see individual comments for small groups, the more likely that manager is seeking to identify individual respondents. As a rule of thumb, I would say that distributing comments of fewer than 100 people to any manager will lead to potential abuse and potential breach of privacy.

DON’T try and guess the author of the comments. By now, this should be obvious, but it still happens regularly and is worth repeating. We once conducted an experiment whereby managers were provided comments by their direct reports (with their permission – for the purposes of the experiment). The result was that managers guessed wrong more than 50% of the time. Once a manager draws a conclusion as to who may have authored a comment (particularly an unflattering one), the manager’s behaviour towards that individual will change in a noticeable manner (often unconsciously). The affected employee will lose complete trust in the survey and employee engagement initiative as a result. Not exactly what we’re going for.

So, HR managers and senior leadership: the comments provided should be read as they provide a great deal of insight into the underlying issues of engagement in your organization, but recognize also that it is human nature to try and guess who said what. Protecting your employees’ anonymity must take precedence over everything else.