Tom Kane had a nice post today on a topic he and I have discussed repeatedly -- client satisfaction surveys. I could not agree more about the critical importance of understanding how your clients feel about the quality of your services. As a matter of fact, I'm in the process of creating a partnership to begin offering client satisfaction assessments as a formalized service myself.
However, I think it's important to recognize the very real limitations of quantitative surveys. And they are quite real. Survey results need to be taken with a grain of salt, or two, or several thousand.
I recently reviewed the results of a survey fielded by a very well-known legal consulting firm which perfectly illustrates what I'm talking about. This survey was not cheap -- it cost the client a LOT.
First of all, the percentage of mailed/emailed surveys that were actually returned was under 30%, which is typical. With this response, you are not getting the entire picture by any means.
Second, the sample size for law firm surveys is usually quite small -- a couple hundred surveys mailed out, at best. If this were Best Buy or Delta Airlines, and you were receiving 2,000 survey responses a month, then you would have enough data to really derive some insights. The smaller the sample size, the less meaningful the data.
Third, most client satisfaction surveys generate some kind of quantified score on a number of different metrics. You know, "Our client satisfaction score was a 4.3 out of a possible 5." The relationship between a client and an attorney is nuanced, highly interpersonal, and very, very complex, to say the least. Reducing it to a number, or even a set of numbers, creates a deceptively precise result. You have to be extremely careful about reading too much into these figures, and especially into relatively minor trends and changes in them over time. It's very hard to quantify a relationship, which is what a quantitative survey does.
Law firms like quantitative surveys because they appear to eliminate ambiguity. Lawyers love precision and specificity, and quantitative surveys seem to deliver both. But that number is a little bit of an illusion. It's a two-dimensional picture of something with many, many dimensions.
Fourth, all clients are not created equal. For midsized firms, at least, the top ten clients tend to provide the bulk of the firm's revenues. Their perceptions are vastly more important to the firm's strategy and profitability than the more numerous, lesser clients. Unless some kind of weighting is applied, small clients will have a disproportionate impact on the final numbers of a quantitative survey.
And, finally, as a great Harvard Business Review article points out, even "satisfied" clients can and will defect. Unless your clients absolutely love your service, are completely satisfied and are willing to say so, they are by no means secure. A client who is a not a "5" in satisfaction is at risk, no matter what the survey percentiles tell you.
The point here is not that surveys are useless -- they're not. They have value. The point is that particularly with key clients, but basically with all clients, there is no substitute for an experienced, emotionally intelligent human being sitting in the same room, asking questions, and listening very carefully to the answer. There just isn't.