In my last post, I listed a number of important questions to ask potential partners before signing a contract. To briefly recap here, it’s essential to move beyond looking at canned slide decks and polished methodologies. The process of asking detailed and often difficult questions increases the chances of a successful outcome.

Show Me, Don’t Tell Me

Sure, it’s one of my favorite Rush songs, but the concept is very apropos here.

Detailed queries can only get you so far. To properly assess a potential fit, it’s silly not to ask, Have you done this for a similar organization before? To state the obvious, you’re looking for more than a simple yes or no.

Yes, I’m talking about case studies. To me, the absence of one is a big bête noire.

Consider Hadoop for a moment. Sure, it’s relatively nascent, but it’s 2015, not 2012. Rare is the client that wants to be a guinea pig. If your organization is thinking about actually doing something with Big Data, it’s fair to ask prospective partners to produce something other than glossy marketing materials.

Properly vetting a partner involves more than reading a case study.

Considerations and Disclaimers

Against this backdrop, here’s my advice on how to read case studies:

• I’m totally biased. All of my books contain detailed case studies. Others can learn a great deal about what to do—and what not to do—by examining in depth how organizations have tackled related issues.
• Not every client wants to be named. Many partners sign partners non-disclosure agreements (NDAs) with their clients. As such, it’s possible if not likely that senior management at Partner X would love to tell you its amazing success stories but legally cannot.
• Still, you can get a little bit pregnant. Not all NDAs are equally restrictive. All else being equal, a government agency is going to insist upon stricter terms than a a mid-market hospital. A case study need not detail precisely which technologies a client used and how. Yes, we live in a transparent era, but it’s reasonable to expect organizations to maintain some semblance of privacy—or at least try.
• Case studies have to pass several levels of internal approval. Case studies aren’t just written willy-nilly. Senior execs at both the client and partner scrutinize every word. Don’t expect an honest and “objective” account of every challenge, political battle, and internal argument.
• Be wary of availability bias. You’re not going to find detailed stories on how an engagement blew up. Expect a certain level of marketing gloss. That’s not to say that all case studies need to be either saccharine or anodyne. Be wary of those that seem overly rosy. Does something seem to good to be true? It probably is.

In this way, vetting a consulting partner is much like vetting a PR firm.

Simon Says: Case Studies Are Necessary, Not Sufficient

Properly vetting a partner involves more than reading a case study. Change is tough and technology is inherently political. Sometimes new technologies fail to take root because of toxic organizational cultures, stubborn and change-resistant execs, and other issues that have nothing to do with the partner.

This post comes from IBM for MSPs. The opinions expressed here are my own.