evaluating a vendor's SOC-2 report
the journal of Michael Werneburg
twenty-seven years and one million words
Someone asked me how to evaluate a service organization's SOC-2 report. Here's what I told them. This describes the situation in Canada today (which should be partially obsoleted when Section 3000 comes online in 2017-2018, but just sub in 3000 where 5025 appears below).
SOC 2, instantiated in Canada for now as CSAE Section 5025. Watch for a Type II, a Type I is a point in time and is of dramatically less value. Look for full year coverage. Moreover, on the first page of the “Report of the Independent Auditors”, you’ll note carve outs for major participants (e.g. PortfolioAid’s Scalar). You’ll want to hear that each of those carved out parties provides a report, too, because they’ll have your data. Under “Opinion”, you’ll see three paragraphs. They should say: “this description fairly represents the aforementioned” (scope) and the period (again, one year); that the controls were suitably designed (again, referencing the date); and that the control objectives were achieved (again, date).
If you’re feeling thorough, read section 5 on the description of the control environment, look for things like an org chart and some semblance of overall coherence. Everything should follow from the sort of activities necessary to conduct the service they’re providing.
Section 6 must accurately describe the system. I don’t know why it’s called “Description of Controls”, but make sure it’s on scope for the service you’re acquiring.
Section 7 is the long list of controls, watch for anything like “exceptions noted”. If there are *any*, you should be able to learn at once how it’s being addressed. I’ve seen the remediation plans drafted into the control evaluation, but you should probably be able to hear from someone on their end about the current project in place to fix the problem, and it should sound credible.
SAS 70 is long since obsolete. CSAE 3416 is about controls on financial reporting. Because that was the channel that inherited Section 5970 (our SAS 70), some clients (and auditors!) think that that’s what they should be issuing. The rules that govern 3416 are different in ways I don’t fully understand, but my guess is they can be leveraged to provide a less meaningful report. Lord knows 5970 was.
The various authorities rattle their sabers about it, but I’ve no idea how the practice can be stopped. Anyone who issues a 3416 on ops controls is just giving the clients what they think they want, and probably don’t do that kind of service audit for a living.