Skip to content
Compliance

SOC 2 and penetration testing

The Trust Services Criteria never say the words you are expecting. No criterion mandates a penetration test, and yet almost every service organisation with a SOC 2 report has one. This explains why, what auditors actually ask for, and when in the cycle to run it.

9 min read

Key takeaways

  • No Trust Services Criterion requires a penetration test. The criteria are written as outcomes, and testing is one of the ways an organisation demonstrates it meets them.
  • In practice auditors commonly accept a penetration test as evidence supporting CC4.1, which covers evaluations of internal control, and CC7.1, which covers detecting vulnerabilities.
  • For a Type 2 report the test needs to fall inside the observation window, not just inside the calendar year.
  • What the auditor looks at is scope, date, independence, methodology, findings and what happened to them afterwards. A clean report with no evidence of remediation is weaker than a report with findings that were closed.
  • A penetration test is evidence that may support an examination. It is not a certification and it does not make an organisation SOC 2 compliant on its own.

What SOC 2 actually says about penetration testing

SOC 2 is an examination performed by a licensed CPA firm against the AICPA Trust Services Criteria. The criteria are deliberately written as outcomes rather than as a control checklist, because they have to apply to a payroll provider, a hosting company and a medical records platform without being rewritten for each. That design is why no criterion says an organisation must commission an annual penetration test. It is also why the question of whether you need one has to be answered by looking at what the criteria ask you to demonstrate.

Alongside each criterion the AICPA publishes points of focus, which are illustrative considerations rather than requirements. Penetration testing appears in that supporting material as one example of the sort of evaluation an entity might perform. Points of focus are explicitly not a checklist an auditor scores you against, and an auditor cannot fail you for skipping one. What they do is tell you what the criterion has in mind, which is usually enough to settle the argument internally.

The accurate way to say it

A penetration test is not required by SOC 2. It is the most common form of evidence organisations use to demonstrate that vulnerabilities are identified and evaluated, and most auditors expect to see one or an explanation of what you do instead. Those are different statements and the difference matters when somebody senior asks whether the spend is mandatory.

Where a test lands in the criteria

Two of the common criteria are where testing evidence is normally applied. Both sit in the security category, which is the one every SOC 2 report includes.

Trust Services Criteria commonly supported by penetration testing evidence
CriterionWhat it addressesHow a test supports it
CC4.1Performing ongoing and separate evaluations to establish whether the components of internal control are present and functioning.An independent test performed at a point in time is a separate evaluation. It produces a dated, scoped assessment by somebody outside the control owner, which is exactly the shape of evidence this criterion contemplates.
CC7.1Using detection and monitoring procedures to identify configuration changes that introduce vulnerabilities, and susceptibility to newly discovered vulnerabilities.Testing identifies vulnerabilities in the environment as it stands. It is usually presented alongside vulnerability scanning and patch management, because the criterion is about ongoing detection and a point-in-time test alone does not demonstrate ongoing.
CC7.2 and CC7.3Monitoring for anomalies and evaluating security events.Relevant only where the engagement was designed to exercise detection, such as a red team assessment. A standard application test says little about your monitoring.
CC8.1Change management, including authorising and testing changes.Testing performed after a significant change, and evidence that it happened as part of the change process, supports this rather than a single annual test does.

CC7.1 is worth reading carefully, because it is the one people over-claim against. It is about detection over time. A single annual test does not on its own demonstrate ongoing detection, which is why the strong answer pairs the test with regular vulnerability scanning and a patch process. Our platform includes self-service scanning between engagements, kept separate from the findings a tester wrote, so it is clear to an auditor which evidence came from which activity.

Type 1 and Type 2, and why the difference decides your timing

How the two report types differ
Type 1Type 2
What is examinedWhether controls are suitably designed as at a specified date.Whether controls are suitably designed and operated effectively throughout a period.
Period coveredA single point in time.An observation window, commonly three to twelve months.
What testing evidence is expectedThat the control exists and is designed to work. A documented policy and a scheduled test can be enough.That the control actually operated during the window. The test needs to have happened inside it.
Typical useA first report, produced quickly to answer a customer that cannot wait.What enterprise buyers and most procurement processes ask for after the first year.

The practical consequence is straightforward and regularly missed. For a Type 2 report the test must fall inside the observation window. A test finished the month before the window opened demonstrates nothing about whether the control operated during it, and an auditor is right to say so. If your window runs from January to June, the test needs to be in that period, and it needs to be far enough inside it that any remediation and verification also land before fieldwork.

When to run the test

  1. Early in the observation window rather than at the end. Findings need time to be fixed, and fixes need time to be verified.
  2. With at least four to six weeks between the test finishing and the start of audit fieldwork. That is roughly what it takes to remediate a moderate list and have the fixes checked.
  3. After any significant change to architecture, authentication or the public surface, if that change falls inside the window. This is also what CC8.1 has in mind.
  4. Annually as a baseline, because that is the cadence auditors and enterprise customers are used to seeing and the one your next customer questionnaire will ask about.

The failure pattern is always the same: the test is booked for the last month of the window, findings arrive, there is no time to fix them, and the report goes to the auditor with open issues in it. That is not fatal, but it means the conversation is about your remediation plan rather than about your closed findings. Because our engagements run five to ten working days and retests are included rather than quoted separately, the whole find-fix-verify cycle fits comfortably inside a normal window if it is started with time in hand.

What the auditor actually asks for

Auditors are consistent about this. They are establishing that an evaluation happened, that it was independent, that it covered the system in the report, and that the organisation did something about what it found.

  • The report itself, with dates. Both the testing dates and the report date, because they are frequently weeks apart and the testing date is the one that counts against the window.
  • The scope, written clearly enough that the auditor can see it covers the system described in the SOC 2 report. A test of a marketing site does not evidence anything about the product.
  • Who performed it, and evidence that they were independent of the people who built and operate the system. An external firm answers this immediately; an internal team needs an organisational separation you can explain.
  • The methodology, or at least a statement of what standard the work followed.
  • The findings, with severities and enough detail to show they were real.
  • What happened next: the remediation record, the dates, and evidence of verification.
  • Where a finding was accepted rather than fixed, a documented risk acceptance with a named owner and a date.

A report with no findings is not automatically the best outcome

Auditors see a lot of reports. One with several findings, all remediated and verified within a sensible period, demonstrates the control operating. One with nothing at all raises a fair question about scope and depth, particularly if the scope statement is vague. The evidence you are producing is that the process works, not that the system was perfect.

What the report needs to contain

A report that satisfies an auditor and a report that helps your engineers are not quite the same document, which is why our platform produces several profiles from the same engagement rather than one file that tries to serve everybody. The SOC 2 profile groups findings under the Trust Services Criteria and includes the caveat about what a test is and is not, and there is a fuller description of the report profiles if you want to see what comes out.

The parts an auditor reads

  • Scope statement: the systems, environments and applications examined, and what was excluded.
  • Testing dates and report date, stated separately.
  • The testing organisation and the basis for its independence.
  • Methodology and the standards followed.
  • A findings summary with severities.
  • Remediation and retest status per finding, or a companion document carrying it.

The parts your engineers read

  • Reproduction steps precise enough to follow without asking anyone.
  • The request and response, or equivalent evidence, that demonstrates the issue.
  • Affected endpoints or components, named exactly.
  • Remediation guidance specific to the finding rather than a generic reference.

Remediation evidence, and what closed means

This is where most SOC 2 preparation is weakest. Organisations produce a good test report and then evidence the remediation with a spreadsheet somebody maintained by hand, in which every row says fixed and no row says who checked. An auditor is entitled to ask how you know.

The strong version of this evidence has three properties. It is dated, so the timeline between discovery and correction is visible. It attributes the verdict to somebody, so closure is a decision rather than an assertion. And where the finding was verified rather than merely reported fixed, it says who verified it and when. On our engagements you mark a finding fixed and request a retest from the same screen, a tester rules on it by hand, and every transition is written to an append-only audit trail with the actor and the time.

Findings you are not going to fix need the same rigour in the other direction. A documented risk acceptance with a named owner, a stated rationale and a review date is a perfectly respectable answer. An undocumented decision to ignore something is the answer that causes trouble, because from the outside it is indistinguishable from having missed it. There is more on doing this properly in the guide to reading a penetration test report.

Scoping a test for a SOC 2 examination

Scope the test to the system described in your SOC 2 report. That sounds obvious and it is regularly got wrong, usually by testing the customer-facing application and omitting the administrative interface, the internal tooling with production access, or the infrastructure the service runs on. If it is in the system description, an auditor may reasonably expect it to be in scope for the evaluation.

  • The production application and its API, across every role that exists.
  • The administrative interface and any internal tool with access to customer data.
  • The authentication path, including single sign-on integrations, invitations and account recovery.
  • The cloud configuration and identity model where the service runs, which is often assessed as a separate piece of work.
  • Tenant isolation, if you are multi-tenant. It is the finding your customers care about most.

If you are a software company approaching this for the first time, the sector page on penetration testing for SaaS platforms covers the surface in more detail, and the guide to scoping an engagement covers the practical decisions. If ISO 27001 is also in your future, scope both at once: the underlying testing is the same work and only the reporting differs.

FAQ

Questions this raises

Is a penetration test required for SOC 2?

No. No Trust Services Criterion mandates one. The criteria describe outcomes, and a penetration test is one of the ways organisations demonstrate that they evaluate their controls and identify vulnerabilities. In practice most auditors expect to see either a test or a clear explanation of what you do instead, and most service organisations find that commissioning one is simpler than defending the alternative to every customer who asks.

Which criteria does a penetration test map to?

Most commonly CC4.1, which covers ongoing and separate evaluations of internal control, and CC7.1, which covers detecting configuration changes that introduce vulnerabilities and susceptibility to newly discovered ones. Testing after a significant change can also support CC8.1 on change management. Your auditor decides what they accept and how they map it, so confirm with them rather than assuming.

Does the test have to be inside the observation window?

For a Type 2 report, yes in practice. The examination covers whether controls operated throughout a period, and a test performed outside that period does not evidence anything about it. For a Type 1, which assesses design at a point in time, a documented process and a scheduled test can be sufficient. Ask your auditor early, because the answer determines when you book the test.

Can we use an internal team instead of an external firm?

It is possible where you can demonstrate genuine independence between the testers and the people who build and run the system, and where the testers are qualified. Many organisations find that harder to evidence than simply engaging an external firm, and enterprise customers reviewing your report tend to prefer an independent third party regardless of what the auditor accepts.

Will open findings in the report cause a qualification?

Not by themselves. Auditors are assessing whether the control operates, not whether the system is perfect. What matters is that findings were identified, assessed against your own risk process, and either remediated within a reasonable period or formally accepted with a rationale and an owner. Findings left open with no record of a decision are the ones that create difficulty.

How often do we need to test for SOC 2?

The criteria do not state a frequency. Annually is the near-universal practice, plus testing after significant changes, and it aligns with the observation window most organisations use. If your product changes rapidly you may want to test more often, and pairing an annual test with continuous scanning is the usual way to evidence the ongoing part of CC7.1.