Vanta and Drata automate infrastructure compliance. The 20% application testing gap is still manual screenshots. Here’s how to close it.
SOC 2’s Trust Services Criteria define dozens of controls, but only a handful require evidence that your application actually works as intended. The three that matter most for engineering teams are CC7.2 (Change Management), CC6.1 (Access Control), and CC8.1 (Monitoring). CC7.2 — Change Management — requires you to prove that code changes are tested before they reach production. The control asks: do you have a systematic process for validating changes? For infrastructure, Vanta checks that branch protection rules exist and PRs require approvals. But branch protection doesn’t prove the code was tested. It proves the PR was approved. The auditor wants to see that tests actually ran against the change and passed. That means test execution evidence tied to specific commits. CC6.1 — Access Control — requires proof that your application enforces authentication and authorization. Vanta can verify that Okta SSO is configured or that AWS IAM policies restrict access. But it can’t prove that your login page actually works, that RBAC rules are enforced in the UI, or that a deprovisioned user is truly locked out at the application layer. Those require functional tests that exercise the access control behavior. CC8.1 — Monitoring — requires evidence that you actively monitor system health and detect anomalies. Infrastructure monitoring and incident alerts cover part of this. But auditors increasingly ask: do you verify that your application’s core functionality is working? An API returning 200 OK doesn’t mean the checkout flow actually processes payments. Application-level health verification — real user flows running continuously — fills this gap. The common thread: infrastructure compliance tools cover configuration-level evidence. They confirm that controls are set up. But they can’t confirm that controls work at the application layer. “The checkout flow works on this commit” is a statement about application behavior, not infrastructure configuration. That’s the evidence gap.
Evidence format matters more than most engineering teams think. Auditors are not engineers — they’re evaluating whether your controls are effective based on documentation they can independently verify. The gold standard is evidence that is timestamped, immutable, and traceable to a specific system state. Screenshots have been the traditional format for application testing evidence. A screenshot showing a passed test suite with a visible timestamp and commit SHA is universally accepted. The problem is collection: someone has to take the screenshot, label it with the control ID, link it to the commit, and store it in the evidence repository. Multiply that by dozens of controls and quarterly audit windows, and you’re looking at days of manual labor. Structured artifacts - JSON run details plus screenshot, recording, and trace links - are easier to verify than screenshots alone. Zerocheck's current product output is run data and browser artifacts for executed tests, not packaged audit reports or built-in SOC 2 mapping. CI logs are not sufficient. They expire (GitHub Actions logs are deleted after 90 days by default), they’re not formatted for audit review, and they don’t map to control IDs. An auditor handed a raw CI log has to manually search for relevant test results, match them to controls, and verify timestamps — work they shouldn’t have to do and often won’t. The key principle: evidence must be immutable and traceable. Once generated, it shouldn’t change. It should link directly to the code state (commit SHA) and the control it demonstrates (control ID). If an auditor can follow the chain from control requirement to test result to code change without asking you for help, your evidence is good.
Vanta, Drata, and Sprinto have transformed SOC 2 compliance by automating roughly 80% of evidence collection. They pull infrastructure configurations from AWS, GCP, Azure, Okta, GitHub, and dozens of other integrations to prove that security controls are in place. IAM policies are correct. Branch protection is enabled. Vulnerability scans are running. Access reviews are completed. This is genuinely valuable work that used to consume weeks of manual effort. But these platforms have a fundamental limitation: they operate at the infrastructure and configuration layer. They can verify that MFA is enabled in Okta. They cannot verify that MFA actually works when a user logs into your application. They can confirm that GitHub branch protection requires PR reviews. They cannot confirm that your test suite actually runs and passes on those PRs. They can check that Datadog alerts are configured. They cannot check that your checkout flow processes payments correctly. This is the Vanta gap: the 20% of SOC 2 evidence that requires testing your application’s actual behavior. And ironically, this 20% is where teams spend the most manual hours. Infrastructure evidence is automated. Application evidence is still screenshots in a Google Doc. As Thomas Ptacek of Fly.io observed: “The guts of a SOC 2 audit are a giant spreadsheet questionnaire and a battery of screenshots.” Those screenshots are overwhelmingly application-level evidence — proof that the thing works, not just that it’s configured. The teams that struggle most with SOC 2 aren’t the ones missing Vanta. They’re the ones who have Vanta and assume compliance is handled. When the auditor asks for proof that access controls work at the application layer, or that code changes are tested before deployment, Vanta’s dashboard shows green checkmarks for the infrastructure side. The application side is a blank. That blank is where 2 engineers spend 2 weeks taking screenshots, mapping them to controls, and assembling evidence packets — every single audit cycle.
Mapping evidence to audit requirements is still a compliance workflow, not a hidden Zerocheck feature. The useful pattern is simple: keep a small table that links approved test names to the controls they support, then attach Zerocheck run JSON and artifact links when those tests execute. A login test can support CC6.1. A PR run with approved blocking tests can support CC7.2. A production monitor run for a critical flow can support CC8.1. Zerocheck gives you the execution proof: what ran, when it ran, what passed or failed, and what the browser saw. Your compliance system or evidence owner maps that proof to the control language your auditor expects. That distinction matters. It is credible to say Zerocheck produces raw application-level evidence. It is not credible to say V1 automatically organizes evidence by SOC 2 control or exports auditor-ready audit packets.
# compliance-map.yml - maintained outside Zerocheck today
controls:
CC6.1:
tests:
- Login with valid credentials
- RBAC: non-admin rejected from admin panel
CC7.2:
tests:
- Approved suite ran on pull request
CC8.1:
tests:
- Production monitor: checkout smoke
evidence:
source: Zerocheck run JSON and artifact links
owner: compliance workspaceThe evidence is timestamped test results with screenshots and step traces — the same thing you’d collect manually. If AI suggested the test, a human approves it before it becomes run evidence. The proof is factual output from execution.
Vanta automates infrastructure evidence: access reviews, config checks, vulnerability scans. It can’t prove your login flow works, your checkout processes payments correctly, or your access controls enforce RBAC at the application layer. That’s the 20% gap.
Zerocheck does not currently map tests to SOC 2 controls. Export JSON run evidence, then map it to controls in your compliance process as needed.
Yes. Start by discovering critical flows, review the suggested tests, and approve the ones that should run. Evidence starts accumulating once approved tests execute on PRs or monitors.
Skip the setup. Zerocheck handles it in plain English.
Get a demo