How to run a security review of a data platform
A plain guide to reviewing a data platform with a security team: what to cover, what evidence to bring, and how to close what you find.
TL;DR
- Agree the scope first. Then cover each of the six areas, and bring current evidence for each.
- Use the vendor’s independent audit reports for what the vendor runs. Spend your own time on what you configure.
- A finding is closed when its fix is made, checked and recorded with the evidence. Writing it down does not close it.
- Keep the platform safe between reviews, with daily admin controls and controlled releases.
Before you start
I led a security review of our data platform with the company’s security team. This guide is what I learned.
A security review has one purpose: to show, with evidence, that the platform protects its data. It is not a form to fill in. It is a set of questions, and each answer needs proof.
Three groups take part:
- The platform owner knows how the platform is set up.
- The security team knows the questions and the standards.
- The data owners know who must see which data, and why.
Agree the scope first. Write down which systems are in the review, which are not, and why. A clear scope stops the review from growing without end.
The areas to cover
Our review had about 55 controls: the safeguards that protect the platform. They fell into six areas.
- Sign-in: how people prove who they are. Check the single sign-on setup, the rules for strong sign-in, and what happens when a person leaves.
- Access: who can see which data, and how that is set and checked. See the data protection model this review covered.
- Logging: what the platform records, how long it keeps the records, and who reads them.
- Encryption: how the platform protects data on disk and on the network.
- Network: what can reach the platform, and what the platform can reach. Check each outbound connection, and remove the ones nobody uses.
- Integrations: the other systems that connect to the platform, and the access each one has. Each one needs an owner.
For each area, know what to check and what proof to bring:
| Area | What to check | Evidence to bring |
|---|---|---|
| Sign-in | Single sign-on, strong sign-in, leavers | The settings export and a leaver test |
| Access | Roles, groups and data rules | A role export and a limited test user |
| Logging | What is logged, for how long, who reads it | The log settings and a sample search |
| Encryption | TLS on the network, encryption on disk, who holds the keys | The vendor's reports and your key settings |
| Network | Inbound and outbound connections | The allowed outbound list, each with an owner |
| Integrations | Each connected system and its access | A register of apps, with owners and scopes |
What the vendor covers and what you cover
The platform runs on a vendor’s service. The vendor controls its own data centres, staff and software. You cannot inspect those yourself.
You do not need to. The vendor has independent audit reports. An outside auditor checks the vendor’s controls and writes a report. Ask the vendor for these reports, and use them as evidence for the vendor’s part.
Your part is what you configure: your users, your roles, your data rules and your connections. The vendor’s reports do not cover these. This is where you spend your time.
What you configure You
- Users
- Roles
- Data rules
- Connections
Proof: your own exports and tests
What the vendor runs The vendor
- Data centres
- Staff
- Software
Proof: the vendor's independent audit reports
Read the vendor’s trust centre
Most vendors share their security documents in an online trust centre. You sign a non-disclosure agreement, and then you can download them. In our review, I used the vendor’s SOC 2 Type II and SOC 3 reports from its trust centre.
Know what each document proves before you rely on it:
| Document | What it proves | What to check |
|---|---|---|
| SOC 2 Type II report | An auditor tested the controls over a period | Dates, opinion, exceptions, your own duties |
| SOC 2 Type I report | The controls were designed well, on one day | Weaker than Type II: ask why there is no Type II |
| SOC 3 report | A short public summary of the SOC 2 result | Use it for a first look only |
| Penetration test summary | An outside team tried to break in | Date, scope, and serious findings fixed |
| ISO 27001 certificate | A security management system is in place | The scope covers the service you use |
| Encryption and data documents | How data is protected, and where it lives | TLS version, disk encryption, keys, subprocessors |
SOC 2 Type II: the main report
SOC stands for System and Organization Controls. An independent auditor writes the report. A Type II report tests the controls over a period, often six to twelve months. A Type I report looks at one day only, so it proves less.
Read these parts of the report:
- The period. Check that it ended recently. If there is a gap since then, ask for a bridge letter. In it, the vendor confirms that nothing important changed.
- The opinion. A clean opinion means the auditor found the controls worked. A qualified opinion means some did not. Read why.
- The exceptions. The auditor lists each test that failed. Check each one, and the vendor’s answer to it.
- Your own duties. The report lists the controls the vendor expects you to run, such as user access reviews. These belong in your part of the review.
The last part is easy to miss. It tells you exactly where the vendor’s proof stops and yours starts.
SOC 3: the short version
A SOC 3 report is a short summary of the same audit, made for the public. It gives the opinion but not the tests or the exceptions. Use it to start, then ask for the SOC 2 Type II report.
The penetration test summary
In a penetration test, an outside team attacks the service on purpose to find weak points. Vendors rarely share the full report. They share a summary or a letter from the testing firm.
Check three things. The test is recent, usually within the last year. The scope covers the service you use. Each serious finding is fixed, or has a date to fix it.
Encryption: on the network and on disk
Check how data is protected in two places:
- On the network. Connections must use Transport Layer Security (TLS), version 1.2 or later. Older versions have known weak points.
- On disk. Stored data must be encrypted, usually with AES-256. Check backups and logs too, not only the main data.
Then ask who holds the keys. With vendor-managed keys, the vendor controls them. With customer-managed keys, you control them, and you can revoke them. Also check how often the keys rotate.
Other documents to ask for
- The subprocessor list: the other companies that handle your data for the vendor.
- Data location: the regions where your data and its backups are stored.
- Incident response: how fast the vendor tells you about a breach, and how.
- Backup and recovery tests: proof that a restore works, not only that backups exist.
Evidence that holds up
Good evidence is current, complete and easy to check.
- Export the settings when you can. A screenshot shows one moment. An export shows the full list.
- Show that a rule works. Do not only show the rule. Sign in as a test user with limited access, and show what that user cannot see.
- Keep the evidence current. Settings change. Put a date on each piece of evidence, and plan when to check it again.
Changes need control too. In our platform, a second person approves each change before it reaches users. This gives the security review a clear record of who changed what.
Releases follow the same idea. A release moves from development to test to production. A script promotes it and records who did it. A check in CI stops a test version from reaching production.
Controls you run every day
A review checks the platform once. Daily admin keeps it safe between reviews. I run four admin controls on our platform:
- Data connections. Each connection to a source system is set up centrally, with an owner.
- Outbound network rules. The platform can reach only the addresses on an approved list.
- Third-party apps. Each outside app is registered, with only the access it needs.
- Projects and access. Each project has its own roles, and people get access through those roles.
One example shows why app registration matters. An app shown inside another page asked users to sign in twice. The fix was to enable it as a registered third-party app. I also checked the page’s content security rules. Some entries failed to match because of a missing trailing slash.
Closing access risks
The review is not the end. After it, the risk team still raises findings. For example, a person has access that their role does not need.
Each finding follows the same loop:
- Investigate. Find out what access the person has, and why.
- Agree a fix. Talk to the person’s manager. Agree what access the person needs.
- Make the change. Remove or change the access.
- Check it. Confirm that the change worked.
- Close it. Record the fix and the evidence.
- 1InvestigateWhat access, and why?
- 2Agree a fixWith the person's manager
- 3Make the changeRemove or change the access
- 4Check itConfirm the change worked
- 5Close itRecord the fix and the evidence
The next finding starts again at step 1.
Do not skip step 2. If you remove access without the manager, you can stop someone’s work. Then they ask for the access again, and the risk comes back.
What I’d do differently
Collect the evidence as you build, not at review time.
Most of the work in our review was finding proof for settings that were already correct. If each change had saved its own evidence, the review would have taken much less time.