A penetration test buys you a dated, bounded answer. When the report lands, you can say that on the days the testers worked inside the agreed boundary, the tested controls were configured in line with good practice and the tested components carried no common or publicly known vulnerabilities the team could find. That is a real, useful, defensible statement. It is also the whole of what the exercise delivers.
It does not tell you the organisation is secure, and it does not tell you what changed the following Monday.
The NCSC defines penetration testing as a method for gaining assurance in the security of an IT system by attempting to breach some or all of that system’s security, using the same tools and techniques an adversary might use. The same guidance places testing inside your vulnerability assessment and management process, not at the front of it as the primary way you find vulnerabilities, and it describes the work as powerful but expensive with a blunt warning that it is no magic bullet.
NIST’s glossary entry agrees on the mechanics. Evaluators mimic real-world attacks, often issuing real attacks on real systems and data, and most tests look for combinations of vulnerabilities rather than single defects.
That combination of real attacks, real data and a bounded window is why scoping carries more weight here than in almost any other security purchase. A badly scoped test produces a report nobody can act on. An unscoped one can create a legal problem.
Quick Take
- Decide what you need to prove before you shop. If you cannot finish the sentence “after this test we will be able to say…”, you are not ready to buy.
- Authorisation is the first deliverable. A signed scope that names systems, windows and contacts protects the tester as much as it protects you.
- The approach follows the question. Open-box, closed-box, scenario-driven and detection-and-response tests prove different things, so choose deliberately.
- Budget for the fix, not only the test. Findings that sit in a PDF change nothing, and a scheduled test does not replace routine security checks.
Where Penetration Testing Fits, and What to Confirm Before You Buy
Info
Penetration testing is an authorised, adversary-style security test in which assessors attempt to circumvent or defeat the security features of a system, network, or application. It is an assurance activity that sits alongside your vulnerability management programme, and it is not a primary method for identifying vulnerabilities.
Read the NCSC’s audit analogy and the buying decision gets easier. Your finance team tracks income and expenditure every day; an external audit checks whether those internal processes are sound. Penetration testing does the same job for security. You should have a decent idea of what the testers will find before they find it, because your own scanning, patching and configuration management should already be surfacing the obvious. Third-party testers verify that expectation, and occasionally a highly experienced team catches something subtle your internal process missed.
Four boundaries keep that definition honest:
- Routine security testing continues regardless. The NCSC is direct that a planned penetration test does not mean your normal testing regime stops including security tests. Checking that defined controls function, such as the login box appearing or a wrong password being rejected, is not a valuable use of penetration-testing resources.
- Product-specific testing is out of scope. The same guidance states that penetration testing is not an appropriate technique for testing a specific product. It suits an operational system assembled from multiple vendors, or something you built in-house and now run.
- Vulnerability assessment is a different deliverable. Automated scanning inventories possible weaknesses at volume. A penetration test spends skilled human time chaining weaknesses together to see whether access is actually achievable.
- Frameworks are reference layers. MITRE ATT&CK is a free knowledge base of adversary tactics and techniques used for threat modelling. The OWASP Web Security Testing Guide is a web-application testing resource. Neither is an assurance standard, an authorisation to test, or a scoping document.
Before the eight commissioning steps, confirm four prerequisites. They are the difference between an authorised test and an incident.
Prerequisites
- Written authorisation from whoever owns the systems, including any third-party or cloud host whose terms apply.
- A working internal vulnerability assessment and management process, so the test verifies something instead of substituting for it.
- Named risk owners, technical staff who know the target, and a representative of the testing team available for scoping.
- An agreed severity scale and a route for escalating a critical finding mid-engagement.
Commission a Penetration Test in Eight Ordered Steps
A penetration test follows a recognisable shape: planning and conducting the test, analysing the findings, and developing mitigation strategies. Work through these in sequence. Skipping step two is the failure mode that turns a security exercise into an incident.
- Write the assurance question down. Name the decision the report will inform, whether that is a launch, a migration, a supplier review or a board assurance statement. If the question is “are we secure”, narrow it until it names a system, a control set and a point in time.
- Get a signed Rules of Engagement. Both parties sign a document describing the scope of the engagement, which is the practice CISA describes on its penetration-testing service page. Confirm in writing which systems are excluded before any traffic is sent.
- Decide how much the testers are told. Open-box testing shares full information about the target; closed-box shares none of the internals and works from an external perspective. The choice changes what the report can prove.
- Decide whether a scenario drives the test. If a specific incident type worries you, such as a lost laptop, an unauthorised device on the internal network or a compromised DMZ host, say so, and pick scenarios from your own incident history.
- Engage qualified, experienced testers. The NCSC states that third-party tests should be performed by qualified and experienced staff only, because tests cannot be fully procedural and quality tracks tester ability. For UK government organisations it recommends the CHECK scheme.
- Book the window and name a technical point of contact. CISA’s service page describes tests running only during agreed scheduled times on pre-determined systems, with the tester in constant communication with that contact throughout.
- Require evidence of any successful penetration. A placed file, a screenshot or another artefact you can inspect separates a confirmed compromise from a theoretical one, and it gives remediation work something concrete to close against.
- Analyse the findings and plan the mitigation. Triage by severity and business impact, assign owners, and schedule retesting for anything fixed. The aim is to improve the internal vulnerability process the test was checking.
What that looks like in practice: a worked scope for a fictional booking portal
ExampleCo is a made-up company, used here to show the shape of a finished scope rather than a template to copy word for word. It runs a customer-facing booking portal and an API behind it, and it wants more than a scan before a peak season.
- The assurance question. “After this test we will be able to say that an unauthenticated attacker on the internet cannot reach customer booking records through the public portal or its API.”
- In scope. The public portal, the version 2 API endpoints, the customer login, and the staging copy of the portal at the address written into the scope document.
- Out of scope. The payment provider’s hosted checkout, the shared hosting control panel, staff laptops and the production database server itself. The exclusions are written down and signed, not assumed.
- Approach. Closed-box against the public surface, because the question is about an outsider with no documentation. Open-box against the API with test credentials, because the same question asks whether the internal review process catches API misconfiguration.
- Window and contact. Testing between 22:00 and 06:00 on the agreed dates, with a named engineer reachable throughout and an escalation route to the incident lead for anything critical found mid-engagement.
- Evidence and close-out. Confirmed compromises evidenced by file placement or screenshots, findings rated on the agreed severity scale, owners named, and a retest booked for anything rated high.
The value of that brief is not its length. Every line can be checked, argued with and signed before a single packet leaves the tester’s machine.
Four Reference Points Worth Keeping Open
These are the documents worth keeping open while you brief a supplier, draft a scope or read a report. Each answers a different question, and each carries a limit you should know before you lean on it.
| Resource | Question it answers | Best fit for | Trade-off or limit |
|---|---|---|---|
| NCSC penetration testing guidance | What can this exercise actually prove, and how should it be commissioned? | Decision-makers and risk owners | UK-focused guidance; offers no price and states plainly that testing is expensive |
| NIST SP 800-115 | How do we plan, conduct, analyse and mitigate in a repeatable technical process? | Security teams building a testing programme | Published September 2008; a technical guide, explicitly not a comprehensive testing programme |
| OWASP Web Security Testing Guide | What does structured web-application security testing cover? | Web and application security teams | Web-application focus; v4.2 is the available release with v5.0 in development, and the project page does not detail procedures |
| MITRE ATT&CK | Which adversary tactics and techniques should we be modelling against? | Threat-informed defence and detection mapping | A free knowledge base; not an assurance standard, scoping document or testing methodology |
NCSC guidance: the commissioning backbone
If you read one thing, read this. One page covers the definition, the assurance framing, the open-box and closed-box trade-offs, the tester-quality requirement and the point-in-time limitation. Use it when you need to explain to a budget holder why a test is not a security guarantee. Trade-off: it is UK government guidance aimed at organisations commissioning tests, and it will not give you a price or a legal opinion.
NIST SP 800-115: the process document
This is the technical spine: planning and conducting tests, analysing findings, and developing mitigation strategies, with an emphasis on specific techniques and their limitations. It supersedes SP 800-42 and dates from September 2008, which shows in places, but the workflow holds up. Use it when your team needs a repeatable procedure instead of a principles statement. Trade-off: by its own description it is an overview of key elements, not a full testing and examination programme.
OWASP WSTG: the web-application reference
Version 4.2 is available as a web-hosted release and PDF, with version 5.0 in development. Use it when the scope is a web application and the team needs a testing taxonomy. Trade-off: it is a web-testing resource; it does not authorise testing, define your scope, or establish the efficacy of a penetration test.
MITRE ATT&CK: the adversary map
A free, globally accessible knowledge base of adversary tactics and techniques drawn from real-world observations, used to build threat models and methodologies. Use it when you want to describe what a test did or did not exercise in terms of adversary behaviour. Trade-off: ATT&CK describes behaviour; it says nothing about whether your controls were configured well.
Choosing an Approach: Open-Box, Closed-Box, Scenario-Driven, Detection and Response
The approach determines the assurance question. Pick it deliberately, and write the choice into the Rules of Engagement.
| Approach | What testers are given | Assurance it supports | Main blind spot |
|---|---|---|---|
| Open-box (transparent) | Full information about the target | Confirms the efficacy of internal vulnerability assessment and management controls by finding known software vulnerabilities and common misconfigurations | Does not model an unknown external attacker as directly |
| Closed-box (opaque) | No internal information | External perspective on ways to reach internal IT assets, more closely modelling attackers who are unknown or unaffiliated to you | Missing information can leave vulnerabilities undiscovered within the time allocated |
| Scenario-driven (vulnerability) | An agreed scenario, such as a lost laptop, an unauthorised device on the internal network, or a compromised DMZ host | Whether that specific path leads to a vulnerability in your defences | Illustrative examples; a scenario set is never comprehensive |
| Scenario-driven (detection and response) | A scenario plus an explicit aim of gauging detection and response | Efficacy and coverage of your detection and response capability in that scenario | Measures response, not the breadth of your vulnerability exposure |
Which approach fits your question
Use these rules against the assurance question you wrote in step one.
- Choose open-box when you want a verdict on your own patching and configuration discipline, and your team already scans the estate. The trade-off is that you learn less about how an outsider would approach you.
- Choose closed-box when the board or a customer wants an external-attacker view of what is reachable from the internet. Accept that time-boxed discovery may miss things, and say so in the report’s own terms.
- Choose scenario-driven vulnerability testing when a specific incident type keeps you awake, and pick scenarios from your own incident history instead of a generic list.
- Choose detection and response testing when the question is “would we notice and react”, not “what is exposed”. It is a different question with a different report.
- Pair open-box and closed-box when you need both answers and can fund two engagements. Comparing the two reports is often more informative than either alone; see internal and external network penetration testing for how that split works in practice.
What the Results Can Show and What They Miss
A well-scoped test can give genuine confidence that the products and security controls tested were configured in accordance with good practice, and that no common or publicly known vulnerabilities existed in the tested components at the time of the test. Read that sentence carefully, because every qualifier is load-bearing.
What it misses is structural:
- It is dated. The test validates that your systems were not vulnerable to known issues on the day of the test. NCSC guidance notes that a year or more between tests is not uncommon, so if this is your only validation method, vulnerabilities can sit there for a long time.
- It covers tested components only. Systems outside the agreed scope were never in the room.
- It covers known and common issues. A novel flaw, a business-logic abuse case nobody thought to model, or a configuration that is technically fine but operationally wrong may survive the exercise.
- Closed-box work is time-boxed. A lack of information can leave vulnerabilities undiscovered within the time allocated, which is a limit of the approach and not a failure of the tester.
- It is the wrong tool for product-specific testing, where you need a different assurance method entirely.
- It does not replace routine testing. Functional checks of security controls belong in your normal testing plan.
Warning
No penetration test establishes that a system is safe, that a breach cannot happen, or that every gap has been found. Treat any report summary that implies otherwise as a warning sign about the supplier’s judgement. If authorisation, scope or system criticality is unclear at any point, stop and escalate to legal and security leadership before testing proceeds. Systems you do not own, or have no written authorisation to test, are out of scope by definition.
Practical Scoping Checklist and Red Flags
Verify the engagement on paper before anyone tests
- The scope document names the in-scope systems and the exclusions, and both parties have signed it.
- The assurance question fits in one sentence, and the chosen approach visibly answers it.
- The scheduled windows on the calendar match the pre-determined systems in the scope.
- A named technical point of contact is reachable throughout, with an escalation route agreed in advance.
- The contract requires inspectable evidence of any successful penetration instead of a narrative claim.
- Every finding will carry a severity rating and a named internal owner for remediation.
- Routine security testing on the target system is still scheduled after the engagement ends.
Red flags worth walking away from: a proposal with no written scope, a fixed price quoted before anyone asked what you need to prove, a tester who cannot describe the difference between open-box and closed-box work, a report template with no severity scale, and any summary promising that you will not be breached. The last one is the clearest signal, because a credible testing team should not make that claim, and the point-in-time framing rules it out.
If your estate includes cloud infrastructure, scope the provider boundary explicitly. Cloud platforms publish their own testing policies and authorisation requirements, and a test that ignores them can breach terms of service even when your own Rules of Engagement looks complete. A working starting point is this AWS penetration testing tools and checklist, which you can adapt into the scope annex instead of treating it as authorisation in itself.
Conclusion: Buy Verification, Not Volume
Penetration testing is a verification purchase. It earns its place when the report will settle a decision you can name, for a system you can describe, with a remedy you are prepared to fund. If those conditions are missing, more scanning, better patching, or a tighter internal assessment process will usually do more for the money.
The strongest report is not the longest one. It is the one where the scope, the approach and the evidence line up so a risk owner can act without translating the document. That is the standard to hold the supplier to, and it is the reason to walk away from a broad test that answers no particular question.
If you are buying this as a package, ask how the provider separates automated vulnerability scanning from manual adversary-style testing, because the confidence you can take from the report depends on which deliverable produced each finding. Suppliers often make the case for the service plainly, saying that VAPT is an essential cybersecurity practice. That is the pitch. The split between scanning and manual testing is the proof, and it should be in writing before you sign.
Finally, have anyone who drafts or approves authorisation and scope language review it with qualified legal or security counsel for your jurisdiction. That work sits outside what any guidance page can settle for you.
💬 Comments