There is a framing I want to get out of the way before making the main argument of this post, because it tends to derail the conversation before it gets useful. The claim is not that compliance is bad or that frameworks like NIST CSF, ISO 27001, PCI DSS, and SOC 2 are worthless. They are not. Compliance frameworks standardize controls, create organizational accountability, establish minimum baselines, and provide a common language for discussing security posture across teams and with external stakeholders. These are real and valuable contributions. The problem is not the frameworks themselves. The problem is the way compliance scores and audit results have been elevated into measures of security effectiveness that they were never designed to provide, and the way security programs have organized themselves around achieving that score rather than building genuine security capability.
I have watched this dynamic play out across multiple organizations in multiple industries. The compliance program fires up. The framework is selected. The controls are mapped. The evidence is collected. The audit passes. The score goes up. Leadership sees a green dashboard and concludes that the security program is working. And then the breach happens, in an area the framework addressed on paper but the organization never actually implemented with operational depth.
What Compliance Scores Actually Measure
Compliance frameworks measure the existence and documentation of controls. They do not, in most cases, measure whether those controls are working effectively in the real environment against real threats. This distinction is the core of the compliance score problem, and it is worth being technically precise about it.
When an auditor assesses whether an organization has an access control policy, they are verifying that the policy exists, that it has been approved by appropriate stakeholders, and that it has been communicated to relevant personnel. They are not typically running a technical test to confirm that the policy is being enforced at the system level, that exceptions have not created unmonitored pathways, or that the implementation has not drifted since the last audit cycle. Control existence and control effectiveness are different things, and compliance frameworks are primarily designed to measure the former. [1]
The gap between those two columns is where organizations get breached. And the compliance score, which measures the left column, will look perfectly healthy right up until the moment something in the right column fails catastrophically.
The Numbers Behind the Gap
This is not a theoretical concern. The data on the relationship between compliance posture and actual breach exposure is sobering in a very specific way: organizations are complying and getting breached anyway, at a rate that should prompt a serious re-examination of what the compliance program is actually producing.
That second number should stop anyone in their tracks. Only 37% of compliance leaders feel fully confident they can assess whether their compliance program is actually effective. [2] The people running the programs that generate the scores are themselves uncertain about whether those scores reflect real security capability. That is a significant admission about the state of compliance measurement in the industry, and it points directly to the structural problem: compliance programs are often better at producing evidence of control existence than at producing honest assessments of control effectiveness.
How Compliance Programs Drift Towards Theater
The drift from genuine security investment to compliance theater does not usually happen through bad intentions. It happens through a set of rational responses to the incentives the compliance program creates. Once passing the audit becomes the goal, behavior naturally organizes around the audit requirements rather than around the underlying security objective those requirements were intended to serve.
Training completion rates become the metric rather than behavioral change. Logging is enabled to satisfy the requirement rather than tuned to produce actionable intelligence. Incident response plans are documented but never tested, because testing reveals gaps and documentation does not. Vulnerability scans are run to produce evidence of scanning, not to drive risk-based remediation. Each of these patterns produces a higher compliance score while making the security program, in operational terms, weaker than the score suggests. [3]
There is also a timing problem baked into the compliance model itself. Most compliance assessments are point-in-time events. An annual audit verifies that controls were in place and documented at the moment of assessment. It does not verify that they remained in place and effective throughout the intervening year. Security environments change continuously: new cloud workloads come online, configurations drift, staff turn over, detection rules degrade, and the threat landscape shifts. A compliance score earned in January does not tell you anything reliable about the security posture in July. Continuous monitoring of control effectiveness, which some frameworks are beginning to require but most organizations are still building toward, is the operational capability that closes this gap. Without it, the compliance score is a historical artifact rather than a current measure. [4]
Using Compliance as the Floor It Should Be
The practical alternative to treating compliance scores as security effectiveness measures is not to abandon compliance programs. It is to reposition them correctly within the broader measurement architecture. Compliance is a floor, not a ceiling. It establishes the minimum acceptable baseline of control existence across a defined framework. Treating it as anything more than that leads to misplaced confidence. Treating it as anything less creates regulatory and legal exposure that no security program can afford to ignore.
The measurement that belongs above the compliance floor is control effectiveness testing. This means going beyond evidence of control existence to validate that controls are actually functioning as designed under realistic conditions. For technical controls, this looks like continuous automated validation: configuration monitoring that flags drift in real time, detection rule testing against live telemetry, and regular adversary simulation that confirms defenses fire when they should. For process controls, it looks like tabletop exercises, red team engagements, and post-incident analysis that examines whether the processes documented in the compliance program actually held up under operational pressure.
The organizations that navigate this most successfully are the ones that maintain two parallel tracking systems: a compliance posture view that monitors framework adherence and audit readiness, and an operational security effectiveness view that monitors whether controls are actually working in the current environment. These two views will sometimes show very different pictures of the same program, and that difference is itself important information. A high compliance score paired with low operational effectiveness ratings is a signal that the compliance program has optimized for the audit rather than for the security outcome. A lower compliance score with strong operational effectiveness may indicate that the organization has prioritized operational security depth over documentation completeness—which is a defensible choice in many contexts, but one that carries its own regulatory risk.
Compliance programs are necessary. The frameworks they implement are largely well-designed and built on hard-won industry experience. The compliance score problem is not a problem with the frameworks. It is a problem with what organizations do with the score once they have it. Using it as a proxy for security effectiveness, presenting it to leadership as evidence that the organization is protected, and organizing security investment around moving the number up rather than building operational capability down—these are the patterns that create the gap between the dashboard and the door.
Post 6 in this series turns to communicating security posture to non-technical audiences: how CISOs and security leaders translate the kind of honest security measurement this series has been building toward into language that executives and boards can understand and act on.
