It is a comforting to think about information security as a form of computer science - but the reality of securing complex enterprises is as unscientific as it gets. We can theoretize how to write perfectly secure software, but no large organization will ever be in a meaningful vicinity of that goal. We can also try to objectively measure our performance, and the resilience of our defenses - but by doing so, we casually stroll into a trap.
Why? I think there are two qualities that make all the difference in our line of work. One of them is adaptability - the capacity to identify and respond to new business circumstances and incremental risks that appear every day. The other is agility - the ability to make changes really fast. Despite its hypnotic allure, perfection is not a practical trait; in fact, I'm tempted to say that it is not that desirable to begin with.
Almost every framework for constructing security metrics is centered around that last pursuit - perfection. It may not seem that way, but it's usually the bottom line: the whole idea is to entice security teams to define more or less static benchmarks of their performance. From that follows the focus on continually improving the readings in order to demonstrate progress.
Many frameworks also promise to advance one's adaptability and agility, but that outcome is very seldom true. These two attributes depend entirely on having bright, inquisitive security engineers thriving in a healthy corporate culture. A dysfunctional organization, or a security team with no technical insight, will find false comfort in a checklist and a set of indicators - but will not be able to competently respond to the threats they
need to worry about the most.
A healthy team is no better off: they risk being lulled into complacency by linking their apparent performance to the result of a recurring numerical measurement. It's not that taking measurements is a bad idea; in fact it's an indispensable tool of our trade. But using metrics as long-term performance indicators is a very dangerous path: they do not really tell you how secure you are, because we have absolutely no clue how to compute that. Instead, by focusing on hundreds of trivial and often irrelevant data points, they take your eyes off the new and the unknown.
And this brings me to the other concern: the existence of predefined benchmarks impairs flexibility. Quite simply, yesterday's approach, enshrined in quarterly statistics and hundreds of pages of policy docs, will always overstay it welcome. It's not that the security landscape is constantly undergoing dramatic shifts; but if you don't observe the environment and adjust your course and goals daily, the errors do accumulate... until there is no going back.
Okay, okay, it's official. You can now buy The Tangled Web from Amazon, Barnes & Noble, and all the other usual retailers for around $30. You can also order directly from the publisher, in which case, discount code 
