Measure for measure

Information security metrics have a bad rep. Mention metrics to a CISO and immediately his thoughts may well turn to sigmas, standard deviations and, probably, probability. To many, metrics equals statistics.

But the truth is that not all of security metrics need be complex. In its simplest form, security metrics is the process and tools involved in measuring the effectiveness of the security investments made by an organisation.

“Security metrics are a set of functions that allow CIOs to evaluate their existing security infrastructure and establish a baseline to measure the impact of actions they take on their overall security objectives. There are a wide set of solutions available today that can help CIOs define more accurate, measurable and consistent security metrics,” says Tarek Abbas, systems engineering director at Juniper Networks.

Nigel Hawthorn, VP of EMEA marketing at Blue Coat agrees, “Security metrics deliver information that allows senior management to evaluate the benefits of the IT security systems comparing either against the past or other organisations to get an understanding of security threats over time or between similar organisations.”

Others take a slightly different approach to defining security metrics.

“All business projects need to define the metrics and indicators that are cared for from a business perspective up front.  For too long, security has been outside the norm and not tied to business metrics and business context.  So if we tie this down, all projects, security or otherwise, need to go through a metrics phase that will be used to instrument and measure success and these need to matter in business terms. The definition of security metrics, therefore, is a collection of indicators in security operations that are markers of business success or failure, and are used to optimise security as a business function,” states Sam Curry, CTO of marketing at EMC.

Jeff Ogden, senior director of services for the emerging region at Symantec’s Services Group reminds us that security metrics can mean different things for people situated at different parts of the organisational hierarchy. Whatever the definition you agree with however, one cannot deny the necessity of metrics inside an organisation.

“Security metrics are used for determining business success or failure. These are analogous to vital signs that are monitored by a surgeon. Every vital sign requires mental energy and attention, so only the ones that matter to the success of the surgery and the patient’s health should be monitored.  The same is true of business.  The difficulty here is that, like a patient in a hospital, every business is unique and the metrics need to be determined as part of an engagement process around a project or function,” says Curry.

Ogden adds, “For decades, security practitioners reverted to fear, uncertainty and doubt as a way of selling themselves to the organisation. But from the boardroom to the server room, modern-day organisations now demand “value” Unless security professionals manage to quantify their value to the organisation, business leaders are simply not going to pay attention. This is precisely why so many organisations are still reactive in their security posture. Security metrics is what makes it possible to demonstrate the value of a well-defined security program. We need metrics to determine priorities, we need metrics to measure performance against pre-defined targets, we need metrics to predict future trends and then adjust our control measures to meet those challenges.”

With the right use of business-aligned security metrics inside the organisation, the CIO or his security counterpart will be able to identify and highlight risks to higher management with ease, and ensure that they get the budgets they require for continuous protection of the enterprise.

Ogden states, “The key to getting more money from the board is to demonstrate how effectively you applied the previous budget already approved for security. You need to be able to express this in business language. Metrics is the language senior executives understand. On a daily basis they pour over metrics such as ROI, IRR or ROE. When you express security objectives in the form of quantifiable metrics you are more likely to be understood and respected as a contributor to sustainable business rather than an unavoidable expense.”

However, in order to get the higher management on the side of increased security investments, the CIO will first have to calculate the ROI for the defence expenditure he is planning.
Juniper’s Abbas says, “Accurate measurement of the ROI on security investments depends on the business objectives of an organisation. However, a good starting point would be the number of detected and identified security attacks and the corrective measures taken consequently. Enterprises can also measure the yearly downtime related to security attacks (very important for banks), percentage spending on security as a total of the IT budget, and data leakage (internal and external).”

“Take a bottom-up and a top-down approach. The CIO is in the unique position of having to be both a technologist and a business person, and both voices need to be heard in the business forum.  So assemble the things you care about in the bowels of IT and roll them to the top to be scrutinised under a business light and take the business imperatives and drag them down into technology. What emerges will often surprise and enlighten the whole company,” points out EMC’s Curry.

Ogden states, “The trick is to tailor the presentation of metrics to each intended audience. In other words, those who develop the metrics must understand the “nature of the decisions” being made by CEOs. Secondly, the metric must be expressed in business terms that CEOs are likely to understand. By using visual intelligence techniques such as icons and dials to indicate trends executives can derive an intuitive and instantaneous understanding of what’s being communicated.”

 While the CIO is taking steps to get across the ROI of his proposition, the CEO can also take steps to understand and evaluate security metrics.

Abbas points out, “Simple questions can help the CEO of an organisation understand and evaluate security metrics: How do our security measures compare to best practices in the industry? With the investments in security, are we more secure today than we were last quarter? Where do we stand compared to similar organisations in the industry?”

CIOs can also help the higher management along the path to understanding security metrics better by using standards and best practices as reference points for the development of metrics. International standards, like the ISO 27001, can highlight areas or bring focus to sections for which metrics need to be developed.

Despite what would appear to be a critical need for the wider use of security metrics, most regional enterprises are yet to pick it up and use it in their organisation.

“Lots of customers have so called security policies. But it is often nothing other than a few reports that have been written by consultants. They have just been stored on the shelves and allowed to gather dust. It never really got translated to security measures. So are people adopting metrics? I would say no. They are full of good intent, a lot of them understand the need, but very few people put a policy with metrics in practice,” says Philippe Roggeband, business development manager, Cisco EMEA.

Ogden points out that this is largely related to the maturity of an organisation.

“Unless an enterprise is at a certain level of maturity and control (as advocated within the first four levels of the capability maturity model (CMM)), security measures do not mean anything. Not every organisation has reached the level of maturity that allows them to put a robust measuring process in place,” Ogden says.

Abbas adds, “Security metrics are a part of a wider security policy that should be in place for any organisation. These metrics are used in certain verticals across the region (government, financial) more than others, but in general, there is a lot more to be done. Factors contributing to low adoption vary from budget constraints to lack of experienced IT staff. Additionally, the definition of security metrics and proper documentation of best practices are still works in progress despite improvement.”

Most industry experts believe that the use of security metrics is just in its infancy in the Middle East. To illustrate the situation better, if there are ten banks around, only one of them would be considering  implementing anything related to security metrics. The situation being as it is, consultants and experts advocate easing enterprises into the arena of security metrics by starting out with small measures that are easily quantifiable and whose ROI can be determined without much hassle (such as spam).

“Once enterprises start with proper security metrics they will notice that it will and does go into and touch several other areas of the business, like bandwidth, and they will be able to improve cumulative efficiency across the organisation,” points out Ogden.

One way or the other, it is time that regional enterprises started paying a lot more attention to security metrics and worked towards employing them.

As Curry puts it, “Without metrics, security is an exercise in academic futility doomed to be misunderstood and unappreciated. Embrace security metrics and strive for as efficient a translation of security, IT and business as is possible.”