It’s often said that you can’t manage what you can’t measure.
When it comes to cybersecurity, there is almost too much that can be measured, both in the volume of data available and in the variety of data that can be measured.
It can be difficult to identify what is strategically important and what is noise.
What follows is an overview of how to tune out the noise and home in on the signal so you can successfully defend your organization from cybercriminals by paying attention to the right metrics.
The Key Question of Cybersecurity
Regardless of the size of your business or industry you’re in, if you are connected to the Internet, cybersecurity is essentially about this single question: “Are we secure?”
The title of this post is a bit of a misnomer. Executive leadership, by and large, doesn’t care about the “metrics.” They care about the story -- the pattern and trends -- that the metrics reveal. Organizations track metrics so that they can understand the broad story so that they can answer objectively “yes” to the question. As always, remember that no information security strategy is foolproof and can only decrease your risk, not eliminate it.
NIST’s Directions in Security Metrics Research outlines three uses for metrics:
- Strategic support – Assessments of security properties can be used to aid different kinds of decision making, such as:
- program planning,
- resource allocation, and
- product and service selection.
- Quality assurance – Security metrics can be used during the software development lifecycle to eliminate vulnerabilities, particularly during code production, by performing functions such as:
- measuring adherence to secure coding standards,
- identifying likely vulnerabilities that may exist, and tracking and
- analyzing security flaws that are eventually discovered.
- Tactical oversight – Monitoring and reporting of the security status or posture of an IT system can be carried out to:
- determine compliance with security requirements (e.g., policy, procedures, and regulations),
- gauge the effectiveness of security controls and manage risk,
- provide a basis for trend analysis, and
- identify specific areas for improvement.
With so much data available to be analyzed , it’s easy to lose sight of what’s important to answer this key question of “Are we secure” affirmatively.
The Five Essential Categories of Cybersecurity Metrics
The National Institute for Standards and Technology outlines five functions in its Cybersecurity Framework:
- Identify: Is our cybersecurity posture appropriate?
- Protect: Are we protected from reasonable threats?
- Detect: Can we detect an incident?
- Respond: Can we respond to an incident?
- Recover: Can we recover from an incident?
Each of these functions has metrics that can be used to determine how well each function is performing.
The metrics to be observed must be:
- Quantifiable (percentages, averages, and numbers)
- From data that is easily obtainable
- From a repeatable security process
- Useful for tracking performance and directing resources
Identify
Where is the organization vulnerable to hacking or other cybercriminal activity? The “Identify” piece of the framework is focused on steps to assess cybersecurity risks and prioritize efforts to reduce them.
The metrics to focus on here involve identifying all physical and software assets that need to be protected, including those that are most important and data that is most at risk. Creating a risk management strategy is a crucial building block of your cybersecurity strategy.
Protect
These are your defenses that allow you to continue to deliver safe, secure infrastructure services. Included in this function are activities such as:
- Access control - keep a record of who is entitled to access all elements of your IT infrastructure, including information assets
- Awareness and training - implement an ongoing training program for all staff, including tracking who has completed your security training program, considering testing to measure knowledge or even a phishing test that will test user knowledge with a controlled “attack”
- Maintenance - IT infrastructure requires ongoing updates and patching, track how much lag time there is before a security patch is released and applied your team. Lag time should be as low as possible
- Protective technology - anti-malware, firewalls, etc.
Detect
You must be able to discover cybersecurity events. Metrics to track here include anomalies against your baseline network operations and data flows. Continuous monitoring for events, including malicious code; personnel activity; unauthorized personnel, connections, devices, and software to verify the effectiveness of your protective measures.
Respond
Once an incident is discovered, your organization must act promptly to understand the event, mitigate the effects, and end the threat.
The key metric here is mean time to resolve (MTTR); once your team detected the threat, how long did it take to respond and address it? How compromised is your network, i.e., what level of business operation remains possible? Identify the attack path.
Every business should create an incident response plan. Read about What to Include in Your Incident Response Plan here.
Recover
The final function of the NIST framework focuses on ensuring you can restore systems and assets affected by a cybersecurity event to normal operations.
How long will it take to restore systems from backup? Were open ports exploited that need to be closed? Did the attack expose holes in your security posture?
Improvements to your security posture by including lessons learned into your future cybersecurity planning is a key part of this function. For events involving data breaches of customer data, a communication plan is also essential, both for regulatory compliance reasons and to restore customer trust.
9 Metrics That Matter
Business leaders have no use for overly technical metrics. They want to see metrics on how the bottom line or the company’s reputation may be impacted. Here are nine useful metrics that will help you tell your organization’s cybersecurity story in a way that matters to business users and executives.
- Monitored employees. Identify all employees with super-user and admin access and monitor them. An internal employee can act maliciously and seriously harm a business. A hacker who successfully gains access to an employee with access to your systems can mine your data to sell, hold your data for ransom, engage in corporate espionage, or cause other damage. Another reason to monitor employees is you can more easily see if you’re providing unlimited access privileges to more individuals than required.
- Time to detection. Tracking the types of attacks over time will allow you to adapt your defenses to better protect yourself from most common attacks. For example, frequent attacks by botnets rather than smurf or ping of death attacks for denial-of-service attacks or a higher incidence of worms versus stealth viruses, macro viruses, or file infectors in malware attacks. You can also decrease your “detection deficit” - the time from successful attack to when you identify and fix it. The ideal is as close to zero as possible.
- Time to remediation. Once a breach is discovered, how long does it take to resolve the issue? This should be measured in hours.
- Reported incidents. How many hacking incidents did you experience over a period of time and are incidents increasing or decreasing?
- Number of large and small security incidents. Track these numbers independently. While a small incident may cause only minor aggravation, the same type of incident can add up to many hours of lost productivity over a year. Identifying these small incidents can reveal that they equal the cost of a large hack over time.
- Cost per incident. Identify the resources needed to respond to and resolve an attack, but don’t stop there. Costs also could include staff overtime and the need to bring in additional staff, the cost of an investigation, lost employee productivity, and a communication plan to customers and the public.
- Downtime. When systems are down, employees can’t work, customers can’t make purchases or schedule appointments, etc. Consider tracking lost sales by referencing historical data to measure the impact on your revenue.
- Time to resolve the incident. Log the amount of time it takes to resolve a cyber attack, from when the event was noticed to the final report.
- Average time to patch. The faster you update software, the smaller the window for cybercriminals to attack a known weakness.
Using Metrics
Metrics will help you track and determine the most appropriate level of cybersecurity for your organization.
Tracking time measurements may reveal the need for additional staff, or a need to outsource certain functions.
Whatever the story your cybersecurity metrics tell about your organization today, know that the story will change over time. Keep tracking metrics to identify new cybersecurity strategies, determine appropriate staffing levels, and have the capability to respond swiftly and effectively to cybersecurity incidents.
