External Data Privacy Metrics and KPIs – A How to Guide for Strong Compliance

Information security, cyber security and the latest iteration, external data security are critical to the health and viability of any organization.  Companies of all sizes and types routinely invest significant sums into developing and maintaining robust defenses of their information systems.  International Data Corporation (IDC) forecasts annual global security spend will reach $300 billion by 2026.   That’s a lot of money spent in pursuit of a very necessary and vitally important goal.  So, it makes perfect sense that there should be equally significant processes in place to measure the effectiveness of the security practices and programs implemented to guard against unauthorized data access, breaches, IP theft, ransomware, reputational damage and all the other damaging consequences of security failures.

To this end, IT leadership has, for decades, employed and refined strategies to measure the effects of the practices and policies they adopt for information security and cyber security.  After all, they need to be able to prove first that their efforts are effective in reducing (if not fully neutralizing) the threat.  Second, they must justify the cost of their investments in the tools they implement to achieve secure operating environments.  It simply cannot be left to assumption that the steps taken, and investments made will deliver the desired results. 

In pursuit of data-driven proof of the efficacy of their efforts and investments, IT leadership and information security leaders like CISOs, CIOs, Chief Privacy Officers and other stakeholders have – over many years – built an entire discipline surrounding IT metrics monitoring and management.  Broad metrics provide the framework for success.  But to further understand the extent to which each metric is being satisfactorily achieved in the context of delivering sufficient security, CIOs drill into Key Performance Indicators or “KPIs”. These more granular indicators help provide greater context about how and why any step in the security practice, or segment of the security apparatus, is either meeting its stated objective or failing to do so.  This also helps identify where any weakness exists in a practice so that remedial steps can be taken to bring performance into compliance with governance and business rules.

Metrics and KPIs are not new to IT leadership like CIOs and CISOs.  However, as threats evolve and technology changes, metrics and corresponding KPIs must evolve and change.  They must adapt to stay aligned with the overall goal of strong cyber, information and external data security  The latter of which is the newly vulnerable attack surface.   While most organizations already observe metrics and KPIs for infosec and cyber security, many do not yet have this framework built specific to the challenges of external data privacy management practices. 

This paper argues the imperative for strong metrics and associated KPIs for external data security and distills widely accepted best practices for developing KPIs and metrics.  Read further to learn how to develop new (or extend existing) metrics and KPIs to effectively ensure compliance with data privacy and external data security laws as well as the emerging Governance, Risk and Compliance (GRC) requirements being adopted by industry leading organizations.

Typical Metrics Observed for Information Technology (IT) Practices

IT departments are commonly tasked with managing information and cyber security.  But they’re also responsible for management of the infrastructure of information systems.  That includes hardware like servers, workstations, desktops, laptops, cell phones, tablets, telephony systems, etc.  It also includes software systems – both on premise/installed and in the cloud – to manage myriad operational functions across the enterprise such as HR, procurement, financial management, legal, sales/marketing, supply chain logistics and others.  Each system – hardware and software – must perform at optimal levels for the organization to be productive and profitable.  The methodologies they’ve developed over time to measure effectiveness in many critical areas are only partly relevant to information security and emerging, external data security challenges.  Let’s look at the traditional broad stroke metrics gathered and recorded by IT:

• Response
  • Time to respond to performance issues
  • Time to resolve issues
  • Resolution rates
  • User satisfaction
• Performance
  • Systems downtime
  • Network outages
  • Mean time to failure
  • Frequency of outages
  • Network capacity
  • Online application availability
• Organizational
  • Workforce productivity
  • Overtime hours
  • Worker satisfaction
• Cost
  • Capital expenses
  • Cost per user
  • Cost per ticket
  • Cost per unit asset

Since the advent of the internet and cloud computing, IT has added the following infrastructure metrics and additional considerations focusing on information and cyber security:

• Patch management for every device (server, desktop, laptop, phone, tablet, etc.)
• Antivirus and anti malware updates
• Web filtering and web security detection
• Data encryption
• Email security and spam software
• Firewall management
• Endpoint detection and response (EDR)
• Workforce/user training
• Vulnerability scanning
• Multi-factor authentication
• Risk assessment

Today however, emerging threats to security posed by unsecured external data are the newest front in the war against cyber attacks.  Social engineering, phishing, email scams, data theft, IP theft, employee poaching, doxxing, spam and other productivity sapping risks are plaguing organizations. Most of these threats are made possible by unsecured external data and incomplete or weak data privacy.  Mounting an effective defense to these emerging threats requires IT to add privacy-centric metrics and the associated KPIs to their existing efforts. 

CISOs/CIOs working together with legal departments and other key stakeholders should (if they haven’t already) articulate a thoughtful GRC framework to provide objectives for external data security aligned with existing such objectives for broader info sec and cyber security.  Then, it will be possible to articulate the key performance indicators for compliance with each metric.

The global Governance, Risk Management and Compliance (GRC) market size was valued at USD 50 billion in 2022 and is expected to expand at a CAGR of 15.79percent during the forecast period, reaching USD 121 billion by 2028!

Marketwatch.com

What are Compliance KPIs and Why do they Matter?

Articulating strong KPIs ensures positive outcomes.  Asking the right questions is as important as gathering the answers.  Perhaps even more so, because the correct answers to the wrong questions are irrelevant and don’t help drive the desired outcome (in this case, exceptional data hygiene, and de-risked external data privacy).

Compliance KPIs according to the Complylog blog, “are metrics that help you measure how successful your compliance performance is in relation to your strategic goals. These include how compliant your organization is in its internal and external policies as well as in terms of the regulatory landscape in which you work.” 

KPIs help an organization measure the effectiveness of the actions called for in the compliance portion of its GRC.  KPIs also provide a means for identifying and remediating the early signs of non-compliance. 

In a data-driven business world, KPIs offer the information you need to quantify how you are progressing towards the strategic aims of the business.

Complylog blog

Developing detailed metrics and associated external data privacy KPIs as part of a GRC document matter for a host of critical reasons.  At the root of them all is the imperative to protect the organization from the ravages of information security, cyber security and external data security breaches.  The potential costs of breaches resulting from weak external data management practices alone are often enough to severely damage an organization.  The costs of such data breaches continue to rise.  In a separate white paper, Privacy Bee recently examined the Cost Benefit Analysis Proves the Necessity of Business Privacy Management.   The evidence in that document supports the imperative to enact and measure compliance goals for external data privacy management. 

This is further supported by the 2021 Data Privacy Benchmark Study produced by Cisco Secure which surveyed 4700 security professionals from 25 countries.  The study reported that privacy budgets doubled in 2020 and drive an impressive 35% return on investment into privacy protection.  Additionally, it was revealed that organizations with more mature privacy practices enjoy higher business benefits than average and are much better equipped to handle new and evolving privacy regulations around the world.

As security professionals embrace the new set of responsibilities surrounding external data privacy management (the Cisco report says currently 34% of respondents consider it a core competency and responsibility) the need for clear metrics and KPIs is further clarified.   The Data Privacy Benchmark Study revealed an eye-popping 93% of organizations polled say they are regularly reporting privacy metrics like program audit findings, privacy impact assessments and data breaches to their boards of directors. 

This is why strong, effective External Data Privacy KPIs matter.  Because without them, CIOs, CPOs, CISOs and others cannot effectively validate how their efforts:

• Ensure the organization has done all it can to reduce or eliminate the risk of data breaches via the leading attack vector today.
• Break down the many facets of an effective compliance strategy into manageable elements.
• Support routines for identifying and addressing weaknesses and gaps in security practices – helping determine whether improvements must focus on software, workflows, employee practices, trainings, etc. 
• Keep the organization actively engaged with emerging regulations and laws issued by governments. 
• Demonstrate to customers, business partners and the broader market the organization’s commitment and dedication to compliance with strong privacy and security posture and policy.

Plus, the regulatory environment continues to experience rapid change as legislation struggles to keep pace with technological and social evolution in commercial markets. Similarly, consumers, supply chain partners, investors and others are growing equally informed and concerned about privacy and security in their decision-making. 

The International Association of Privacy Professionals (IAPP) confirms the imperative facing all organizations and underscores how the best privacy leaders collect data and use metrics to measure, assess and improve the efficacy of their data security and privacy practices.  In 2022, IAPP convened a meeting of Chief Privacy Officers at their annual “Future of Privacy Forum” to discuss key issues and challenges facing those tasked with deploying and managing security and privacy practices. In an article sharing some takeaways from this forum, IAPP writes, “We learned that beyond demonstrating compliance, privacy metrics have emerged as key to measuring and improving privacy program performance and maturity in terms of customer trust, risk mitigation, and business enablement.”

There are metrics and associated KPIs relevant to EDP that are not presently being observed by IT departments and/or CISOs CIOs.  This section of the paper will illustrate what those EDP-specific metrics should look like and discuss how to capture and interpret these indicators.

What are the Characteristics of Viable and Instructive KPIs?

Fantastic work in this area has been underwritten by the Future of Privacy Forum (FPF).  A Washington DC based think tank and advocacy group focused on issues of data privacy, the forum is jointly supported by corporate sponsors and foundations. Corporate members include F100 industry leaders like AT&T, Comcast, Facebook, Google, Intelius and Microsoft.  Foundational support comes from luminaries like the Bill & Melinda Gates Foundation, Robert Wood Johnson Foundation, National Science Foundation, and Digital Trust Foundation.

The organization is run by Jules Polonetsky, the former chief privacy officer for AOL and Doubleclick. The founder and co-chair is Christopher Wolf, a lawyer who leads the privacy group at the law firm of Hogan Lovells. The advisory board includes representatives of LinkedIn, IAPP, Dell, Facebook, Microsoft, WalMart, ViacomCBS, T-Mobile, SAP, LiveRamp, Reddit, eBay or Uber.   

The aggregate perspective of these thought leaders and corporate giants is evident in the Privacy Metrics Report, produced by the Future of Privacy Forum in 2021. The report published an exhaustive breakdown of common metrics specific to privacy.  Most of which directly apply to EDP.  

Defining Audiences for Relevant Metrics and Reporting

It is critical to provide a framework for developing metrics to communicate with various internal stakeholders.  This is important because as the report notes, “Different metrics resonate differently depending on the audience. For example, members of the privacy team will view different reports than those presented to legal, compliance officers, or business leaders.”  It is important for privacy leaders to be able to fashion metrics into narrative stories tailored to a range of stakeholders.  From C-Suite execs to boards of directors, to investor groups, to customer and business partners, etc.

The Privacy Bee for Business platform supports the development of metrics and reporting to convey the to the following disparate audiences:

Executives – CEOs – Boards of Directors

• To drive buy-in and support for privacy initiatives, inform on risks to reputation and profitability, and report on program status/progress

Senior Leadership – CIO, CISO, CITO, CPO, CRO, etc.

• To assign executive oversight roles, align priorities, determine acceptable risk tolerances, and ensure data gathering is done uniformly

Privacy Teams

• To organize issue spotting and identify gaps, develop dashboards and other data visualizations, align goals, allocate internal workforce resources, performance reviews

Business Units

• To communicate with other corporate functions such as marketing, HR, development, legal and any other function accountable for privacy concerns – or in the case of sales/marketing and procurement, can leverage privacy reputation in their activities – and hold each unit accountable for managing its respective privacy risks

External Groups – Business Partners, Customers, Regulators, Investors, Media

• To build transparency and trust with investors and shareholders, differentiate the brand in the public eye and media, prove compliance with laws for regulators, demonstrate accountability to customers and prospects,

Internal Risk Management Auditors

• To deliver data-driven evidence of compliance with GRC, risk thresholds and cost guidelines

Employees

• To facilitate engagement, training and awareness, arm sales teams and business relationship managers with privacy practices to share with prospective customers/partners, and spur the workforce to exert a proprietary concern over their own privacy practices

Defining Common Metrics for External Data Privacy Management

After having identified the relevant audiences and narratives the metrics should inform, the metrics themselves must be carefully considered.  As noted earlier, asking the right questions is critical to a successful program of External Data Privacy Management metrics and KPIs. Again, the FPF’s Privacy Metrics Report is instructive.  The guidance categorizes the metrics into critical categories or buckets:

• Individual rights
• Training and awareness
• Commercial
• Accountability
• Privacy stewardship
• Policy

In each of these buckets, there must be key performance indicators to help determine whether or not the objectives of each metric are being met and why.  Here are several examples of metrics and correlated KPIs that illustrate the correlations and highlight how KPIs provide insight into the success or failure of each measure.

Data Subject Access Requests or “DSARs” is a relevant example of a measurable activity falling into the category of Individual Rights.  DSARs are the requests consumers lodge with any organization that holds their data and which companies are legally required to answer.  A program of External Data Privacy Metrics and KPIs might look at the numbers of DSARs received, closed, in-progress, time to close, percent of requests satisfied within required time, requests by type or region, etc.  Establishing tolerances for each of these indicators as part of a robust GRC practice, regular capture of these data points and review of KPIs helps a company ensure satisfactory performance of this element of the individual rights category. 

Privacy Awareness Education and Training is a relevant example of a metric falling into the category of “Training and Awareness”.  KPIs for this metric could include numbers of training events offered, percentage of trainings completed, percentage of employees passing the courses, number of trainings completed on time, and many others.  If any of these individual indicators is revealed to be performing poorly, there is reason for concern about the effectiveness of training and awareness as a goal.  And remedial efforts can be applied to bring performance indicators back into acceptable tolerances.    

Commercial external data privacy metrics and KPIs are important for re-risking third party vendor relationships, safeguarding supply chains as a potential attack vector, increasing chances of winning new business via RFx process and securing new business contracts for your business. 

Some of the indicators for commercial metrics could include the enforcement of data processing agreements, vendor privacy risk reviews and assessments issued and completed, and severity of vendor privacy issues. 

In the RFx process, KPIs could include the number of privacy attestation requests made/completed, quantity of standardized RFi/p/q questions and answers maintained in the RFx library and times to completion of privacy segments of RFx.  

When it comes to supply chain risk management, KPIs could examine the number of agreements for data sharing and corresponding risk assessments performed.  Percentage of agreements with privacy levels stipulated within and when these clauses were last updated.   For the software supply chain – a channel very reliant on strong external data privacy – the KPIs could include percentage of code sources presenting SBOM (software bill of materials) or assessment of privacy risks performed by project worker organizations.

The common metrics and KPIs for external data privacy management will vary widely by organization.  The nature of the business/industry, the size and scope of operations, the unique contours of the workforce management strategies and supply chain composition and many other variables contribute to the development of an effective privacy and external data security practice.  This is why industry leading organizations are investing significant cost and effort into developing the GRC structures necessary to govern their specific operations in an evolving threat environment. 

Privacy Bee for Business offers all the tools and methodologies necessary to support this complex set of tasks and to simplify the process of designing and deploying effective external data privacy metrics and KPIs.  Employee Privacy Risk Assessments, External Data Privacy Audits, Vendor Risk Management tools, Vendor and Cookie Consent tools, Consent Core’s user privacy preference management tools, Privacy Bee University for training and education, privacy trust programs for ecommerce and more are part of the comprehensive external data privacy management platform.  Each of these platform elements captures hard privacy data and delivers visualization dashboards to help measure KPIs and gauge the effectiveness of any organization’s efforts.  With Privacy Bee for Business, organizations have the tools and services they need to deploy a strong, well-developed Governance, Risk Management and Compliance (GRC) framework that succeeds at protecting information, cyber and external data security.