By Mark B. Cooper, President & Founder, PKI Solutions
Public Key Infrastructure (PKI) is a foundational technology that enables almost all modern identity and data encryption used in enterprises, including CIP infrastructure systems. It works to support all of the higher-level protection, management, identity, and access management solutions organizations deploy. To ensure organizations can adequately defend against and potentially operate during a cyber incident, the necessary tools must be resilient and protected from compromise and malicious use.
PKI Spotlight
PKI Spotlight is a revolutionary tool, as it is designed to focus on the most overlooked technology used by utility providers – their PKI. PKI Spotlight monitors all aspects of the PKI in real-time to ensure it is available and free from vulnerabilities and follows security best practices. Its unique capabilities provide defensive abilities to help prevent malicious activity that can subvert and circumvent higher-level authentication and encryption systems.
Threat intelligence researchers know adversaries are specifically targeting PKI with tools that are readily available to do so. PKI Spotlight provides real-time status of all components in the PKI and activity logs consolidated into a single dashboard that can provide real-time intelligence and reporting on activity in the environment.
In a natural disaster scenario, PKI Spotlight ensures redundant systems are appropriately configured, secure, and ready to assume operational status. This complex and time-intensive task may only be noticed at many utilities once it is too late. PKI Spotlight makes it easier to ensure these systems are configured and ready at a moment’s notice.
Additionally, during a disaster, utilities can quickly assess their system’s performance and resilience to ensure they will not suffer a secondary loss or incident due to a failing component or system that failed to operate correctly. During natural disasters, utilities shouldn’t have to worry about a critical cyber system failing while they focus on serving their customers and recovering from the disaster.
Assess and Monitor in Real Time
Historically, the only options organizations have had to enhance and plan resilience in their PKI required extensive and expensive expertise, which is harder to find in the workplace, and needed more tools to accomplish the task efficiently. Much attention has been paid to using tools like Certificate Lifecycle Management (CLM) products as a PKI tool. But that is a disservice to organizations as none of the CLM products on the market do anything for PKI itself. These tools provide alerting and reporting and potentially automation of end-point certificates. This is a tiny part of the problem but has historically gotten all the visibility, and it has left organizations with weak, misconfigured, vulnerable, and unstable PKI systems that result in outages and disruptions to service.
PKI Spotlight was designed to address the need for more focus on PKI within the security industry. It aims to help organizations move away from the antiquated “reactive management” process and toward a more proactive approach. In doing so, PKI Spotlight aims to prevent PKI failures and compromises rather than simply reacting to them after the fact. This is a unique approach in the modern cybersecurity space, as such a solution exists nowhere else.
This unique solution provides organizations a broad view of operational practices, disaster recovery planning, and BCP needs across their PKI. They can quickly assess and monitor, in real-time, the readiness of this critical technology. Rather than deploying BCP and DR systems and leaving them to sit for months and years until needed and hoping they work, PKI Spotlight can easily consolidate and present information so that organizations can design and prepare recovery plans and ensure their operational readiness in real time. Additionally, those DR and BCP-dependent systems are monitored for activity and vulnerabilities so that adversarial compromises aren’t lying in wait until they are activated.
Visibility, Reporting, and Alerting
The power of PKI Spotlight is, in many ways, the ease of visibility, reporting, and alerting. Organizations often deploy and operate their PKI without ever having the time or ability to test their assumptions or protections within the PKI. PKI Spotlight makes it extremely easy to see where the most critical keys for the PKI are stored and how they are protected.
Components such as Certificate Authorities, Online Certificate Status Providers (OCSP), and Registration Authorities have some of the organization’s most critical and sensitive keys but often go unmonitored. PKI Spotlight can quickly report and alert if those keys are stored in an inherently insecure location in software. Our integration with leading hardware security modules (HSMs) means we can determine if these components correctly secure their keys in an HSM and ensure those HSMs are operational and ready for future incidents.
Trying to mitigate potential key exposure during a cyber-attack is a losing battle. A cryptographic key secured in software during a cyber-attack will lose its presumed trustworthiness immediately. There is no way of ensuring an exposed key is valid once a compromise has occurred anywhere in the environment.
Additionally, during high-risk issues like a cyber-attack or natural disaster, PKI Spotlight continues to monitor the use of those critical keys for signs of malicious activity and provides real-time reporting. This consolidated, holistic view of the PKI and real-time events and activities has never been available before. It’s a powerful real-time view of activity and offers an auditable history of the environment for post-incident root cause analysis and breach response remediation.
Safeguarding Critical Infrastructure Against Emerging Cyber Threats
PKI Spotlight plays a significant role in providing real-time resilience, security, and compliance best practices for corporate and OT environments. Additionally, it extends its benefits to the CIP and FAN environments, which have been largely overlooked.
Focusing on collecting, analyzing, and reporting on these security items across critical OT environments in a consolidated view is something entirely new we are excited about. CIP assets shouldn’t be assumed to be secured simply because they are firewalled and managed in segmented networks. While those are good foundational security controls, a deeper, defense-in-depth approach must be implemented to protect these environments from modern, emerging cyber threats.
Real-time monitoring can be deployed within OT environments, including end-user intelligent devices and supporting PKI systems. This approach helps ensure that foundational technology used in modern utility and service provider OT environments is resilient, free of vulnerabilities, and aligned with best practices, which is a significant improvement over older methods.
Organizations should emphasize vendor-provided solutions, intelligent system manufacturers, and any other component in their OT and CIP environment to provide real-time visibility of their PKI. Just because a solution uses a PKI doesn’t make it secure. In fact, you may have lower security and more vulnerabilities by using a weak PKI.
Compliance with Regulatory Requirements and Standards
Modern OT and CIP environments are required to address known and potential cyber threats. Simply choosing and deploying a technology is not sufficient. If you look across standards like SCADA, CIP, and even what the NAESB has done with their PKI framework standard, properly auditing and managing the PKI is critical.
PKI Spotlight provides a simplified and consolidated view of cryptography compliance, security vulnerabilities, and real-time adherence to best practices, focusing on what the modern regulatory frameworks are designed to enforce. It does so in a timely and cost-efficient manner, helping organizations meet or exceed these standards. The standards themselves can and should evolve to reflect the persistent threat and emerging landscape of issues in environments with PKI.
A real-time approach is the only sure way of achieving these goals. Perhaps we are so far ahead of everyone else it will take time before they realize there is a more modern approach than spot audits, reporting, and long periods of assumed security in between.
A Comprehensive Solution for Real-World PKI Challenges
PKI Spotlight was born out of 20 years of consulting in the PKI space and our ongoing partnerships with electric utilities and providers nationwide. We have focused on the real-world challenges and issues we saw repeatedly at organization after organization. As a result, PKI Spotlight is designed to meet these needs out of the box. This is not a generic framework or monitoring tool that leaves the end user to write custom scripts, policies, monitoring parameters, etc.
PKI Spotlight is designed to install and provide intelligence in most environments in a few hours – almost always less than a full day to deploy. This rapid implementation is only possible with a highly specialized tool like PKI Spotlight.
While technology can do miraculous things like easily consolidate the view of multiple PKIs across an environment, it can’t, and shouldn’t, break security protections in place. For organizations with segmentation for OT and Corp networks, while PKI Spotlight does not require trusts, service accounts, or extensive firewall needs (it uses just a common Port 443 TLS connection), it cannot bridge the view of these highly segmented networks. So, PKI Spotlight is designed to be deployed in a module form within these separate networks and provides a consolidated view of everything within that segment.
Additionally, due to the origins of its creation, PKI Spotlight is not a SaaS solution. Our utility customers’ security needs and data exfiltration controls drove us to create PKI Spotlight as a solution entirely deployed within a customer environment. These environments include physical or virtual machines or even permissible private clouds.
At a high level, organizations have only a few prerequisites for deployment – specifically, which environments will be monitored and where they will place the “controller” – the user interface website. Agent software is installed on each monitored PKI Server, typically less than 15 minutes each. The connection is quickly established as long as there is a TLS connection to the controller.
The most common deployment uses an existing SQL Server in the environment for data storage. Though many OT and CIP environments may lack this resource, we provide an integrated SQL Express to address these scenarios. From there, it is a couple of minutes to establish what alerts you wish to receive via email, who should receive them, and to enable integration with Splunk optionally.
We have a Splunk marketplace dashboard and native data formatting, so the user needs no special integration or data parsing. Configure PKI Spotlight to point to your Splunk server, enable the dashboard, and choose to view the information on our custom Controller UI, within Splunk, or both.
Again, all of this can be accomplished in less than a day!
The Most Common Challenges Utilities Face When Implementing PKI Technology
PKI is complex and takes highly specialized skills. In modern cybersecurity, organizations are asking more and more of their resources. There are very few PKI-dedicated resources in organizations today. That differed ten years ago when many organizations had 1, 2, 3, or more people supporting their PKI. Now, organizations have 1, 2, or 3 cybersecurity people, but they support 10-12 systems each.
So many parts make up the PKI, so attempting to achieve all of the features we have in PKI Spotlight through manual processes is too time-consuming to provide value.
When we were preparing to launch our product, one of our resources in marketing asked how much time we saved an organization by automating all this work and providing visibility rather than doing it manually. I told them I couldn’t give them a number of hours for two reasons. First, so much effort was required to do this properly that no one did it. All organizations have assumed trust in their PKI and never have the time to review it thoroughly and regularly. Secondly, the time it took when we did a quick calculation was in the order of multiple days (24 hours a day) to do a single pass. At which time, we would have to do it again to ensure we were getting near real-time results.
In short, we are addressing the modern approach to cybersecurity and expertise and operationalizing and automating the work needed to operate and secure a PKI properly.
How PKI Solutions’ PKI Spotlight Contributes To Minimizing Downtime for Utilities During Disaster Recovery Operations
We approach maximizing uptime in two ways.
First, we prepare for the inevitability of a disaster recovery action through resilience monitoring and readiness. Ensuring the PKI components are properly configured, secure, and operational ready, even when not used daily, ensures that there is no downtime when organizations activate those systems as part of a DR or BCP event.
Secondly, during a disaster recovery event, the security and resilience of the environment will be even more critical to support the organizational focus of recovery. Higher-level systems often lack comparable real-time monitoring and will likely need the attention of those cybersecurity resources. PKI Spotlight will provide the proactive, real-time notifications they will need to ensure no further disruptions from issues within the PKI itself.
As we were developing PKI Spotlight, we were working with one of our electric utility customers to deploy an OT-CIP environment PKI to meet new PKI needs. We had previously deployed PKIs for their Corp and OT environments, and they were operating them like everyone else at the time, namely, cross your fingers and hope nothing fails and build in redundancy just in case.
In this example, we did like we had always done and deployed multiple HSMs to secure the keys for the PKI server components. The HSMs were configured in an HA manner to ensure if one failed, the PKI would continue to operate.
As we were working on the new OT-CIP environment, the customer happened to glance over to a nearby server rack while they were in the data center – a rare occurrence nowadays. They noticed that one of the HSMs for the OT environment was reporting a device failure. The PKI was operational but was now using a single HSM – the fail-over worked great.
They had no idea there was an HSM failure and were now operating in a single point of failure – perilously close to an outage without any redundancy. All because they had no visibility of the operational readiness and configuration of the PKI. Imagine what could have happened if they then had a DR event and were in that weak position.
After we diagnosed the issue and analyzed the root cause, we learned the HSM had failed six weeks prior! If they had had real-time visibility, the HSM would have been replaced and operational well before that point. We vowed at that point to make sure PKI Spotlight, which was under development, would never allow something like that to happen again.
Looking Toward the Future
We are just getting started! The exciting thing is how our existing customers are starting to see their areas of weakness in their OT and CIP environments and are asking how we can help them.
We have had recent conversations about how PKI Spotlight can provide oversight of vendor-provided smart grid electric metering components – something they have largely assumed as secure and overlooked due to a lack of tools. We are also looking at how discrete systems deployed in the renewable energy space can leverage PKI Spotlight to consolidate many discrete systems into a singular view – regardless of network connectivity.
We are also considering expanding our HSM monitoring to address the HSMs in organizations not part of the PKI. Our vendor-agnostic approach and heterogeneous approach mean we will be uniquely situated to provide the modern cyber organization with multiple HSMs through the organization with a singular view.
It’s an exciting time to see how we are helping solve real challenges organizations have struggled to address or may have even ignored in the past. Providing foundational resilience and security in critical infrastructure is essential for these organizations and their customers. We are proud to partner with the country’s leading organizations to address these issues proactively.
Follow Brilliance Security Magazine on Twitter and LinkedIn to ensure you receive alerts for the most up-to-date security and cybersecurity news and information.