CNME Editor Mark Forker spoke to Sascha Giese, Head Geek at SolarWinds, to find out how the company coped with the fallout from the cyberattack on their Orion software system, the best practices needed to drive new innovations in IT – and their effort to make their products and solutions more secure.
What are SolarWinds’ product focus and targeted userbase – and how are you helping IT professionals to better manage their complex IT environments?
SolarWinds products focus on the individuals working in various IT roles: IT professionals. A career in IT can be exciting and rewarding, and most professionals with a passion for technology enjoy the responsibility of driving innovation for their employers.
But the job can be difficult, as technology changes at the same pace as the demands of the business. It’s not easy to stay on top of these changes while maintaining a running infrastructure—the backbone of the organisation—at the same time.
Our products aim to support these professionals in their day-to-day tasks, and they use various technologies to make complex IT environments manageable, whether they’re on-premises, hybrid, or any form of cloud or containers.
We understand 20-year-old storage arrays as well as hyper-converged SDS, and our solutions support applications running locally on Linux as well as highly distributed microservices.
We know SolarWinds was the victim of a high-profile cyberattack, but how did the SolarWinds IT community respond in terms of the investigation you launched and how you communicated the issue?
Early on, it was obviously a shock to many. Outside of the incident response team, few knew much about what happened, or how broad the impact might be. We communicated with customers, partners, and the public as openly and quickly as possible, while supporting ongoing investigations.
We put our customers first and not only kept them up to date, but suggested additional measures to take, and we had hotfixes ready for our software within three days.
Many IT professionals and their organisations depend on our solutions, and we can’t leave them in the dark. Our open communication was well received by our customer base and the IT community in general.
Let’s face it: this incident caused concern for our customers and created a lot of extra work. No one likes that. It was our duty to inform our customers, and we created a free program to actively support them in the process of securing their environment.
This included assisted upgrades, updating software certificates, and more.
Why do current software development procedures that are often considered as best practice industry standards still require even more secure practices?
Best practices are what they are because they’ve proven to be efficient, fail-safe, and easy to follow, just as good process management should be.
They’re the result of a long trial-and-error phase and based on experience. And here’s the thing: because of this, they aren’t always up to date.
Software development in particular is a field with slow innovations—even agile is two decades old. Adding security into the development process wasn’t really a necessity. It was actually quite the contrary, as adding security significantly increases the cost of software development.
As a result, engineers relied on the infrastructure security measures already in place. The sophistication of the attack tells us not only do the requirements for processes need to change, but the whole industry needs to change their mindset, too. This wasn’t the first of these attacks, and unfortunately, it likely won’t be the last.
What do you believe are the key components and fundamental principles in achieving safer software developments and products?
Security needs to be part of each step in software design and development. In the last few years, we saw an advancement in securing networks and infrastructure, but this required a changed mindset, too. This change involved going from perimeter security and thinking “our firewall will protect us” to the zero-trust model. In the zero-trust model, you assume you’re already breached, and you have mitigations in place everywhere.
To improve the security of software development, it’s important to get many experts on board—both internal and external—to spot weaknesses, check what’s already in place, and find room for improvement.
The challenge is to find the sweet spot of being more secure without adding complexity for the day-to-day business. Another lesson we learned is each software vendor—in fact, each business—should evaluate their supply chain more closely.
It’s a good idea to discuss possible risks and remediation plans with their vendors. Just a few weeks ago, there was another instance in the news of a design flaw in a popular VPN vendor, which has been actively abused. You can’t plan for such risks, but it’s possible to mitigate their impact.
What are steps to enhance the product development environment and secure software?
At SolarWinds, we identified our source code hadn’t been touched, but the threat actor was present in the build environment, where source code gets compiled into the final product to make it ready for shipping.
What we’re now implementing is basically multiple independent development lines, with two build environments. This process will help ensure both compiled executables are identical before they’re processed any further. But this isn’t enough. There are additional automated and manual checks in place now to verify the authenticity of both the source code and the products, from the moment the code gets written on a developer’s machine all the way to publishing the product.
And though we already used various solutions to secure both the endpoints and our infrastructure as a whole, we’ve added additional layers of security. We paid attention to the use of service accounts, as they’re of particular interest for attackers, and we removed permissions that weren’t strictly necessary.
We reflected on general security policies, too, and enforced multi-factor authentication for the entire company. Furthermore, we’re continuing to work with security specialists to improve our overall posture. We’ll also continue to share the measures we’re taking and our experiences with other software vendors, as we hope to help them prevent a similar incident.
What is SolarWinds strategy on further innovations, products and features that are necessary for digitisation?
Over the past few months, our focus has been on making our products more secure, and we shifted most of our engineering resources to this area while also researching further innovations.
For example, we added machine learning into one of our products, and we’d like to extend the same technology into other products, too. This will help relieve IT professionals, as it’ll be easier for them to understand relations and the impact individual events have on the bigger picture. We work closely with our customers, and they have a say in what we work on next. This helps us focus on what’s essential for them.
When it comes to digital transformation, we already provide a lot of assistance in our products for connecting to resources in different locations including clouds.
Once connectivity works, the next step is making sure applications talk to each other as intended. We have automated application-dependent mapping in our products, a feature designed to point to the weakest link in the application delivery.
In addition, we already brought microservice and process information into our platform, and one of our next steps is improving the way we make sense of all of this information. Actually, we’re going to attend GITEX this year with our local Partners, and we’re looking forward to showing how this is working.
