Now that cloud computing projects are in their second and third generations, some of the most advanced cloud computing developers and architects, including many of those in the Global 2000, are hitting an unexpected wall. It’s a wall that stops some enterprises dead in their tracks and can often lead to a negative ROI from use of the cloud that could end in disaster for the company.
So, what’s going wrong?
In a word: Complexity. As enterprises move to private clouds, hybrid clouds, and now multi-cloud, the number of platforms, databases, devices, network, and end points continues to explode. All are different, with different types of operational requirements. Also count in the migrated and net-new cloud-based systems that get added to existing legacy systems, with few being replaced.
All of this adds up to a tipping point where the growth of cloud computing increases enterprise IT complexity to a point where some enterprises experience zero or negative value from the cloud. The dream of operational cost reduction and increased agility gets replaced by sky-high operational costs and static and complex systems that can’t be changed.
While there are no easy answers or magical tools to address the complexity issues, there is some fundamental architectural planning that can be done. Couple sound planning with a competent technology stack, and you can avoid cloud complexity, or even fix existing damage complexity may have already caused.
This article explores a path to solutions that can help minimize the effects of cloud complexity for the modern enterprise.
Cloud complexity emerging
The Cloud Complexity Management Survey conducted by Deloitte Consulting LLP discovered that medium-sized businesses (33%) and large enterprises (37%) are believed to be the most vulnerable to the cloud complexity problem. The core message from this survey is that most large enterprises have the perception that they “have more complexity than expected.” Moreover, 47% of those surveyed considered this complexity as the primary risk to ROI.
At issue is the fact that the complexity problem snuck up on most enterprises moving to cloud computing. This is likely a surprise factor for a few common reasons:
- The migration to cloud computing was done with decoupled teams, on aggressive sprints, operating independently of each other. This led to the selection of different brands of public and private clouds, different security solutions, different governance solutions, and different management and monitoring technology stacks and approaches.
- No common command-and-control. There is no significant foresight as to how to define and leverage common services, such as data, storage, and other infrastructure items.
- Operations without consideration.Systems are turned over to the CloudOps teams without regard to the effects of increasing complexity. Budgets and resources are held static, while the workloads increase.
Strategies
There are a few approaches to deal with cloud complexity, both proactive and reactive. Each has their own set of pluses and minuses.
The proactive approach is to fix the issues listed above, namely poorly coordinated cloud projects, synergy with cloud solutions, and the ability to leverage common services, such as security and governance. Moreover, any strategy aimed at addressing cloud complexity needs to span traditional on-premises systems and those in the public cloud.
The upside is that you can avoid most of the complexity challenges before they become problems. However, the price of being proactive is to perhaps slow down the processes that are now highly Agile-focused. More importantly, modern enterprises’ IT shops typically don’t want to behave this way.
Reactive approaches mean that organizations respond to complexity issues after the fact. This assumes that cloud will include many decoupled and aggressive migration and net-new development projects, and there will be no close coordination between the projects. The mess that pops out of those projects will need to undergo the normalization of approaches and tooling associated with managing that complexity.
By fixing complexity issues after the fact, the effort and money spent on normalizing cloud complexity will likely come back 100-fold. The advantage would be not having to deal with enterprises’ IT concerns, and still obtaining the same results.
Cloud complexity management
There are also two additional strategies to address cloud complexity:
First, abstraction. Using an abstraction layer with a least common denominator approach removes you from the complexity of dealing directly with cloud-native tools and interfaces.
This can easily be accomplished by leveraging tools that can remove you from dealing with the underlying native features of the cloud platforms or services. Tools such as cloud management platforms, API management platforms, and cloud services brokers are a few categories to consider.
Second, automation. Automating the use of interfaces can make operations that much easier and thus less complex. Orchestration tools are handy to help solve the complexity problem as well, tools such as Kubernetes, or others that deal with the automation of complex platforms.
However, the most important step in the process is to leverage an approach that will help you define your problem domain a bit better, decompose the domain into sub-domains, and then break those apart into solution patterns. As you can see from this figure on managing cloud complexity, we broke down the problem domain into 9 subdomains that cover the major solution patterns, from data to on-premises systems. Keep in mind that on-premises systems will also be a complexity factor in the use of cloud and should be considered inside of its own domain.
The idea here is to think about the complexity in fewer complex ways, including a list of things that are needed to do so. For instance, data includes abstraction, data hygiene, active metadata management, and data operations. All that must be understood to move to abstraction, automation, or most likely both.
This is a hard problem to solve, no matter what methodologies and technology you toss at the problem. It will be a core challenge for larger enterprises going forward. The message here is to understand that the problem exists or will likely exist and take concrete steps to solve it.
David Linthicum is Chief Cloud Strategy Officer at Deloitte Consulting LLP. He is responsible for building innovative technologies that help clients operate more efficiently while delivering strategies that enable them to disrupt their markets. Linthicum is widely respected as a visionary in cloud computing -- he was recently named the No. 1 cloud influencer in a report by Apollo Research and is the author of more than 13 books and 5,000 articles. For more than 20 years, he has inspired corporations and start-ups to innovate and use resources more productively.