Our cloud complexity bias can hurt businesses

Most cloud architects prefer the most complex solution to the simplest solution. Let’s see why and look for a better way.

I was listening to my podcast feed again this weekend, and beyond the actual police broadcasts, I listened to something more work-related. The question posed about this episode was profound, rarely asked in today’s technology press: “The cloud was supposed to facilitate computing, but it is now as complicated or more complicated as the old data centers and applications. Is there a future in a simpler cloud? »

Those of you who have followed me here for a while or have taken my courses understand that I have tried to find a balance between making cloud architectures complex and making them optimized and efficient. The more I have studied this space, the more I think I am on something: we need to understand what the trade-offs are.

The heart of this problem can be a people problem, not a technology problem. Most architects create and deploy cloud solutions that are often too complex and expensive. They do so under the influence of some conscious and unconscious prejudices.

No need to look beyond complexity bias: “Faced with two competing assumptions, we are likely to choose the most complex. This is usually the option with the most assumptions and regressions. Therefore, when we need to solve a problem, we can ignore simple solutions – think “that will never work” – and prefer complex solutions. »

I’m not an expert at deciding on the psychological problems of making things, including cloud architectures, too complex. It is interesting to note that simpler solutions with the fewest mobile parts (cloud services) are generally much better than trying to push each type of technology into the final architecture deployed. Don’t choose four types of storage when two will suffice. Opt for 10 different native cloud databases, as some of them have features that might be needed at some point in the future… Well, maybe.

The problem is that the complex architecture works very well – initially. However, construction, deployment and operation are three to six times more expensive. There is no security for other company executives who point out that even if the solution is necessary, it is too expensive because it is too complicated and over-designed. In other words, cloud architects are doing well and are probably commended for deploying a solution where innovation is taken for too complex.

Cloud architects (like me) who favor simplicity or abstraction and automation to manage the inevitable complexity must find a balance with those who naturally gravitate towards overly complicated cloud architectures. In addition, I prefer almost fully optimized and minimally viable solutions, which I know work better than complex solutions.

I suspect some things will probably happen:

First, just by trial and error, those who design and build cloud solutions too complex and expensively will be identified and their negative impact better managed. That’s why I always insist on peer reviews of cloud solutions in order to have checks and balances. Unfortunately, for most companies, internal or external reviews are more the exception than the rule.

Second, post-mortems on computer/cloud disasters will become more common. Could excessive cloud complexity have caused security operational problems resulting in accidental data exposure? What happens if an investor audit identifies “complexity and cost issues” that result in a whole new IT management team? None of these options are good for the company.

Is it time to start thinking about how to reduce complexity? I think so.

Leave a Reply