Why do agile & cloud work well together?

Agile & Cloud from an Organizational Perspective

Working solely on cloud projects for last 8+ years , I have seen that cloud and Agile go hand in hand. How is that?Well, cloud is something new and transformative for a lot of enterprise customers and, because it’s so new, these organizations lack the requirements or exact understanding of how they want to utilise the cloud. Naturally they struggle to articulate what they want and need, which as a problem lends itself nicely to the user story and demo methodologies in Agile.

Agile methodologies make it very easy for companies to articulate on a "high level" what they think they want users to be able to achieve with the cloud. They do this using the three pillars of Agile, which are transparency, inspection, and adaptation(http://www.scrumguides.org/scrum-guide.html for more details).

 

You can demo functionality and certain behaviors to your organization early, and learn to adapt to their needs and help them gain an understanding of what they really want to achieve with cloud technologies. This is also critical for organization as it makes the development transparent  and the demo acts as an inspection element for the company to workout if the product is exactly what they wanted. At Cloudreach this has allowed me to present back to large organization and assist them into getting to grips with how they would like to use the cloud, instead of a more traditional way of planning something which means spending months developing something that the business then might realize it doesn’t want.
 

Benefits to Agile planning  as twofold:

    It allows the organization (consultancy or internal IT arm) to be more responsive to demand
    It reduces the demand for change control (even more important from a consulting perspective) or delivering to a fixed scope, which would be highly administrative

 

Agile & Cloud from a Change Management Perspective

Cloud adoption isn’t just about a change of technology, it’s also a massive organizational change.  "The key to success in this new climate isn’t about technology, it’s about management vision. The organization — and its leaders — need a forward-thinking, entrepreneurial culture that capitalizes on this computing power. The mantra above is key to how the Agile mindset can really assist companies in avoiding their pre-existing controlling nature. Instead it can allow the more forward thinking entrepreneurial culture to develop. It’s this culture of entrepreneurship which is required to deliver impactful change via Cloud Adoption. A lot of goals the "C Level" aim for when they decide to adopt the cloud are the same as those the Agile mindset helps to achieve:

    Enabling Innovation
        Organizations moving away from control mindset
        Learning to fail fast and gain organizational learning which is critical for the advancement of technology
    Increased Efficiencies
        Smaller teams with more autonomy and less management
    Quicker Speed of Delivery
        Change in processes
        Iterative development
    Value Realization
        Delivering value earlier in the development chain

I could go on and on, but all of the above benefits are true with both cloud and Agile and, ultimately, in this context they clearly have strong symbiosis.

Agile & Cloud working Together

I have explained why Agile and the cloud work well together from both an organizational perspective and a change perspective and, lastly, I want to take you through a technical perspective. Cloud lends itself well to an Agile approach because of one major concept in the industry – "Infrastructure as Code" – which allows cloud infrastructure deployment to be done in a similar way to software delivery, a way that was impossible for traditional infrastructure.

Understanding Infrastructure as code

Definition: Infrastructure as Code (IaC) is the process of managing and provisioning computing infrastructure via code, rather than physical hardware configuration or the use of interactive configuration tools. This is around the automation of deployments of infrastructure which has been made possible with the growth of Cloud Computing. (

As Infrastructure is now being managed more and more via code like software has in the past, moving and adopting Agile Frameworks makes sense for cloud infrastructure delivery. This is why in the industry we see Agile becoming more and more preferred. The main blockers to this at the moment are organisational rather than technical.

So, as highlighted above, cloud and software development have many similarities. They both:

    Use Programming Languages (python, ruby, json, etc.)
    Use Code Management
    Follow Continuous Integration and Deployment (CI/CD)
    Require Testing (Regression, UAT, Integration)

However there are some differences:

    In the Cloud you are responsible for the environments and the underlying infrastructure of the code. E.g. Having to use code to create and manage your staging environment that other code sits on. This is not the case with software which depends on infrastructure being there.
    Infrastructure tends to be very logical and less tangible than software (i.e. you can’t see it) for example Firewall config just ports and IP address or Access Controls with error messages.

These differences can create challenges in my experience when following an Agile methodology:

    Demos pose a challenge due to the less tangible nature of cloud services, which I have solved in the past by trying to keep them focused on the user and business benefit
    I have also had to split user stories into environments first then use code management to assist with the big picture, this effectively creates a matrix of user stories adding some complexity, but also helps with clarity if done correctly
    Although 95% of cloud has API access and can be done via code some elements are manual which cannot be automated or follow software development practices

So, in conclusion, if I was an enterprise working on any cloud project, I really would recommend an Agile delivery method. It is going to help team’s align with the need for quicker delivery and higher quality of the inputs into the cloud. Additionally it helps organizations get greater value from their project and truly iterate towards the goal they are achieving so that they can recognize some of that value sooner.