Using Chef Tool for DevOps Automation

 

A Devops engineer spends more time in deploying new services and application, installing and updating network packages and making machine server ready for deployment. This takes tedious human efforts and requires huge human resources.By using configuration management tools like Chef or Puppet you can deploy, repair and update the entire application infrastructure using automation.

What is Chef?

Chef is a Configuration management tool that manages the infrastructure by writing code rather than using a manual process so that it can be automated, tested and deployed easily. Chef has Client-server architecture and it supports multiple platforms like Windows, Ubuntu and Solaris etc. It can also be integrated with cloud platform like AWS, Google Cloud, and Open Stack etc. 

Understanding Configuration Management

Let us take an example of a system engineer in an organization who wants to deploy or update software or an operating system on more than hundreds of systems in your organization in one day. This can be done manually but it may cause multiple errors, some software’s may crash while updating and we won’t be able to revert back to the previous version. To solve such kind of issues we use Configuration management tools.

Configuration Management keeps track of all the software and hardware related information of an organization and it also repairs, deploys and updates the entire application with its automated procedures. Configuration management does the work of multiple System Administrators and developers who manage hundreds of server and application. Some popular tools used for Configuration management are Chef, Puppet, Ansible, CF Engine, and SaltStack.

Why I prefer Chef Automation Tools?

Let us take a scenario, suppose we want our system administrator to install, update and deploy software on hundreds of system overnight. When the system engineer does this task manually it may cause Human errors and some software’s may not function properly. At this stage, we use Chef which is a powerful automated tool which transfers infrastructure into code.

Chef automates the application configuration, deployment and management throughout the network even if we are operating it on cloud or hybrid. We can use chef to speed up the application deployment. Chef is a tool for accelerating software delivery, the speed of software development refers to how quickly the software is able to change in response to new requirements or conditions

Benefits of Chef

Accelerating software delivery

 By automating infrastructure provisioning we automate all the software requirements like testing, creating new environments for software deployments etc. becomes faster.

Increased service Resiliency

By making the infrastructure automated we can monitors for bugs & errors before they occur it can also recover from errors more quickly.

Lower Risk Management

Automation tool like Chef or Puppet lowers risk and improves compliance at all stages of deployment. It reduces the conflicts during the development and production environment.

Easy Cloud Adoption  

Chef can be easily adapted to a cloud environment and the servers and infrastructure can be easily configured, installed and managed automatically by Chef.

Managing Data Centers & Cloud Env 

Chef can run on different platforms, under chef you can manage all your cloud and on-premise platforms including servers.

Streamlined IT operation & Workflow 

Chef provides a pipeline for continuous deployment starting from build to test and all the way through delivery, monitoring, and troubleshooting.

In summary Chef tools help IT teams adopt modern day best practices including:

• Test Driven Development: Configuration change testing becomes parallel to application change testing.
• DevOps Support: IT operations can confidently scale with data consolidations and 3rd party integrations.
• Self-Service: Agile delivery teams can provision and deploy infrastructure on-demand.

 

 

Why do agile & cloud work well together?

Agile & Cloud from an Organizational Perspective

Working solely on cloud projects for last 8+ years , I have seen that cloud and Agile go hand in hand. How is that?Well, cloud is something new and transformative for a lot of enterprise customers and, because it’s so new, these organizations lack the requirements or exact understanding of how they want to utilise the cloud. Naturally they struggle to articulate what they want and need, which as a problem lends itself nicely to the user story and demo methodologies in Agile.

Agile methodologies make it very easy for companies to articulate on a "high level" what they think they want users to be able to achieve with the cloud. They do this using the three pillars of Agile, which are transparency, inspection, and adaptation(http://www.scrumguides.org/scrum-guide.html for more details).

 

You can demo functionality and certain behaviors to your organization early, and learn to adapt to their needs and help them gain an understanding of what they really want to achieve with cloud technologies. This is also critical for organization as it makes the development transparent  and the demo acts as an inspection element for the company to workout if the product is exactly what they wanted. At Cloudreach this has allowed me to present back to large organization and assist them into getting to grips with how they would like to use the cloud, instead of a more traditional way of planning something which means spending months developing something that the business then might realize it doesn’t want.
 

Benefits to Agile planning  as twofold:

    It allows the organization (consultancy or internal IT arm) to be more responsive to demand
    It reduces the demand for change control (even more important from a consulting perspective) or delivering to a fixed scope, which would be highly administrative

 

Agile & Cloud from a Change Management Perspective

Cloud adoption isn’t just about a change of technology, it’s also a massive organizational change.  "The key to success in this new climate isn’t about technology, it’s about management vision. The organization — and its leaders — need a forward-thinking, entrepreneurial culture that capitalizes on this computing power. The mantra above is key to how the Agile mindset can really assist companies in avoiding their pre-existing controlling nature. Instead it can allow the more forward thinking entrepreneurial culture to develop. It’s this culture of entrepreneurship which is required to deliver impactful change via Cloud Adoption. A lot of goals the "C Level" aim for when they decide to adopt the cloud are the same as those the Agile mindset helps to achieve:

    Enabling Innovation
        Organizations moving away from control mindset
        Learning to fail fast and gain organizational learning which is critical for the advancement of technology
    Increased Efficiencies
        Smaller teams with more autonomy and less management
    Quicker Speed of Delivery
        Change in processes
        Iterative development
    Value Realization
        Delivering value earlier in the development chain

I could go on and on, but all of the above benefits are true with both cloud and Agile and, ultimately, in this context they clearly have strong symbiosis.

Agile & Cloud working Together

I have explained why Agile and the cloud work well together from both an organizational perspective and a change perspective and, lastly, I want to take you through a technical perspective. Cloud lends itself well to an Agile approach because of one major concept in the industry – "Infrastructure as Code" – which allows cloud infrastructure deployment to be done in a similar way to software delivery, a way that was impossible for traditional infrastructure.

Understanding Infrastructure as code

Definition: Infrastructure as Code (IaC) is the process of managing and provisioning computing infrastructure via code, rather than physical hardware configuration or the use of interactive configuration tools. This is around the automation of deployments of infrastructure which has been made possible with the growth of Cloud Computing. (

As Infrastructure is now being managed more and more via code like software has in the past, moving and adopting Agile Frameworks makes sense for cloud infrastructure delivery. This is why in the industry we see Agile becoming more and more preferred. The main blockers to this at the moment are organisational rather than technical.

So, as highlighted above, cloud and software development have many similarities. They both:

    Use Programming Languages (python, ruby, json, etc.)
    Use Code Management
    Follow Continuous Integration and Deployment (CI/CD)
    Require Testing (Regression, UAT, Integration)

However there are some differences:

    In the Cloud you are responsible for the environments and the underlying infrastructure of the code. E.g. Having to use code to create and manage your staging environment that other code sits on. This is not the case with software which depends on infrastructure being there.
    Infrastructure tends to be very logical and less tangible than software (i.e. you can’t see it) for example Firewall config just ports and IP address or Access Controls with error messages.

These differences can create challenges in my experience when following an Agile methodology:

    Demos pose a challenge due to the less tangible nature of cloud services, which I have solved in the past by trying to keep them focused on the user and business benefit
    I have also had to split user stories into environments first then use code management to assist with the big picture, this effectively creates a matrix of user stories adding some complexity, but also helps with clarity if done correctly
    Although 95% of cloud has API access and can be done via code some elements are manual which cannot be automated or follow software development practices

So, in conclusion, if I was an enterprise working on any cloud project, I really would recommend an Agile delivery method. It is going to help team’s align with the need for quicker delivery and higher quality of the inputs into the cloud. Additionally it helps organizations get greater value from their project and truly iterate towards the goal they are achieving so that they can recognize some of that value sooner.
 

Managing Covid Lockdown - Top-Down & Bottom-Up Strategy to revive industry & reboot economy

Dear #Indian Government,
Lockdown was essential measure to stop #coronavirus from spreading while we decided a strategy to combat it. But #Lockdown is not a solution & we can't #Lockdown the country for ever. If the farmers don't do farming & industry doesn't manufacture we will be out of essential commodities very soon and that will be even bigger problem than #corona.Govt should immediately publish the plan for  Incremental             UNLOCKDOWN as I call it so that States, Cities & Gram Panchayats can start preparing to implement the plan. As a executive I feel the lock-down announcement was unplanned, it forced each state to do a Unplanned Lockdown without creating a plan  (or defy Modi which no one wanted to do) & without any guidance from central government.
                         Our government is spending more money than ever to provide food & healthcare for needy people but #Revenue collection has become zero and soon government will be out of money. So unless we help STOP #CoronaVirus from spreading and unless industry starts working again and government gets its revenue #India will be in a big problem in few months.So I request all citizen of #India to please #StayHomeStaySafe , help our country to #Stop Corona from spreading , help #India to get our economy back in shape and that can only happen when industry/farmers/workers start working again.

In next few slides & text I am going to explain 4 key goals and then  explain my strategy to solve the problem

1. We need #Lockdown so people do not transmit virus & we stop corona virus from spreading
2. We have Limited Stock of grocery,medicine & consumables so we need farmers & industry to start working again   
3. We need a strategy to remove #Lockdown In Stages & need to start Removing #Lockdown
4. We cannot have One Plan for entire India -  We need 2 plans, one for the cities that are  infected by corona and 2nd for non-infected cities.
1. For Infected Cities we need a plan to Cure people & get Cities back to normal.
2. For Non-Infected Cities we need a plan to Protect City & Start industry & farming
3. We need to revive economy before it collapses further
4. We need to provide work & income particularly to the Poor who have no savings

Please note this is a high level plan to explain the  strategy and for implementation a more detailed plan has to be created to take care of all aspect of governance including points like

• How to cure people infected people? 
• How to prevent new infections ?
• How to identify non symptomatic carrier's of corona?
• How we will do more efficient testing with minimum kits ?
• How to protect out healthcare worker and police from infection 
• How to use Data to prevent new infection?
• How to use Data to identify 'who is getting killed by Corona' and 'who are all immune to Corona'? 
• How to use Data to find why infection is higher in some geography and less in others ?

and many more aspects which are kept out of this high level slides.

1)

2)

3)

4)

5)

6)

7)

8)

9)

10)

11)

12)

13)

Happy to answer any questions or correct any mistakes in my strategy. This is just one strategy, there could be other strategies but what is important is to have some strategy along with a risk mitigation plan and then discussion with stakeholders before we finalize a plan for India. My appeal to government of India is to work with a plan because so far they seem to b working on gut feeling and without any idea of what is planning. I hope better sense will prevail and there are thousands of voulenteers who are ready to leave their job and work for India if it can help save few lives. Jai Hind Jai Bharat.

World Health Organization recommended handrub formulations

How to prepare hand sanitizer as recommended by World Health Organization

Note : The official WHO website has detailed guidelines and this blog in only a reference to the WHO recommendations as of 23 March 2020. For latest updates please visit https://www.who.int/ and read the latest guidelines

To help countries and health-care facilities to achieve system change and adopt alcohol-based hand-rubs as the gold standard for hand hygiene in health care, WHO has identified formulations for their local preparation. Logistic, economic, safety, and cultural and religious factors have all been carefully considered by WHO before recommending such formulations for use worldwide. At present, alcohol-based hand-rubs are the only known means for rapidly and effectively inactivating a wide array of potentially harmful microorganisms on hands

WHO recommends alcohol-based handrubs based on the following factors:

1. evidence-based, intrinsic advantages of fast-acting and broad-spectrum microbicidal activity with a minimal risk of generating resistance to antimicrobial agents;
2. suitability for use in resource-limited or remote areas with lack of accessibility to sinks or other facilities for hand hygiene (including clean water, towels, etc.);
3. capacity to promote improved compliance with hand hygiene by making the process faster and more convenient;
4. economic benefit by reducing annual costs for hand hygiene, representing approximately 1% of extra-costs generated by HCAI
5. minimization of risks from adverse events because of increased safety associated with better acceptability and tolerance than other products 

Please read detailed instruction at who website link 
 https://www.who.int/gpsc/5may/Guide_to_Local_Production.pdf

Use Mobile Data & Predictive Analytics To Prevent Spread of #Corona Virus

For most of us, our cell phones are our lifelines. Our cell phone witness and capture a lot of information about what we do, see, and share in our daily lives. We may not note or recall where all we have traveled and which route we took through the but a cell phone data can give accurate log of your activities to the second. , Cell phones are recording your activities every second and sending it to telephone company, app company and Google. As I have mentioned earlier as well, a small set of data by itself seldom has any value but by data analysis and correlation with other data sets we can tell a lot about people. There are these products called Predictive Analytics tools that can this data to predict your next move. Today every person carries a mobile phone and we use CDR (maintained by mobile company) & tower data (collected by mobile company for ech tower) to find how many people were around a particular cell phone user at any given time. (Read my older blog on How Google collects cell phone data including your voice data to know what we are dealing with today, but that's not related to today's post)

Let's assume a person 'A' has been infected with #Corona Virus  and another person 'B' was in close proximity of person A. Predictive analysis can scan through huge bulk of location data and highlight the phone numbers that were present in the locality at the given time. We can also trace their activity for a duration of time and create a profile of the person and predict

1) What is his residential area of the #Carona infected person

2) What are the addresses he frequented in last X days

3) What are the number of his close associates and by correlation who are their close associates?

4) What is a persons daily routine?

5) Does he own a personal vehicle or does he use public transport  (public transport has regular stops & fixed route)

6) By further correlation with data from financial institutions we can predict the persons credit-card or banking details

7) We can tell about his family members or the people who stay with the infected #Carona patient

8) We can identify the people who came within 1 meter of distance and could possibly be infected

Plan of Action for Govetrment of India

In the figure below if we analyze the CDR (Call Data Record of all mobile towers frequented by the #CoronaVirus patient and crete a virtual Geo fence of 1 meter radius around him, we can find how many people went close than 1 meter and so can possibly be infected by yhe  #CaronaVirus patient. All people who were in close prozimity of a infected  #CaronaVirus patient should be quantined as per the current goverment rule and kept under observation for 14 days. Mobile CDR can be analyzed using a Predictive Analytics software that can not onoly predict potential people who can get the virus but also show various trends of the #CoronaVirus. 

Suggested Plan:

1. Make a list of all confirmed #CaronaVirus positive cases
2. Get CDR data of patient from mobile companies ( Need Magistrate permission or blanket approval from PMO)
3. Get details of all mobile phone within 1 meter radius of #Carona Patient as ‘Potential Cases’
4. Ask local authority to contact all ‘Potential Cases’ and inquire about symptoms
5. Local authority & doctors should put all ‘Potential Cases’ on 24/7 monitoring for 14 days and check daily status 
6. Data engineer can write a software to analyze the CDR data and identify phone number of ‘Potential Cases’ who were in close contact of confirmed #CaronaVirus patient
7. Call Data Record  along with Tower Data record can be used to predict gorwth of #Corona cases and also help us in identifying ways to stop further infections 
8. Preditive Analysic has huge potential and can provide surprising predictions and there should be no delay in implementing Data Analytics
9. Government should take help of voulenteers from provate software companies to work on this initiative 

At the minimum a central  Data Analytics team of goverment of #India should be collecting data from all cities that have reported Carona Virus cases. The Data should be sent to the central team ASAP the city team gets some new data about the patient. This data set is cleansed and fed to the Predictive Analysis software to create a model and then predict various scenarios based on historical data ( in this case historical data can be just 3 weeks old) . The Call Data Record data can be collected from mobile serice providers like Vodafone, BSNL, Airtel & Jio on request from goverment.  private individual cannot get this data for security reasons. 

Is Function as a Service for your enterprise?

The name “serverless computing” means that the business or person that owns the software system does not have to provision servers or virtual machines for the back end code to run on. The provisioning of server is managed by service providers like AWS, Azure or Google and guarantees low latency content delivery network.

How does FaaS work?

Developers identifies a serverless architecture provider like Amazon Web Services (AWS) or Google Cloud to enable FaaS functionality for a web application. FaaS integration means some application code gets delivered from the 'Regional Edge Server' and availability and geographical distribution of Edge Servers is an important consideration. A user in India accessing a site that has FaaS code served from an server in Washington may encounter high latency that leads to high bounce rates. Taking example of AWS that has its global network divided as regions and within region edge locations that deliver service from the server closer to the web user delivering low latency and better user experience.So FaaS is very much closer to PaaS (Platform As A Service) , but there’s one difference. In FaaS, you don’t need to deploy the whole application in the cloud. Instead, you can deploy a part of it as a service.

Lets take example of few use cases for various vendors and their serverless offerings

Amazon Lambda

Lambda is a serverless computing offering from Amazon Web Services (AWS). Many of you will wonder what this is. Let me tell you that this offering from AWS will change the way application development and its cloud hosting is done.With Amazon Lambda, you can directly run your code without even managing the servers. For example, you upload an image in the server, now the Lambda function will work for you and will automatically resize the image according to the device a user is using, be it mobile, laptop, desktop or a tablet.To use Lambda services, you need to take a plan that will be “pay as you use.” This means you’ll need to pay only for what you are using.AWS Lambda also helps developers to scale the applications by running the required codes whenever someone triggers.

How does serverless computing works:

1. Developer writes modular code for a functionality in the code editor.
2. Developer deploy the code & configure it to be triggered by other AWS services or in-app  activities/events.
3. Once code is deployed AWS Lambda will run the code when it gets triggered by an event or another service.
4. On AWS Lambada you only pay for the compute time of the deployed code.

Google Cloud Functions

Google Cloud Functions is a serverless computing service launched by Google in 2018. Google cloud functions enable developers to write codes to extend and connect to cloud services i.e., Google’s public cloud. With Google cloud functions, website developers can call particular functions which can perform specific tasks which when triggered gives specific response. These responses are automatically triggered by the Google Cloud Platform (GCP). To provide a seamless experience to the developers, Google introduced the service Google Cloud Functions & made it easier for the developers to to  focus more on the coding part rather than developing infrastructure for the application. Scalability is also one of the significant throwbacks of this service. This service automatically scales the resources whenever they are needed and puts them aside when not. With the presence of conventional coding languages like JavaScript and Python, it is easier for the developers to develop the codes and upload them according to the functions.

Microsoft Azure Functions

Azure Functions is the FaaS launched by Microsoft. Microsoft Azure Functions are designed by keeping the developers in mind. This service helps the developers to accelerate the application designing and development process. Microsoft’s idea behind this FaaS is to eliminate the time consumed by the application infrastructure development process. Now when this is eliminated, the user can easily create the software application and upload its code.  The users now have to pay only for the time which is consumed by the function to run. Also, this service is hosted on Microsoft’s Public Cloud.

IBM Cloud Functions

Based on Apache OpenWhisk, IBM Cloud Functions is a polyglot functions-as-a-service (FaaS) programming platform for developing lightweight code that executes on demand. FaaS helps in developing lightweight codes that are executed when demanded. The service is very similar to that of AWS Lambda and Google Cloud Functions.

Some other popular examples of FaaS are Iron.io & Webtask.io

Summarizing

FaaS lets developers write and update a piece of code on the fly, which can then be executed in response to an event, such as a user clicking on an element in a web page. This makes it easy to scale code and is a cost-efficient way to implement microservices.

Advantages of using FaaS?

Improved developer velocity
With FaaS, developers can spend more time writing application logic and less time worrying about servers and deploys. This typically means a much faster development turnaround.

Built-in scalabilitySince FaaS code is inherently scalable, developers don’t have to worry about creating contingencies for high traffic or heavy use. The serverless provider will handle all of the scaling concerns.

Cost efficiencyUnlike traditional cloud providers, serverless FaaS providers do not charge their clients for idle computation time. Because of this, clients only pay for as much computation time as they use, and do not need to waste money over-provisioning cloud resources.

Drawbacks of FaaS?

Less system controlWhen a third party manage part of the infrastructure it is tough to understand the whole system and debigging becomes more challenging.
Testing becomes more complex  as it is  very difficult to incorporate FaaS code into a local testing environment, making thorough testing of an application a more complex task.

Monitoring become difficult as 

The idea of serverless architectures is appealing or developers. Having sai that FaaS cannot be a complete replacement for traditional application architectures.
To build a basic web application as FaaS, it would take a lot of functions. Application that are using microservices architecture are ideal for replacing potential microservice style architectures and background type services.

Experience Level Agreements (XLAs)