Friday, June 12

Customer Segmentation using Machine Learning

When you get tons of unlabeled data and you want to find some pattern in data to be used for some purpose like segmenting the data on basis of certain characteristics machine learning algorithm can be a big help. Lets take a example of tons of customer data of Target or Amazon or Flipcart. 

To use this data for building some value added service like recommendation engine or showing customer latest treds that he might be interested in or even showing ads that are most appropriate for customer based on his gender, age, location etc we need to first classify the customer into different segments. 

According to a Forrester report, only 33% of companies using customer segmentation find it significantly impactful. The main reason companies fail is that they are still using traditional customer segmentation approaches, without leveraging the breadth of customer data and advanced analytics techniques available today.

What is Customer Segmentation?

Customer Segmentation is one the most important applications of unsupervised learning. Using clustering techniques, companies can identify the several segments of customers allowing them to target the potential user base. In this machine learning project, we will make use of K-means clustering which is the essential algorithm for clustering unlabeled dataset. Before ahead in this project, learn what actually customer segmentation is.

What is Behavioral Segmentation?

Traditional approaches to segmentation focused mainly on who customers are and segments were based on demographic attributes such as gender or age, and firmographic attributes like company size or industry. But just understanding who your customers are is not enough anymore.  Behavioral segmentation is about understanding customers not just by who they are, but by what they do, using insights derived from customers’ actions.

Behavioral Segmentation is a form of customer segmentation that is based on patterns of behavior displayed by customers as they interact with a company/brand or make a purchasing decision. It allows businesses to divide customers into groups according to their knowledge of, attitude towards, use of, or response to a product, service or brand.

Why Segment Customers by Behavior?

Here are four main advantages of grouping customers into different segments based on their behaviors:

  1. Higher Level Of Personalization. Understand how different groups of customers should be targeted with different offers, at the most appropriate times through their preferred channels, to effectively help them advance towards successful outcomes in their journeys.
  2. Behaviourial  Predictivity. Use historical behavioral patterns to predict and influence future customer behaviors and outcomes.
  3. Customer Prioritization. Make smarter decisions on how to best allocate time, budget and resources by identifying high-value customer segments and initiatives with the greatest potential business impact.
  4. Evaluating Segment Performance. Monitor growth patterns and changes in key customer segments over time to gauge business health and track performance against goals. At a high level, this means quantifying the size and value of customer segments, and tracking how “positive” and “negative” segments are growing or shrinking over time.

Sunday, May 24

Arogyasetu - The Health App with doubtful utility!

What is ArogyaSetu application?

ArogyaSetu is a COVID-19 contact tracing mobile application developed by the Ministry of Electronics and IT.  The application detects and tracks the mobile user’s movement with the help of GPS and Bluetooth sensors and sends out a notification if they come in contact with infected people by using its database and algorithms. Users have to enter personal information while registering on the app and the data will be shared only with the Government of India and does not include third parties as mentioned in the app’s privacy policy.

ArogyaSetu will be mandatory for Air travel

From 25th May Indian government is reopened its domestic air travel and made it mandatory for the traveler to have ArogyaSetu application installed on their mobile. So if the status of the ArogyaSetu App is Green and it shows message 'YOU ARE SAFE' then user will be allowed to travel and if the status is Red  and the application shows message the 'You are not safe'  then probably there will be some investigations and traveler may not be allowed to board the flight..

So why did I say ArogyaSetu application should be installed at least 15days before travel

  • #ArogyaSetu Guidelines do not mention time frame for installation of the application. 
  • As of today it is RECOMMENDED but NOT MANDTORY for every Indian to install the ArogyaSetu app. 
  • As per new guidelines if you want to travel by flight then it is mandatory to install the ArogyaSetu app
  • Those who do not use mobile phone will have to give a written declaration about their travel history 
  • This means before traveling the user will be have to install the ArogyaSetuand but security will not know if the user installed app 1 month back or 15 minutes back as the app DOES NOT show that installation DATE on users MOBILE SCREEN. Possibly the data is available in the backend system
  • So for a user who has installed the ArogyaSetufew 15 minutes back the status will always be Green
  • The app is not smart enough to inform the security person that the user USER Status is Safe/Green because USER HAS INSTALLED APPLICATION 15 MINUTES BACK  and it is possible that user may be in contact of infected person before he installed the app.
  • Bottom line is the ArogyaSetu application can be made more smart and intuitive to security & healthcare personal  

Lets understand the issue that I am highlighting with an example -

For example I have the ArogyaSetu application installed on my mobile but my mother is not comfortable with using mobile apps & she does not have ArogyaSetu installed on her phone. IN future if my mother has to travel by air then she will be told to install the ArogyaSetu App on her mobile or she will not be allowed to travel. Problem is there is no guideline on how many days prior to travel should one install ArogyaSetu application. 
        So my mother can install the app as late as just before reaching the airport and the ArogyaSetu app will show Green Status and message that' You are safe'. The dump app does not tell the security personnel that the user has installed the application just before entering the airport. If the Arogyasetu application did not track the users status for last 14 days (or more) and the Dumb App will still show STATUS as GREEN and there is no way to know that user has just installed the app, say 2 hours back or whatever the time period since installation of the ArogyaSetu application


The way ArogyaSetu works is it keeps a record of your movement and if you happen to come in close contact of a #corona patient then it flags you as a potential corona victim and shows your status as RED or UNSAFE Mind you ArogyaSetu showing RED does nto mean you are infected, it only means you were in close proximity of a infected person.But if you did not have ArogyaSetu app installed and then partied with a person who was #corona positive then #ArogyaSetu application is not smart enough to flag you RED. I am not saying ArogyaSetu application cannot be made smart, I am saying this ArogyaSetu app is not SMART.

So what should we do now  to protect Air Travelers from one infected person on the flight ?

Suggestion shared with ArogyaSetu team, Health Ministry Govt of India & Ministry of Civil Aviation are as follows -

  1. A infected passenger traveling on plane is very dangerous so guidelines have to be improved
  2. Modify the ArogyaSetu  application and on the main screen show the "Installation Date"
  3. Unless the application was installed 14 days back the status of the user should be shown as 'Orange' or 'Needs Further Scrutiny'  because for 14 days since contacting a infected person the person can now show any symptoms of corona.
  4. Make it mandatory for users to install application for certain services like Air Travel or travel in AC Train
  5. ArogyaSetu team should inform the government the limitations of the application and dependency on installation date tracking and educate them on what INNOVATIVE THINGS can be added to the existing app.
  6. Take suggestion and ideas from public on social-media platform. Our people should be free to give suggestions. 

Here are couple of slides to explain the above points





Update - 19 flyers on Air India’s Delhi-Wuhan flight test Covid positive on arrival at Wuhan Question is were these passengers not using ArogyaSetu? How did ArogyaSetu fail to identify the Covid positive patient that boarded flight to Wuhan, China ?



 

Friday, May 8

One Slide Plan To Transport MIgrant Workers To Their Home

Here is a one slider on transporting migrant workers from Host State (Example #Maharashtra) to Destination State (Example #Uttarpradesh ) or any other state.






Tuesday, April 28

Using Chef Tool for DevOps Automation

 

A Devops engineer spends more time in deploying new services and application, installing and updating network packages and making machine server ready for deployment. This takes tedious human efforts and requires huge human resources.By using configuration management tools like Chef or Puppet you can deploy, repair and update the entire application infrastructure using automation.

What is Chef?

Chef is a Configuration management tool that manages the infrastructure by writing code rather than using a manual process so that it can be automated, tested and deployed easily. Chef has Client-server architecture and it supports multiple platforms like Windows, Ubuntu and Solaris etc. It can also be integrated with cloud platform like AWS, Google Cloud, and Open Stack etc. 

Understanding Configuration Management


Let us take an example of a system engineer in an organization who wants to deploy or update software or an operating system on more than hundreds of systems in your organization in one day. This can be done manually but it may cause multiple errors, some software’s may crash while updating and we won’t be able to revert back to the previous version. To solve such kind of issues we use Configuration management tools.

Configuration Management keeps track of all the software and hardware related information of an organization and it also repairs, deploys and updates the entire application with its automated procedures. Configuration management does the work of multiple System Administrators and developers who manage hundreds of server and application. Some popular tools used for Configuration management are Chef, Puppet, Ansible, CF Engine, and SaltStack.

Why I prefer Chef Automation Tools?

Let us take a scenario, suppose we want our system administrator to install, update and deploy software on hundreds of system overnight. When the system engineer does this task manually it may cause Human errors and some software’s may not function properly. At this stage, we use Chef which is a powerful automated tool which transfers infrastructure into code.


Chef automates the application configuration, deployment and management throughout the network even if we are operating it on cloud or hybrid. We can use chef to speed up the application deployment. Chef is a tool for accelerating software delivery, the speed of software development refers to how quickly the software is able to change in response to new requirements or conditions

Benefits of Chef


Accelerating software delivery

 By automating infrastructure provisioning we automate all the software requirements like testing, creating new environments for software deployments etc. becomes faster.

Increased service Resiliency

By making the infrastructure automated we can monitors for bugs & errors before they occur it can also recover from errors more quickly.

Lower Risk Management

Automation tool like Chef or Puppet lowers risk and improves compliance at all stages of deployment. It reduces the conflicts during the development and production environment.

Easy Cloud Adoption  

Chef can be easily adapted to a cloud environment and the servers and infrastructure can be easily configured, installed and managed automatically by Chef.

Managing Data Centers & Cloud Env 

Chef can run on different platforms, under chef you can manage all your cloud and on-premise platforms including servers.

Streamlined IT operation & Workflow 

Chef provides a pipeline for continuous deployment starting from build to test and all the way through delivery, monitoring, and troubleshooting.

In summary Chef tools help IT teams adopt modern day best practices including:

  • Test Driven Development: Configuration change testing becomes parallel to application change testing.
  • DevOps Support: IT operations can confidently scale with data consolidations and 3rd party integrations.
  • Self-Service: Agile delivery teams can provision and deploy infrastructure on-demand.

 

 

Sunday, April 19

Why do agile & cloud work well together?

Agile & Cloud from an Organizational Perspective

Working solely on cloud projects for last 8+ years , I have seen that cloud and Agile go hand in hand. How is that?Well, cloud is something new and transformative for a lot of enterprise customers and, because it’s so new, these organizations lack the requirements or exact understanding of how they want to utilise the cloud. Naturally they struggle to articulate what they want and need, which as a problem lends itself nicely to the user story and demo methodologies in Agile.

Agile methodologies make it very easy for companies to articulate on a "high level" what they think they want users to be able to achieve with the cloud. They do this using the three pillars of Agile, which are transparency, inspection, and adaptation(http://www.scrumguides.org/scrum-guide.html for more details).

 

You can demo functionality and certain behaviors to your organization early, and learn to adapt to their needs and help them gain an understanding of what they really want to achieve with cloud technologies. This is also critical for organization as it makes the development transparent  and the demo acts as an inspection element for the company to workout if the product is exactly what they wanted. At Cloudreach this has allowed me to present back to large organization and assist them into getting to grips with how they would like to use the cloud, instead of a more traditional way of planning something which means spending months developing something that the business then might realize it doesn’t want.
 

Benefits to Agile planning  as twofold:


    It allows the organization (consultancy or internal IT arm) to be more responsive to demand
    It reduces the demand for change control (even more important from a consulting perspective) or delivering to a fixed scope, which would be highly administrative

 

Agile & Cloud from a Change Management Perspective


Cloud adoption isn’t just about a change of technology, it’s also a massive organizational change.  "The key to success in this new climate isn’t about technology, it’s about management vision. The organization — and its leaders — need a forward-thinking, entrepreneurial culture that capitalizes on this computing power. The mantra above is key to how the Agile mindset can really assist companies in avoiding their pre-existing controlling nature. Instead it can allow the more forward thinking entrepreneurial culture to develop. It’s this culture of entrepreneurship which is required to deliver impactful change via Cloud Adoption. A lot of goals the "C Level" aim for when they decide to adopt the cloud are the same as those the Agile mindset helps to achieve:

    Enabling Innovation
        Organizations moving away from control mindset
        Learning to fail fast and gain organizational learning which is critical for the advancement of technology
    Increased Efficiencies
        Smaller teams with more autonomy and less management
    Quicker Speed of Delivery
        Change in processes
        Iterative development
    Value Realization
        Delivering value earlier in the development chain

I could go on and on, but all of the above benefits are true with both cloud and Agile and, ultimately, in this context they clearly have strong symbiosis.

Agile & Cloud working Together

I have explained why Agile and the cloud work well together from both an organizational perspective and a change perspective and, lastly, I want to take you through a technical perspective. Cloud lends itself well to an Agile approach because of one major concept in the industry – "Infrastructure as Code" – which allows cloud infrastructure deployment to be done in a similar way to software delivery, a way that was impossible for traditional infrastructure.

Understanding Infrastructure as code

Definition: Infrastructure as Code (IaC) is the process of managing and provisioning computing infrastructure via code, rather than physical hardware configuration or the use of interactive configuration tools. This is around the automation of deployments of infrastructure which has been made possible with the growth of Cloud Computing. (


As Infrastructure is now being managed more and more via code like software has in the past, moving and adopting Agile Frameworks makes sense for cloud infrastructure delivery. This is why in the industry we see Agile becoming more and more preferred. The main blockers to this at the moment are organisational rather than technical.

So, as highlighted above, cloud and software development have many similarities. They both:

    Use Programming Languages (python, ruby, json, etc.)
    Use Code Management
    Follow Continuous Integration and Deployment (CI/CD)
    Require Testing (Regression, UAT, Integration)

However there are some differences:

    In the Cloud you are responsible for the environments and the underlying infrastructure of the code. E.g. Having to use code to create and manage your staging environment that other code sits on. This is not the case with software which depends on infrastructure being there.
    Infrastructure tends to be very logical and less tangible than software (i.e. you can’t see it) for example Firewall config just ports and IP address or Access Controls with error messages.

These differences can create challenges in my experience when following an Agile methodology:

    Demos pose a challenge due to the less tangible nature of cloud services, which I have solved in the past by trying to keep them focused on the user and business benefit
    I have also had to split user stories into environments first then use code management to assist with the big picture, this effectively creates a matrix of user stories adding some complexity, but also helps with clarity if done correctly
    Although 95% of cloud has API access and can be done via code some elements are manual which cannot be automated or follow software development practices

So, in conclusion, if I was an enterprise working on any cloud project, I really would recommend an Agile delivery method. It is going to help team’s align with the need for quicker delivery and higher quality of the inputs into the cloud. Additionally it helps organizations get greater value from their project and truly iterate towards the goal they are achieving so that they can recognize some of that value sooner.
 

Thursday, April 9

Managing Covid Lockdown - Top-Down & Bottom-Up Strategy to revive industry & reboot economy

Dear #Indian Government,
Lockdown was essential measure to stop #coronavirus from spreading while we decided a strategy to combat it. But #Lockdown is not a solution & we can't #Lockdown the country for ever. If the farmers don't do farming & industry doesn't manufacture we will be out of essential commodities very soon and that will be even bigger problem than #corona.Govt should immediately publish the plan for  Incremental             UNLOCKDOWN as I call it so that States, Cities & Gram Panchayats can start preparing to implement the plan. As a executive I feel the lock-down announcement was unplanned, it forced each state to do a Unplanned Lockdown without creating a plan  (or defy Modi which no one wanted to do) & without any guidance from central government.
                         Our government is spending more money than ever to provide food & healthcare for needy people but #Revenue collection has become zero and soon government will be out of money. So unless we help STOP #CoronaVirus from spreading and unless industry starts working again and government gets its revenue #India will be in a big problem in few months.So I request all citizen of #India to please #StayHomeStaySafe , help our country to #Stop Corona from spreading , help #India to get our economy back in shape and that can only happen when industry/farmers/workers start working again.

In next few slides & text I am going to explain 4 key goals and then  explain my strategy to solve the problem
  1. We need #Lockdown so people do not transmit virus & we stop corona virus from spreading
  2. We have Limited Stock of grocery,medicine & consumables so we need farmers & industry to start working again   
  3. We need a strategy to remove #Lockdown In Stages & need to start Removing #Lockdown
  4. We cannot have One Plan for entire India -  We need 2 plans, one for the cities that are  infected by corona and 2nd for non-infected cities.
    1. For Infected Cities we need a plan to Cure people & get Cities back to normal.
    2. For Non-Infected Cities we need a plan to Protect City & Start industry & farming
    3. We need to revive economy before it collapses further
    4. We need to provide work & income particularly to the Poor who have no savings

Please note this is a high level plan to explain the  strategy and for implementation a more detailed plan has to be created to take care of all aspect of governance including points like
  • How to cure people infected people? 
  • How to prevent new infections ?
  • How to identify non symptomatic carrier's of corona?
  • How we will do more efficient testing with minimum kits ?
  • How to protect out healthcare worker and police from infection 
  • How to use Data to prevent new infection?
  • How to use Data to identify 'who is getting killed by Corona' and 'who are all immune to Corona'? 
  • How to use Data to find why infection is higher in some geography and less in others ?
and many more aspects which are kept out of this high level slides.

1)
2)

3)

4)

5)

6)

7)

8)

9)

10)

11)

12)

13)
Happy to answer any questions or correct any mistakes in my strategy. This is just one strategy, there could be other strategies but what is important is to have some strategy along with a risk mitigation plan and then discussion with stakeholders before we finalize a plan for India. My appeal to government of India is to work with a plan because so far they seem to b working on gut feeling and without any idea of what is planning. I hope better sense will prevail and there are thousands of voulenteers who are ready to leave their job and work for India if it can help save few lives. Jai Hind Jai Bharat.

Monday, March 23

World Health Organization recommended handrub formulations

How to prepare hand sanitizer as recommended by World Health Organization


Note : The official WHO website has detailed guidelines and this blog in only a reference to the WHO recommendations as of 23 March 2020. For latest updates please visit https://www.who.int/ and read the latest guidelines

To help countries and health-care facilities to achieve system change and adopt alcohol-based hand-rubs as the gold standard for hand hygiene in health care, WHO has identified formulations for their local preparation. Logistic, economic, safety, and cultural and religious factors have all been carefully considered by WHO before recommending such formulations for use worldwide. At present, alcohol-based hand-rubs are the only known means for rapidly and effectively inactivating a wide array of potentially harmful microorganisms on hands

WHO recommends alcohol-based handrubs based on the following factors:

  1. evidence-based, intrinsic advantages of fast-acting and broad-spectrum microbicidal activity with a minimal risk of generating resistance to antimicrobial agents;
  2. suitability for use in resource-limited or remote areas with lack of accessibility to sinks or other facilities for hand hygiene (including clean water, towels, etc.);
  3. capacity to promote improved compliance with hand hygiene by making the process faster and more convenient;
  4. economic benefit by reducing annual costs for hand hygiene, representing approximately 1% of extra-costs generated by HCAI
  5. minimization of risks from adverse events because of increased safety associated with better acceptability and tolerance than other products 
Please read detailed instruction at who website link 
 https://www.who.int/gpsc/5may/Guide_to_Local_Production.pdf

Wednesday, March 18

Use Mobile Data & Predictive Analytics To Prevent Spread of #Corona Virus




For most of us, our cell phones are our lifelines. Our cell phone witness and capture a lot of information about what we do, see, and share in our daily lives. We may not note or recall where all we have traveled and which route we took through the but a cell phone data can give accurate log of your activities to the second. , Cell phones are recording your activities every second and sending it to telephone company, app company and Google. As I have mentioned earlier as well, a small set of data by itself seldom has any value but by data analysis and correlation with other data sets we can tell a lot about people. There are these products called Predictive Analytics tools that can this data to predict your next move. Today every person carries a mobile phone and we use CDR (maintained by mobile company) & tower data (collected by mobile company for ech tower) to find how many people were around a particular cell phone user at any given time. (Read my older blog on How Google collects cell phone data including your voice data to know what we are dealing with today, but that's not related to today's post)




Let's assume a person 'A' has been infected with #Corona Virus  and another person 'B' was in close proximity of person A. Predictive analysis can scan through huge bulk of location data and highlight the phone numbers that were present in the locality at the given time. We can also trace their activity for a duration of time and create a profile of the person and predict
1) What is his residential area of the #Carona infected person

2) What are the addresses he frequented in last X days
3) What are the number of his close associates and by correlation who are their close associates?
4) What is a persons daily routine?
5) Does he own a personal vehicle or does he use public transport  (public transport has regular stops & fixed route)
6) By further correlation with data from financial institutions we can predict the persons credit-card or banking details
7) We can tell about his family members or the people who stay with the infected #Carona patient
8) We can identify the people who came within 1 meter of distance and could possibly be infected

Plan of Action for Govetrment of India

In the figure below if we analyze the CDR (Call Data Record of all mobile towers frequented by the #CoronaVirus patient and crete a virtual Geo fence of 1 meter radius around him, we can find how many people went close than 1 meter and so can possibly be infected by yhe  #CaronaVirus patient. All people who were in close prozimity of a infected  #CaronaVirus patient should be quantined as per the current goverment rule and kept under observation for 14 days. Mobile CDR can be analyzed using a Predictive Analytics software that can not onoly predict potential people who can get the virus but also show various trends of the #CoronaVirus. 

Suggested Plan:
  1. Make a list of all confirmed #CaronaVirus positive cases
  2. Get CDR data of patient from mobile companies ( Need Magistrate permission or blanket approval from PMO)
  3. Get details of all mobile phone within 1 meter radius of #Carona Patient as ‘Potential Cases’
  4. Ask local authority to contact all ‘Potential Cases’ and inquire about symptoms
  5. Local authority & doctors should put all ‘Potential Cases’ on 24/7 monitoring for 14 days and check daily status 
  6. Data engineer can write a software to analyze the CDR data and identify phone number of ‘Potential Cases’ who were in close contact of confirmed #CaronaVirus patient
  7. Call Data Record  along with Tower Data record can be used to predict gorwth of #Corona cases and also help us in identifying ways to stop further infections 
  8. Preditive Analysic has huge potential and can provide surprising predictions and there should be no delay in implementing Data Analytics
  9. Government should take help of voulenteers from provate software companies to work on this initiative 


At the minimum a central  Data Analytics team of goverment of #India should be collecting data from all cities that have reported Carona Virus cases. The Data should be sent to the central team ASAP the city team gets some new data about the patient. This data set is cleansed and fed to the Predictive Analysis software to create a model and then predict various scenarios based on historical data ( in this case historical data can be just 3 weeks old) . The Call Data Record data can be collected from mobile serice providers like Vodafone, BSNL, Airtel & Jio on request from goverment.  private individual cannot get this data for security reasons. 

MUSTREAD : How can you use Index Funds to help create wealth? HDFC MF Weekend Bytes

https://www.hdfcfund.com/knowledge-stack/mf-vault/weekend-bytes/how-can-you-use-index-funds-help-create-wealth?utm_source=Netcore...