Microservices Architecture - Enabling Agile, Scalable, technology agnostic services

Microservices is an architectural pattern where software is designed as composition of small independent services that communicate over well-defined APIs. The services could be owned by one team or several small independent teams. The advantage of microservices is since each service is independent it can be updated or replaced and redeployed without affecting other services. It is quite easy to scale a micro-service as compared to complex set of services. If we consider the business user of the application then it makes lot more sense to use microservices for those areas where business needs to deliver enhancements with short turnaround time.  

Since I am working on Healthcare Platform let me take example of a traditional  healthcare application developed as a monolithic architecture. This has a significant drawback in limiting the reuse of components in the development of other solutions. For example, if you have a healthcare application targeting people with type 2 diabetes and you want to adapt that solution for managing patients with Chronic Obstructive Pulmonary Disease (COPD) which is linked to diabetes , there is likely a significant portion of your original diabetes solution that you could reuse, but that would be difficult to do.

    Using a chronic disease platform built as microservices, for example, you could replace the component that interfaces and manages data from connected devices. The services that communicated with a companion healthcare mobile application, or service that captures insights from an Amazon Alexa virtual assistant, would be reusable service in these cases. Implementing a microservices architecture provides a number of advantages in architecting back-end solutions while offering particular benefits in developing digital and connected healthcare solutions.

    So 2 key characteristics of microservices pattern are

    1) The microservices is specialized services and addresses one specific problem
    2) Each component service can be updated or replaced ,redeployed and scaled without affecting other services.
    

    Benefits of Microservices

    
    Service Agility
    Microservices Architecture enables small, independent development teams that take ownership of their services. Teams work independently and deliver faster . This shortens development cycle times and business benefits because of fast turn around time..

Highly Scalable Services
Microservices allow each service to be independently scaled to meet demand for the application feature it  supports. This enables teams to right-size infrastructure needs, accurately measure the cost of a feature, and maintain availability if a service experiences a spike in demand.

Swift Deployment  
Microservices enable continuous integration and continuous delivery, making it easy to try out new ideas and to roll back if something doesn’t work. The low cost of failure enables experimentation, makes it easier to update code, and accelerates time-to-market for new features.

Technological Freedom
Microservices architectures enables development teams to choose the best tool and technology to solve the specific problems. As a consequence, teams building microservices can choose the best language and tool for each service component.
    
Reusable Code
Dividing software into small, well-defined modules enables teams to use functions for multiple purposes. A service component written for a certain functionality can be reused to deliver another component. This promotes service reuse.

Application Resilience
Service independence increases an application’s resistance to total failure. In microservices architecture, applications handle total service failure by degrading functionality and not crashing the entire application. In a monolithic architecture, if a single component fails, it may lead to entire application failure.

    
    
    

Have you seen the very interesting 'Jobs of future index' created by cognizant ?

Have you seen the very interesting 'Jobs of future index' created by cognizant ? Their website says "To benchmark the emergence of new jobs, we have created the Cognizant Jobs of the Future Index© (CJoF Index). Our quarterly index explores the trends and patterns of 50 jobs. The index tracks 50 Jobs of future and is supposed to track the growth every quarter.

I  think it is a great initiative as it shows the technology trend and also the future prospects of 50 key technologies. I am going to keep a watch on this index and post my views on my blog.For now go to the URL and check the index or read my views as I update them on this post. 

The 2 job statistics that interest me are CIO & Data Scientist and here is what the report says.

1. CIO Jobs- For now what I have found is that the CIO/ Director of IT jobs have declined by a fraction from 2016 -2017 but increased by 50% over last 1 year and increased by 1.5% over last 1 quarter - Does that show a slow down this quarter in the very important CIO jobs? I guess not. 
2. Data Scientist Jobs- There is 80% increase in Data Scientist jobs over last 1 year - lovely!

According to the report over the past year (2018), the fastest growing jobs in the CJoF Index were:

• Fashion Designer: +279%
• Solar Engineer: +257%
• Career Counselor: +181%
• Social Media Strategist / Specialist: +172%
• Genetic Counselor: +163%

The slowest growing jobs in the CJoF Index over the past year were:

• Registered Nurse: +7%
• Biomedical Engineer: +10%
• Solar Installer: +12%
• Home Health Aide: +18%
• Aerospace Engineer: +27%

I think an index for India would differ from this one and nurses, solar installer, home health aide and even aerospace engineer to some extent would be featured in fastest growing jobs. These industries are picking up and there is enough scope for expansion in sectors like hospitals, solar for home, health care professionals for home. I understand there are few initiatives where indegeneous aircraft manufacturing is also happening in India though I would not call it job growth but there is increasing demand for aerospace service engineers to cater to growing number of air travelers.

Interesting figures.ad I hope #Cognizant modifies the website to filter the demand by country. This post was made on 30 Jan 2019 and will be updated periodically.

What will be the 2019 Technology Trends? (Published Jan 2019)

As we enter 2019 here is my prediction of technology disruption & technology trends that will create an impact in 2019. I have listed 8 disruptions and  will add 8 more disruptions in areas of my interest in my next post.

1) Artificial intelligence and machine learning will play a significant role in how IT teams can engage and empower employee & entire organizations. By gaining the right insights they can point out to an employee that the tools and applications available deliver everything they require to carry out their job function.Robotic Process Automation will use AI to automate up to 40% repetitive tasks done by likes of accountants, doctors & even chief executives!

2) Affordable Health Care will reach out to more consumers who at present do not have access to healthcare. Big data, AI, Machine learning & analytics will play a key role in this delivering health care over mobile devices and internet. Self help applications will help people connect to find and connect with health service providers remotely and distance medicine will cover broader spectrum of diseases. For  research companies AI will begin to see fruition, particularly in the imaging diagnostic, drug discovery, and risk analytics applications.

3) Data Authenticity is essential to the decision-making process in business & if the data is unverified it could be detrimental to the outcomes of these decisions. Machine learning is also playing a key role in helping to verify workplace data. Artificial Intelligence & machine learning empower employees & applications will continue to get more intelligent and help improve the user experience and streamline business processes.

4) Very Small Businesses will increasingly use  platform services,mobile applications, blogs, websites & social-media to reach out to consumers & get more business. Platform services will have to play a big role in profiling & validating the service provider to provide reliable quality of service & improve the customer experience.

5) AI driven Cybersecurity along with continuous risk profiling, will become more intelligence-driven, and machine learning will play a critical role in gathering this intelligence. Moreover, machines will start making automated decisions and implement them to minimize organizations cyber-risk. Enterprise goal will be to better curate, correlate, and enrich high-volume security alerts to piece together a cohesive incident detection story across disparate landscape.

6) Game changing Blockchain innovations will rock 2019. I am all for concept of immutable record data (which itself is not a new idea) minus blockchain set of technologies! End of 2019 should see a major shift in the way blockchain is implemented. Enterprises that are still waiting to implement blockchain may finally get 1or 2 reasons to smile.

7) Internet Based Video Services will kill DTH in 2019, new service providers will provide stiff competition and will end monopoly of large DTH operators by 2020. DTH policy & tariff changes along with  penetration of broadband and wi-fi will see exponential growth in internet media consumers and steep churn in existing DTH connections in 2019. We should see a steep increase in people buying internet enabled TV and using home wi-fi to watch video/news. DTH operators who move to wi-fi based service will win a large market share of the WIFI_SITCOM Consumer.

8) Finally something that has worried us since 2016 and is likely to strike in 2019 is IoT Security breach. Hackers can attack unsecured IoT devices to create an extensive botnet, so they could push enough traffic to take down the DNS provider. With the number of connected devices growing rapidly, this is a problem. A world full of connected devices and autonomous things has the potential to be a dangerous world. Not because all these robots, drones and connected fridges will turn against us, but because most likely many of these products will have weak security. Security is not a core competence for IoT manufacturers and hackers have an unfair advantage. Any day hackers will exploit connected devices to create a new, global, attack on internet systems.

Another learning resource - Quick Introduction To R and an interactive Datacamp tutorial

For those who want to learn R I am sharing a URL of a short and snappy R Tutorial. This will serve as a handly guide for those beginners who are used to learning from ebook/websites. Check this URL https://www.statmethods.net/index.html


For those who like interactive tutorials Datacamp.com has a free interactive R tutorial which is really good for beginners. Check this tutorial at https://www.datacamp.com/courses/free-introduction-to-r

Datacamp also has many advanced R and Python paid tutorials and its subscription starts from USD 29 monthly to USD 200 yearly. If you are lucky you may get special 1 year package deal to all of their tutorials for  USD 99.





 

Does India have a comprehensive IT Strategy ? Why does India need a CTO ?

For years I have asked one question to leaders from government that I have met at seminars & conferences, if #Yahoo & #Google can afford investing in a CTO, why is India not hiring a CTO? India's data will deliver much more value to #Indian Government than the value from data collected by #Google. Political leaders can't even imagine the monetary and value returns from the insights just from the data analysis. Let me reiterate that here I am just referring to 'value that we will derive from data analytics on a pile of data which is really a fraction of the over all value'. Under s CTO the unified-distributed data strategy will deliver exponential value. 

                     There is no doubt that data empowers people and #India is way behind in using its data because our politicians never took help of IT experts except Congress which brought in Nandan Nilekani in 2009, to lead #Aadhar. As of date #Aadhar seems to be messed up pretty badly because of political ambitions of different parties. Every IT software is built on a vision on what needs to be achieved by that software. Once vision is documented ( as in final ) then a plan is created and software is built. It does not mean once you define a vision it cannot change, a vision can change but not drastically. You cannot build a software to distribute social security and then one fine day make it mandatory for buying a SIM card or use Aadhar for taking daily attendance of employee at work as it is being done in Mumbai Municipal Corporation today. 

                      What we as people need to understand is that we cannot have different political parties, scrapping up previous governments implementation and implement something new just to score political brownies or else because the new leader fancies it! In India the current PM seems to announce first and then tell his team to implement it and it does not work that way! We have seen a series of data leaks reported by Tribune new paper and government has not taken any action against tribune so it is safe to assume the data leaks happened. 

                                 Software like Aadhar is custom made and very complex code. If you keep changing it, eventually it will start having issues like data security as we read in news papers and even more complex issues which politicians cannot even imagine. What is worrying Indian IT leaders is that Unique Identification and Bio-metrics of each Indian is at stake and one mistake and you will take away the privacy of common man, does not matter if he is rich or poor. India needs a formal & steady #ITVision vetoed by all parties No politics! Obviously this cannot be done by Rahul Gandhil or Narendra.Modi so government need to rehire an IT expert like Nandan Nilekani or rehire  some one with similar experience to educate the elected politicians and then coordinate with all political parties & help define the India's IT Vision for next 25 years. So if tomorrow if Mamata Banerjee or some other person becomes Prime minister that person should not scrap or completely alter all the #IT Strategy of the past governments. 

                                 What #Indian people should realize is 

1. The current political leadership is making a mistake by not on-boarding all political parties to define India's IT Strategy. 
2. Just giving a slogan of Digital India does not make the Prime Minister an IT expert so unless he hires someone like Nilekani, Narayana Murthy or someone with similar experience we will not havea sound IT Strategy
3. If a political leader imposes his vision on the country without on-boarding key political parties the strategy is bound to fai when he leaves power. 

 Nandan Nilekani is talking about Data Strategy for India but political leaders are yet to understand the need for a long term IT Strategy  that includes a Data Strategy. What India need is a mature political leadership who understand Information & Technology and instead of thrusting their half baked ideas on the country they hire the CTO and let him do the work ( way too late to talk about it but better late than never )!  

 Note - Nandan Nilekani's post How To Empower 1.3 Billion Citizens With Their Data

Mr Nilekani may not agree but the #Vision for Aadhar and subsequently thr #ITStrategy seems to be changing far too frequently. The politicians don''t understand that you do not play flip-flop with IT strategy every 5 yrs. If this game of political one-upmanship continue it can permanently destroy the systems like Aadhar so much that it will have to be redesigned from scratch. 
                                         We have seen that the concerned citizen of India don''t like to confront the government except a few people who filed Public Interest Litigation in #SupremeCourt. It is fortnate that SupremeCourt was wise enough to strike down the  nefarious changes in #Aadhar Guideline that allowed private companies like banks, telecom companies and insurance companies to use Aadhar. At one point of time citizen have to realize that once we elect government it is our job to monitor it and bock any moves that is not good for the country. If citizen don't speak & act against any nefarious law then we are responsible for the state of the nation. You may not understand what is this data and internet revolution that everyone is talking about but you have to know that it internet & related technologies are already affecting lives of people, it is changing the world, and it is affecting everybody including those in the remote villages. Some of these villages don't even have electricity in 2019 but they have mobile connectivity & internet and people actually travel to nearby town to charge their mobile phone (t!)  US elections and even Indian elections saw matriculation and misuse of of citizens data. Some crazy &ignorant person from government agency TRAI shared his Aadhar number and challenged people to hack his bank account. He did not understand the concerns regarding Aadhar. No one is saying that Aadhar is so fragile that if your number is leaked your bank account can be hacked. At the same time do understand that today there is so much of persons data available on internet that by searching and correlating a lot can be found about the person and it can be misused to do commit some fraud. or at least create trouble for you. The concerns about Aadhar are much larger and much different than simply misusing someones Aadhar number. Beware! India's IT & data strategy or lack of it will decide if #India rules the future or goes back dark age 

Note - I have posted a series of posts about on my blog about different concerns with #Aadhar and they can be read over here What's wrong with Aadhar? My blog posts in chronological order 

How To Plan Your Innovation Journey ? Vision To Implementation In One PIcture

I received queries in response to my earlier post Building The Innovation Culture. So here is a diagram on the go to explain the steps an organization needs to follow to start its innovation journey.

Innovation Journey - Big Picture

Company's Innovation Vision has to be brainstormed, defined & documented (in that order) by the organizations leadership team. When a enterprise invests in Innovation it is critical that the Innovation should deliver some tangible benefits to the enterprise to justify next round of internal-investments. A great innovation without any purpose or an innovation which does not have tangible benefits for the enterprise is a dead investment and may lead to blockage of future Innovation funding. That is why the innovation journey of an enterprise should starts by answering the following key questions. Why , What, Who & How should help you define the Innovation Vision for your organization.

Why? 

• Organization's leadership should answer why should their organization invest in innovation?
• Each leader brings his list of points to support of the case for investment in innovation
• The leadership team brain storms Enterprise's Need For Innovation
• Leadership agrees on 'Why Innovation'

What?

• What should be the areas of focus for innovation?
• What innovations will help business growth, speed to market & value added products?
• What innovation will give the company competitive advantage?
• What innovations will help the organization build disruptive products/solutions? 
• Leadership agrees on Innovation Landscape.

Who?

• Which groups & teams need to be part of innovation journey?
• Not everybody in the organization can work on innovation      
• How will we identify people who will invest in innovation?
• How will the people contribute to innovation?

How?

• How will organization provide groups/employee time & resources to invest in innovation?
• Who and how will we monitor the investment in innovation work?
• How should we set goals & targets for 'Quick Win' innovation?
• How do we decide when to 'Pull the plus' from any innovation initiative?
• How do we recognize & reward innovation?

Once the leadership has answered the above questions they can come out with Innovation vision.
Innovation Vision helps us define the Innovation road-map and the plan to implement the innovation road-map along with key milestones. Once the implementation starts it is critical to monitor innovation investment and ensure adequate support to individuals working on innovation initiatives. The should guidelines clearly define the milestones and  innovation investment has to be reviewed at each milestone. Regular milestone reviews help the organization to monitor the progress and take a decision on continuing the investment or scrapping it.

Recommended Reading - 10 Big Data Visualization Tools Everyone in the Industry Should Be Using

Data Visualization tools like Tableau (my personal favorite) and Qlikview have been a game changer. Today the CEO/CIO/CTO are using these tools to show 'Real Time Data points' in board room meeting, using the tool to slice and dice the data, view historical data graphically and everything else you can/cannot imagine that can be done with a Visualization Dashboard.

If you have never heard of these tools here is a quick read Big Data Data Visualization at  promptcloud.com.

How Can Mobile Device Data,Tower Data,Call Data Records & Predictive Analysis Help Prevent Terror Attacks?

For most of us, our cell phones are our lifelines. Our cell phone witness and capture a lot of information about what we do, see, and share in our daily lives. We may not note or recall where all we have traveled and which route we took through the but a cell phone data can give accurate log of your activities to the second. Cell phones are recording your activities every second and sending it to telephone company, app company and Google, As I have mentioned earlier as well, a small set of data by itself seldom has any value but by data analysis and correlation with other data sets we can tell a lot about people. There are these products called Predictive Analytics tools that can this data to predict your next move. Today criminals & terrorist use this mobile phones and we can cell phones and use CDR & tower data to catch the terrorist. (Just read my blog on how Google collects cell phone data including your voice sample to know what we are dealing with today, but that's not related to today's post)

For example the terrorist who did recent kidnapping and murder of security personal in J& K can we caught using their cell phone data and future kidnapping can be prevented by using predictive analysis of this data. You might think that your cell phone is safe so long as you keep it tucked away in your pocket and do not use applications, but companies have been developing technologies to “force” our phones into giving away information without physically accessing our device. These technologies are called "Cell Site Simulators"or IMSI catchers. One of such popular products is called “Stingray.” The names may change, but all of these devices are regularly being used by USA & UK for law enforcement surveillance, These devices are designed to interfere with cell phone signals by pretending to be cell phone towers. By mimicking towers, the devices intercept signals to gather data, such as metadata and content of phone-calls, personally identifying information, and data usage, and have been especially popular as a tool for tracking the location of particular cell phones.

Technology Architecture for Mobile Surveillance

Let's assume a criminal 'A' has been involved in 10 terror incidents and another criminal 'B' was involved in 2 terror incidents they may be carrying 1 or 2 mobile phone. Predictive analysis can scan through huge bulk of location data and highlight the phone numbers that were present in the locality at the given time. We can also trace their activity for a duration of time and create a 

profile of the person and predict

1) What is his residential area

2) What are the addresses he frequents

3) What are the number of his associates and by correlation who are his associates?

4) What is a persons daily routine?

5) Does he own a personal vehicle or not

6) By further correlation with data from financial institutions we can predict the persons credit-card or banking details

7) We can tell about his family members or the people who stay with him 

As you can see this data analysis tells a lot about people that can help safely identify a persons link with crime incidents.  Security agencies can monitor their activities of shortlisted candidates and trap them to prevent crime. Apart from using the data and tools that I have mentioned Mobile Data activity surveillance requires coordination between telecom companies and government security agencies and USA/UK have been successfully able to build such mechanism and prevent crime.

Drawback - Misuse of telecom data  & prevention 

Increasingly broad use of cell site simulators by law enforcement is also controversial for many reasons. Apart from the fact that the devices themselves indiscriminately invade the privacy of everyone because they connect to and can capture data from, all cell phones within their range, the fact is these devices can also been used in controversial ways for example by deploying them disproportionately in areas made up predominantly of people of color. But this is not a serious threat and a democratic government can define protocol and monitor the use of these devices and captured data by its officials. What is important is that such monitoring devices along with "Smart Software" can monitor cell phones and at the same time secure the captured data thus protecting the privacy of non-criminal elements by way of negating cell profiles that do not have any suspicious activities over a time. Right now the priority of government is to minimize causality in areas affected by terrorism and it is critical to use Cellular Data & Data Intelligence tools to aid our security agencies.

How to upgrade R version without losing your existing installed packages

R is a language & environment for statistical computing and graphics. R was created by Ross Ihaka and Robert Gentleman at the University of Auckland, New Zealand. Currently it is a GNU project which is similar to the S language and environment which was developed at Bell Laboratories.

Here are the steps I performed to reuse the downloaded libraries (saves the paint o reinstall each library) when I upgraded R from 3.5.0 to 3.5.1.

Before any update of R, start the software or Rstudio to know where all packages are installed by typing - .libPaths()

1. Before you upgrade, build a temp file with all of your old packages.

tmp <- installed.packages()
installedpkgs <- as.vector(tmp[is.na(tmp[,"Priority"]), 1])
save(installedpkgs, file="installed_old.rda")

2. Install the new version of R ( as of Oct 2018 latest version is R3.5.1)
3. Once you’ve got the new version up and running, reload the saved packages and re-install them from CRAN.

tmp <- installed.packages()
installedpkgs.new <- as.vector(tmp[is.na(tmp[,"Priority"]), 1])
missing <- setdiff(installedpkgs, installedpkgs.new)
install.packages(missing)
update.packages()