14) Enterprise Vision Needs To Be Revisited Before You Prioritize Your Big Data Journey

Big Data planning has to support the enterprise vision and that is why it is critical to revisit and update your 'Enterprise Business & Technology Vision & Road-map' and then start planning for Big Data. This will ensure that  Big Data solution will help the enterprise
1) To identify the Big Data sources within the enterprise and outside the enterprise
2) To identify the Big Data sources that can be tapped for information
3) To filter the Big Data sources that have lower information value
4) To prioritize the Big Data adaption
5) To ensure Enterprise targets 'low hanging' fruits and can demonstrate value from Big Data sooner

An agile enterprise that continues to update its vision and road-map frequently will ensure that the investment in Big Data helps achieve its goals faster and with lower investments.

13) How Big Data Can Fuel Bigger Growth for Your Company

Why should you care about Big Data?

In today’s challenging environment, how will companies grow? The answer is the most valuable yet untapped asset - Customer Information.  Consider how companies are using 'sensors & telematics systems'  in cars to offers lower insurance premiums to its car customers, How online retailers are integrating social media to understand the customer sentiments, how credit card companies are Customer Behavior data to target direct marketers.  It is clear that companies are opening new organic revenue streams by tapping their data & building an information services growth engine.

What can companies do about it?

Before launching a new information business, companies should assess whether they have enough information to differentiate themselves in the marketplace, have data elements that are difficult to replicate in the marketplace, and can combine their data with information from others sources & use sophisticated analysis to create differentiated products. If the assessment reveals there is sufficient revenue potential from an information services business, companies should consider :

1. Consider the best applications of your data (Ex- Marketing, Risk Management, R&D, Operational effectiveness)
2. Compare the attractiveness of the information service to the service offered by existing players (also validate the information services strategy with potential strategic customers) 
3. Understand which important capabilities or assets they will need to win business in the information services 
4. Assess the legal, privacy and policy implications of monetizing information assets
5. Design the most appropriate go-to market approach
6. Use an accelerated prototyping approach to get the new offers into the market quickly

How do you get the started on the journey?

Focus on following 7 step could help companies get started on using Big Data to build a information service business

1. To what extent does your organization use customer data to drive growth?
2. Have you launched new products, services, or businesses focused on the value of your customer information? If so, how successful have those offers been in the marketplace?
3. Do you feel your customer information-based offers are sufficiently differentiated?
4. Do you have a detailed information management strategy tailored to your organization’s unique requirements?
5. How does your organization prioritize information sources as they relate to corporate strategy?
6. With the proper analysis of data from multiple sources, could you develop more innovative and competitive offerings?
7. How much potential insight do you think is locked up in inaccessible data sources?

12) Recap - What is Big Data?

Quick & short recap on 'What is Big Data?'

Today we have got computers is in our pocket, in cash registers, in cars, in TV , in our credit card & everywhere else. Data is being generated by these computers (aka devices) and it is up to the companies to decide if they want to tap this data.  The hardware cost have gone down considerably and companies have realized that they can afford to store more of their data & this analyzing this data will give them a 'more detailed insight of what they are, what their customers want, how companies can improve themselves, how to retain existing customer & get new customer & also save operations cost at the same time'.

According to a research estimate Health Care industry can save up to 300 Billion $. These are just numbers but the fact is Heath Care industry is an early starter and companies have used Big Data to optimize their processes and deliver better, faster,. There are transport companies that are using big dat to optimize the trains running on the lines by optimizing the routes, by optimizing their engines and are helping companies save billions of dollars in fuel cost.

The change has started and companies are adopting Big Data based on the opportunities. The fact is with Big Data related technologies it is possible to store large amount of data, process it in 1/10th of the time and derive value from the data. Companies who adapt to the change over next 10 years will survive the competition. Just collecting data is not enough and unless the data is analyzed companies will not get the value from their Big Data. 

11) A Big Data Use Case - Retail Industry

Let's take an example of a Online Retail Company to understand how they can leverage today's software technologies to leverage Big Data

Challenge faces by online retail company:

•The need to examine massive amounts of unstructured social media and search data to find out what are the 'products that consumers are talking about'

•The growing data volumes causing a major storage problem - leading to data regret on a regular basis

• The need to strategize the ad buying strategy on sites like Google, with the goal of competing for e-commerce sales

• The need to track products, sales, and customers (pet bytes of data) to win pricing concessions from suppliers.

Solution

• Primary basis of solution is co-location of storage and compute layer

• Solution proposes using 'Hadoop' for efficient data transformation

• Solution proposes large proportion of analysis to be performed by Hadoop, MapReduce

   Result

* Over-night processing of data now completes in minutes each day, enabling faster and improved search results

* Data volumes are reduced by as large margin of more than 60%

* Faster analytics that quickly react to changing customer sentiments & market conditions.

Technologies in the solution

• Hadoop, MapReduce, Hive , Pig, Flume, Pentaho, Java 

10) Value of Big Data - All that is old is not Gold & all that is Big Data is not Valuable

The Raw Data from various channels like sensors, instruments, social media usually has to be processed to filter the 'noise or junk data that has no business value' before it be consumed by business system. 

When Big Data is created by a system (a device or an enterprise application or an external source like Facebook) it can either be directly consumed (Ex- For Complex Event Processing) or the data has to be pre-processed before storing in database and this moving data is called as 'Data in Motion'. 

When data is finally stored in some database or a warehouse it is called 'Data at Rest'. There are different benefits that can be extracted from 'Data in Motion & Data at Rest' and the chart below explains the typical steps followed in Big Data processing before Big Data is leveraged for some business outcome. To perform analytic on the data it has to be pre-processed and stored in a database - analytics cannot be performed on raw Big Data.

9) How are Digital Technologies changing the way enterprises (& governments) work?

Digital technologies are changing the way businesses works and also the way employees do their work. I will try to explain the value of Digital Technologies and how they help build Real Time Enterprise Systems that help companies & government to continuously improve & improvise their business process to have a competitive edge in business. In the image below I have explained high level steps to build 'Real Time Enterprise Systems' that leverage standard enterprise data as well as Big Data ( In the image below read, outer circle 1st & then the inner circles)

1) Traditional & Big Data Sources : For any enterprise data is being created by
sensor devices, instruments, emails, social media and the data is of text,audio,video and audio format. Lots of the data is 'noise' and data needs to be cleansed and filtered using smart algorithm before it can be consumed as 'information'
2) Integrated Data & Systems :  Enterprise has many systems & applications and some of them are in silo. A integration bus is essential to integrates the systems and insures smart & secured flow of information across the enterprise.

3) Processed Data : Enterprise runs by taking informed decisions by applying business rules & Business processes to the business data. Data processing & data enrichment is an essential part of a smart enterprise that can take 'Real Time' decisions. Data Processing also involves removing 'noise' from the Big Data that is being created by various sources.

4) Integrated Business Process  : BPM (Business Process Management), Rules Engines, Portal, Mobile applications are the technologies that  help implement integrated BPM in the enterprise and help build an smart enterprise that  takes informed decisions in Real Time as the business events occur in any department in any part of the world. Another advantage of BPM is that a large amount of business decisions can be automated (& work 24/7) & thus business processing can be accelerated. End to end integration ensures that any 'notable event' across the enterprise is monitored & automated decisions are taken in real time.

5) Real Time Enterprise taking Intelligent Business Decisions: The most competitive enterprise have to be Real Time Enterprises. A real time enterprise is an smart enterprise that can take decisions in real time based on information made available to the business. A real time enterprise reduces manual work and uses software to automate a large number of business process and reduces human intervention. The circle in the center of the image above is 'Intelligent Business Decisions' which is the value of implementing a smart real time enterprise by implementing the 4 step steps in the outer circles of the image above. I hope this quick overview was helpful. Do message me if you have any questions.

8) Making a success of Digital India Program kicked off by PM on 1st July 2015

Digital India Program is a new project by Indian government to extend e-governance to the "Gram Panchayat' thus connecting the central government to the basic governance body in smallest entity of our population which is 'The Village" (Gram is Hindi word for Village").

Is Digital India a Revolutionary Step? Yes, it is. This is the first time theIndian  government has committed to connect 'basic governance body of the country to the central governance system' and allocated funds for the program. It is a big thing. This kick starts the Digital India in reality because 'Villages or the Gram' form 75% of India and unless they are connected 'In Real Time' the government is not connected to its people.  The program, according to Govt of India has already got 4.5 Lakh Crore by companies which is very promising.

Here are some key steps that government should take to ensure that Digital India is a success

1) Connectivity : Internet connectivity for all villages in India - Major Challenge
2) Training: Computer Education for office holders of villages & districts - Minor challenge
3) Infrastructure: Electricity or generators for each 'Gram or Village' to use computer.
4) Data Storage: Data is the most important aspect of e-governance. Governments will have to plan for PPP to build a Cloud Data Storage that can be used by Public & Government bodies - from Gram Panchayat and above.
5) Security - Biggest concerns of e-enabling any data is the security risk.
6) Application Design: Mobile friendly design for all government portal applications - Minor Challenge
7) Scalable & High Availability Applications: Existing systems will not be able to handle the huge load of new entities and the applications have to be made scalable and I see this as a major architecture challenge.
8) Continuous Improvement: No systems is perfect and there is need to setup 2 way Communication between Government & Gram Panchayat to get feedback and improve the systems and make them more user friendly and robust
9) Banking:  Banks have to make mobile banking more user friendly and they should reach villages. Public sector banks will have to really push their M-Banking for Digital India to be successful

These are the top 9 concerns that the government should address for successful Digital India initiative.