What Indian goerment should consider for planning Smart City

Smart Cities Mission, sometimes referred to as Smart City Mission, is an urban renewal and retrofitting program by the Government of India with the mission to develop 100 smart cities across the country making them citizen friendly and sustainable. The Union Ministry of Urban Development is responsible for implementing the mission in collaboration with the state governments of the respective cities.

Well defined vision,good planning and technology leaders/architects are needed to build this smart city ecosystem. They need to operate in the intersection of technology, innovation, business, operations, strategy and people. This is the “no man’s” land where traditional boundaries, processes, policies and rules fail. This is where the hardest problems are. and that's the key challenge to implement smart city.

In building the cities of tomorrow, these smart city ecosystem architects must focus on some key areas:

1. Break silos and build bridges. 

A sustainable and well functioning smart city is a seamless integration and smooth orchestration of people, processes, policies and technologies working together across the smart city ecosystem. The architects unify teams across municipal departments to achieve the goal. There is need to connect public and private organizations within the ecosystem & build consensus to co-create the new city.

2. Sound Vision and well defined goals. 

A smart city is not about technology, but about using technology together with the various ecosystem layers to create the ecosystem. These results should be aligned around the needs of the city – government efficiency, sustainability, health and wellness, mobility, economic development, public safety and quality of life.

3. Engage a broader community of innovators. 

Within the smart city, innovation and value creation comes not only from municipal agencies but from businesses, communities (business districts, “smart” buildings, housing complexes), and individual residents. Smart city ecosystem architects unify the various layers to enable, incentivize, facilitate and scale this larger community to co-create the smart city together.

4. Invest in policy making and partnerships at the beginning

Policies and partnerships are the catalysts of the smart city. Policies and partnerships leverage and amplify limited city resources and capabilities, help to scale faster, while minimizing risk. The smart city ecosystem architects address the needs of policymakers, technologists and innovators to create sensible policies that create the right outcomes. Policy makers need to create platform to proactively seek out public and private collaborators and build sustainable and synergistic partnerships.

5. Create unified data and not data islands

Data is the lifeblood of the smart city. Open data, generated by municipal organizations, is only one source of data. When supplemented with data created by businesses and private citizens, it yields richer insights and better outcomes. Smart city ecosystem architects utilize the full extent of the ecosystem to create “unified data”. They plan and build data marketplaces, robust data sharing and privacy policies, data analytics skills, and monetization models that facilitate the sourcing and usage of “city data”.

6. Manage connectivity as a strategic capability. 

While connectivity is mission critical, today’s smart city ecosystem architects are faced with several challenges – unequal access to basic connectivity, inadequacy of existing services & a confusing array of emerging wireless network. In the smart city, connectivity is not an option nor is it someone else’s problem to solve. Smart city architects must lead with new policies and public private partnerships. They must develop new innovative investment strategies & create new connectivity ecosystems with city owned, service provider owned, and community owned infrastructure

7. Smart City needs modern IT infrastructure. 

Most of the smart city infrastructure is confused integration of legacy systems, purpose built departmental technology and smart city point solutions. Cities must modernize their digital infrastructure, while expanding integration to the broader external ecosystem. Cyber-security and technology policies, processes and systems must be revised to be smart city centric, not IT centric. Digital skills, from data analytics, machine learning to software engineering, must be the new competencies of the smart city.

8. Design  Secure Systems

The smart city is only as smart as the trust its stakeholders have in it. Smart city architects must design for trust across the entire ecosystem. The technology infrastructure must be secure. Information collected must be protected, and used protecting owners’ privacy. Policies, legislation and technology must be continuously aligned to maintain the right balance of protection, privacy, transparency & utility. The infrastructure must be robust, resilient and reliable.

7) Discussing Digital - What is Digitization? What are the advantages of Digitization? What do we mean by Digital India?

What is' Digital'?  What is Digitization?

Digital information exists as one of two digits, either 0 or 1. The term Digitization refer to converting information of diverse nature (ex text, audio, video, image) getting converted to binary code. Digitization is 1st step in making information available to share and collaborate across the government and enabling software systems to implement Business Process Management (which means to automate the government business over internet).

Digitization has many advantages

1) Information in digital format can be made accessible to users via internet in a controlled and secure manner. In analog format (paper/ photograph, video tape) it is not easy to share the information.

2) Information can be easily stored and maintained preserved as compared to analog format. 

3) With cost of hardware and internet bandwidth decreasing the 'operations cost to store and maintain information'

4) Once information is in digital format it can be shared and consumed across 'Business Process' from different Business Systems and partners and improve the speed of doing business.

5) Digitized data can be shared, searched, processed & analyzed using sophisticated software and this helps in providing Business Insights to the enterprise (public, private, government) enabling the enterprise do better business. 

Once can imaging Digitization as a process to pull information from different papers and putting it into a single 'word document'. Once the information is in this word document one can share it with another department in the company, search the documents, extract information from the document to complete some 'business process' and at the same time one can 'provide secured access' to the document to selected people based on their access right. 

What is Digital India program? 

Digital India program is Indian government's program to 'Digitize' all information created & consumed by the government agencies (such that it can be saved, shared, searched in a secured manner by the government & its partners) & integrating all public and government agencies from 'Villages, to Towns, to Cities to Indian Government systems'.  Digital India will enable government to  expand its existing e-governance program to entire population & provide a seamless e-governance. 

One can say that digital revolution started way back when 'internet of things' was made affordable and available to common man. E-governance took off in a big way in private companies & government over last 10 years and most Indian states have implemented e-governance to some degree. E-Business model became successful in 1990's and shortly Governments started implementing E-Governance by e-enabling Government to Government services, Government to Citizen services and Government to Business services.  

6) What real life business problems does Big Data solutions solve?

What types of business problems can a big data platform help you address? 

a) Each industry have multiple sources of Big Data (Sensor devices data, Location specific data, GPS data, Email, social networking data, transaction data,instrument logs etc) 

b) Each industry has unique 'information' that can be extracted from big data – by analyzing larger volumes of data than was previously 'not possible', to derive precise answers, to analyzing 'big data in motion as it is created' and to capitalize on the business opportunities that were previously lost. 

c) A big data solution enables the organization to tackle complex problems that previously could not be solved because of the complexity due to sheer volume, required processing speed, the number of different data sources that needed to be processed and the time required to process the entire set of data

Here are a few examples of industries that are leveraging Big Data : 

1) Healthcare Industry : Healthcare industry is among the top 10 industries that is leveraging Big Data. A hospital can reduce patient mortality rate by using Big Data solution to analyze huge amount of patient health data & use it to aid diagnosis and better treatment for the patient. 

 2) Telecommunication : By using Big Data solution for analyzing CDR (call data records) and switch and tower data telecom companies can reduce the processing time by as much as 75% 

3) Electricity & Power Industry : By using Big Data solution power companies can analyze the logs and prevent power outages by performing preventive repairs. 

4) Airline Industry : Airplane has a complex software management system and it generates a huge amount of data when a plane flies. By using Big Data solution airlines are analyzing the plane instrument logs to detect issues and perform preventive maintenance. 

 I have given few examples from real life solutions implemented by various industries and there are many more examples for different industries where big data has been processed and consumed to give the Business a competitive edge.

5) Who is creating Big Data? How fast?

Just in case you have not seen this image of how data is getting created every minute - this minute!  (source pininterest)

 

Another favorite image  that gives an idea of Big Data generation is this one (source wikibon)

 

 

 

4) Why today's business should leverage their company's Big Data?

Integration of Big Data solution (that leverages Big Data relevant to the company) with the traditional Business Intelligence solution is what will give the complete value to any business. As the competition increases and business are looking for 'intelligence' to improve their product, reduce inventory, increase sales, understand customer sentiments,  prevent losses and wastage, it is imperative that all data 'relevant' sources are analyzed and tapped for information and the data is leveraged to give 'Edge' to the company business.

The following table gives a comparison of BI vs New Age BI leveraging Big Data

 

3) Big Data Characteristics - 5 Vs of Big Data

To understand Big Data let's discuss the characteristics of Big Data. Big Data has 5 dimensions (or characteristics) : Volume, Variety, Velocity, Veracity and Value. Let's briefly go through the popular definitions of the 5 V's

1) Volume: Volume refers to the vast amounts of data generated every second. If we take all the data generated in the world between the beginning of time and 2008, the same amount of data will soon be generated every minute. This makes most data sets too large to store and analyze using traditional database technology. New big data tools use distributed systems so that we can store and analyze data across databases that are dotted around anywhere in the world.

2) Variety: Variety refers to the variety of data generated today. Text, Audio, Video, Device Data, GPS data, Facebook data, Call Data Records, Air Flight Logs and 100s of other data types contribute to Big Data.

3) Velocity: Velocity refers to the High Speed at which data is getting generated today. For example- Data generated by Stock Exchanges is high speed data, GPS of a travelling car or a plane generates data at high velocity, Each mobile towers generates CDR data at very high velocity and one of the Big Data challenge is how to process huge volume of data that is generated at such high velocity.

4) Veracity:  Having a lot of data in different volumes coming in at high speed is worthless if that data is incorrect. Incorrect data can cause a lot of problems for organizations as well as for consumers. Therefore, organizations need to ensure that the data is correct as well as the analyses performed on the data are correct. Especially in automated decision-making, where no human is involved anymore, you need to be sure that both the data and the analyses are correct.

5) Value - All the data generated by different devices may or may not have any value for your business. While designing the Big Data solution it is required to decide which data is relevant for business and also filter the 'Noise Data' before you store & process the Big Data.

 The following image from IBM is my favorite Big Data infographic. Picture this image and you will never forget the key characteristics of Big Data. 

Some data scientists consider 'Visualization' as the 6th V of Big Data but I do not agree that Visualization is a characteristic of Big Data. So what is visualization? Is it related to Big Data? What is Big Data Analytics? Visualization is a discipline of business analytics and it is about using tools to play with your data & analyze it to derive business value. Tools like Tableau, Qlikview are some of the leading visualization tools. We will discuss Visualization in my future post. 

Image Source - http://www.ibmbigdatahub.com/infographic/four-vs-big-data

2) What is driving Big Data?

Current patterns of thought on storage, compute and analytics are being challenged.

1) Need to Dealing with Unstructured Data (Ex- Log Processing,Firewall activity, Image/ Video processing,Seismic processing)
2) Need To Reducing Data Storage & Processing Costs (Ex-Move ETL / ETL into parallel environment,  Pre-Proessing EDW, Integrating Enterprise DW with unstructured data sources)

3) Demand for Large Scale Data Analytics (Ex-Modeling the individual user,Large data sets without sampling,Cross enterprise data sets)

4) Require Agile Business Intelligence (Ex-Flexible “schema on read” data space, The “magnetic” data warehouse)

Which means there is a need to integrate the New Gen Data with traditional Business Intelligence Data

1) What is Big Data?

What is Big Data?

•Big Data comes into play as data sets become big enough to obscure underlying meaning and traditional methods of storing, accessing, and analyzing break down. Adding unstructured or semi-structured data to the mix creates additional layers of complexity.

•Corporations are dealing with exploding quantities of data…

•Lots of available data is unused; lots of unstructured data is being generated and…

•The demand to combine this data and explore it to drive deeper insights, to predict rather than react is creating a demand for complex analytical capabilities with agility.

Image source - http://blog.sqlauthority.com

GOTO 2014 • Microservices • Martin Fowler

GOTO 2014 • Microservices • Martin Fowler

https://www.youtube.com/watch?v=wgdBVIX9ifA&list=TLPQMjcxMjIwMTn4vRu8EOXJ-g&index=3

Understanding the SOA Reference Architecture

What is SOA?

A quick refresh of  SOA definition, a service-oriented architecture is essentially a collection of services. These services communicate with each other. The communication can involve either simple data passing or it could involve two or more services coordinating some activity.  Service-oriented architectures are not a new thing. The first service-oriented architecture for many people in the past was with the use DCOM or Object Request Brokers (ORBs) based on the CORBA specification 

Wikipedia defines SOA (Service-oriented architecture) as a style of software design where services are provided to the other components by application components, through a communication protocol over a network. The basic principles of service-oriented architecture are independent of vendors, products and technologies.[1] A service is a discrete unit of functionality that can be accessed remotely and acted upon and updated independently, such as retrieving a credit card statement online.

Key points to note are that a service has four properties according to one of many definitions of SOA:

•     It logically represents a business activity with a specified outcome.
•     It is self-contained.
•     It is a black box for its consumers.
•     It may consist of other underlying services

Key Business Benefits of SOA

As a flexible and extensible architectural framework, SOA has the following defining capabilities:

1.  Reducing Cost:Through providing the opportunity to consolidate redundant application functionality and decouple functionality from obsolete and increasingly costly applications while leveraging existing investments.
2. Agility: Structure business solutions based on a set of business and IT services in such as way as to facilitate the rapid restructuring and reconfiguration of the business processes and solutions that consume them.
3. Increasing Competitive Advantage: Provide the opportunity to enter into new markets and leverage existing business capabilities in new and innovative ways using a set of loosely-coupled IT services. Potentially increase market share and business value by offering new and better business services.
4. Time-to-Market :Deliver business-aligned solutions faster by allowing the business to decide on the key drivers of a solution and allowing IT to rapidly support and implement that direction.
5. Consolidation :Integrate across silo’ed solutions and organizations, reduce the physical number of systems, and enable consolidation of platforms under a program of “graceful transition” from legacy spaghetti dependencies to a more organized and integrated set of coexisting systems.
6. Alignment : SOA enables organizations to better align IT to business goals, enabling the business to associate IT with capabilities that an organization wants to achieve in alignment with its strategic plan, leading to both sustained agility and re-use over time.

However, significant challenges in creating an SOA solution still remain:

•     Service identification,
•     Service selection,
•     Service design,
•     Solution element selection and combination,
•     Service modelling
•     Service governance,
•     Interoperability and the ability to identify different components key to the effective design, usage, and evolution of SOA.

For example, from a technical perspective, the architect needs to answer questions such as:

1.     What are the considerations and criteria for producing an SOA solution?
2.     How can an SOA solution be organized as an architectural framework with inter-connected architectures and transformation capabilities?
3.     How can an SOA solution be designed in a manner that maximizes asset re-use?
4.     How can automated tools take the guesswork out of architecture validation and capacity planning?

In order to address these issues, there is need for a SOA Reference Architecture for SOA-based solutions. SOA reference architecture provides a high-level abstraction of an SOA partitioned and factored into layers, each of which provides a set of capabilities required to enable the working of an SOA. Each layer addresses a specific set of characteristics and responsibilities that relate to unique value propositions within an SOA. As mentioned above, underlying this layered architecture is a meta-model consisting of layers, capabilities, Architecture Building Blocks (ABB) or ABB, interactions, patterns, options, and architectural decisions and the relation between capabilities, ABB, and layers. These will guide the architect in the creation and evaluation of the architecture.  Likewise, an ABB represents a basic element of re-usable functionality, providing support for one or more capabilities, that can be realized by one or more components or products; examples of the responsibilities of an ABB include: service definition, mediation, routing, etc.

Addo Agnio Award(A3):Congratulaons Ajay Barve! You have been recognized! Certificate

Addo Agnio Award(A3):Congratulaons Ajay Barve! You have been recognized!

What are the key steps for SOA implementation ?

Planning Key phases of SOA implementation

1.     SOA Adoption Planning
2.     Service Inventory Analysis
3.     Service-Oriented Design (Service Contract)
4.     Service Logic Design
5.     Service Development
6.     Service Testing
7.     Service Deployment and Maintenance
8.     Service Usage and Monitoring
9.     Service Discovery
10.     Service Versioning and Retirement

 

SOA Adoption Planning

It is during this initial stage that foundational planning decisions are made. These decisions will shape the entire project, which is why this is considered a critical stage that may require separately allocated funding and time in order to carry out significant studies required to assess and determine a range of factors, including:

• scope of planned service inventory and the ultimate target state
• milestones representing intermediate target states
• timeline for the completion of milestones and the overall adoption effort
• available funding and suitable funding model
• governance system
• management system
• methodology
• risk assessment

Additionally, prerequisite requirements need to be defined in order to establish criteria used to determine the overall viability of the SOA adoption. The basis of these requirements typically originate with the four pillars of service-orientation described in the SOA Planning Section of this Website.

 Service Inventory Analysis

A service inventory represents a collection of independently standardized, owned, and governed services. The scope of a service inventory is expected to be meaningfully "cross-silo," which generally implies that it encompasses multiple business processes or operational areas within an organization.

Service-Oriented Analysis (Service Modeling)

Service-Oriented Analysis represents one of the early stages in an SOA initiative and the first phase in the service delivery cycle. It is a process that begins with preparatory information gathering steps that are completed in support of a service modeling a sub-process that results in the creation of conceptual service candidates, service capability candidates, and service composition candidates
The Service-Oriented Analysis process is commonly carried out iteratively, once for each business process. Typically, the delivery of a service inventory determines a scope that represents a meaningful domain of the enterprise, or the enterprise as a whole. All iterations of a Service-Oriented Analysis then pertain to that scope, with each iteration contributing to the service inventory blueprint.
A key success factor of the Service-Oriented Analysis process is the hands-on collaboration of both Business Analysts and Technology Architects. The former group is especially involved in the definition of service candidates within a business-centric functional context because they understand the business processes used as input for the analysis and because service-orientation aims to align business and IT more closely.

Service-Oriented Design (Service Contract)

The Service-Oriented Design phase represents a service delivery lifecycle stage dedicated to producing service contracts in support of the well-established "contract-first" approach to software development. The typical starting point for the Service-Oriented Design process is a service candidate that was produced as a result of completing all required iterations of the Service-Oriented Analysis process (Figure). Service-Oriented Design subjects this service candidate to additional considerations that shape it into a technical service contract in alignment with other service contracts being produced for the same service inventory. Different approaches are used during this stage for the design of REST services, as these types of services share a common universal contract.
As a precursor to the Service Logic Design stage, Service-Oriented Design is comprised of a process that steps Service Architects through a series of considerations to ensure that the service contract being produced fulfills business requirements while representing a normalized functional context that further adheres to service-orientation principles. Part of this process further includes the authoring of the SLA, which may be especially of significance for cloud-based services being offered to a broader consumer base via the SaaS cloud delivery model.

Service Logic Design
 

The Service-Oriented Design phase represents a service delivery lifecycle stage dedicated to producing service contracts in support of the well-established "contract-first" approach to software development.
The typical starting point for the Service-Oriented Design process is a service candidate that was produced as a result of completing all required iterations of the Service-Oriented Analysis process (Figure 1). Service-Oriented Design subjects this service candidate to additional considerations that shape it into a technical service contract in alignment with other service contracts being produced for the same service inventory. Different approaches are used during this stage for the design of REST services, as these types of services share a common universal contract.
As a precursor to the Service Logic Design stage, Service-Oriented Design is comprised of a process that steps Service Architects through a series of considerations to ensure that the service contract being produced fulfills business requirements while representing a normalized functional context that further adheres to service-orientation principles.
Part of this process further includes the authoring of the SLA, which may be especially of significance for cloud-based services being offered to a broader consumer base via the SaaS cloud delivery model.

Service Development 

After all design specifications have been completed, the actual programming of the service can begin. Because the service architecture will already have been well-defined as a result of the previous stages and the involvement of custom design standards, service developers will generally have clear direction as to how to build the various parts of the service architecture.
For organizations employing the PaaS cloud delivery model, the service development platform itself may be offered by a ready-made environment hosted by virtual servers and geared toward the development and maintenance of cloud-based services and solutions.

Service Testing

Services need to undergo the same types of testing and quality assurance cycles as traditional custom-developed applications. However, in addition, there are new requirements that introduce the need for additional testing methods and effort.
For example, to support the realization of the Service Composability principle, newly delivered services need to be tested individually and as part of service compositions. Agnostic services that provide reusable logic especially require rigorous testing to ensure that they are ready for repeated usage (both concurrently as part of the same service compositions and by different service compositions).
Below are examples of common Service Testing considerations:

1. What types of service consumers could potentially access a service?
2. Will the service need to be deployed in a cloud environment?
3. What types of exception conditions and security threats could a service be potentially subjected to?
4. Are there any security considerations specific to public clouds that need to be taken into account?
5. How well do service contract documents communicate the functional scope and capabilities of a service?
6. Are there SLA guarantees that need to be tested and verified?
7. How easily can the service be composed and recomposed?
8. Can the service be moved between on-premise and cloud environments?
9. How easily can the service be discovered?
10. Is compliance with any industry standards or profiles (such as WS-I profiles) required?
11. If cloud deployed, are there proprietary characteristics being imposed by the cloud provider that are not compatible with on-premise service characteristics?
12. How effective are the validation rules within the service contract and within the service logic?
13. Have all possible service activities and service compositions been mapped out?
14. For service compositions that span on-premise and cloud environments, is the performance and behavior consistent and reliable?

Because services are positioned as IT assets with runtime usage requirements comparable to commercial software products, similar quality assurance processes are generally required.

 

Service Deployment and Maintenance

Service deployment represents the actual implementation of a service into the production environment. This stage can involve numerous inter-dependent parts of the underlying service architecture and supporting infrastructure, such as:

• distributed components
• Web service contract documents
• middleware (such as ESB and orchestration platforms)
• cloud service implementation considerations
• cloud-based IT resources encompassed by an on-premise or cloud-based service
• custom service agents and intermediaries
• system agents and processors
• cloud-based service agents, such as automated scaling listeners and pay-for-use monitors
• on-demand and dynamic scaling and billing configurations
• proprietary runtime platform extensions
• administration and monitoring products

Service maintenance refers to upgrades or changes that need to be made to the deployment environment, either as part of the initial implementation or subsequently. It does not pertain to changes that need to be made to the service contract or the service logic, nor does it relate to any changes that need to be made as part of the environment that would constitute a new version of the service.

Service Usage and Monitoring

The on-going monitoring of the services generates metrics that are necessary to measure service usage for maintenance (such as scalability, reliability, performance etc.), as well as for business assessment reasons (such as when calculating cost of ownership and ROI).
Special considerations regarding this stage apply to cloud-based services, such as:

• the cloud service may be hosted by virtualized IT resources that are further hosted by physical IT resources shared by multiple cloud consumer organizations
• the cloud service usage may be monitored not only for performance, but also for billing purposes when its implementation is based on a per-usage fee license
• the elasticity of the cloud service may be configured to allow for limited or unlimited scalability, thereby increasing the range of behavior (and changing its usage thresholds) when compared to an on-premise implementation

Service monitoring is required for various service governance considerations.

Service Discovery

The primary goal of Service Discovery process is to identify one or more existing agnostic services, such as utility or entity services, within a given service inventory that can fulfill generic requirements for whatever business process the project team is tasked with automating.
The primary mechanism involved in performing Service Discovery is a service registry, which contains relevant metadata about available and upcoming services as well as pointers to the corresponding service contract documents that can include SLAs. The communications quality of the metadata and service contract documents play a significant role in how successful this process can be carried out. This is why one of the eight service-orientation principles, the Service Discoverability principle, is dedicated solely to ensuring that information published about services is highly interpretable and discoverable.

Service Lifecycle,Versioning and Retirement 

After a service has been implemented and used in production environments, the need may arise to make changes to the existing service logic or to increase the functional scope of the service. In cases like this, a new version of the service logic and/or the service contract will likely need to be introduced. To ensure that the versioning of a service can be carried out with minimal impact and disruption to service consumers that have already formed dependencies on the service, a formal service versioning process needs to be in place.
There are different versioning strategies, each of which introduces its own set of rules and priorities when it comes to managing the backwards and forwards compatibilities of services.
The service versioning stage is associated with SOA governance because it can be a recurring and on-going part of the overall service life-cycle. Governance approaches that dictate how service versioning is to be carried out will have a significant influence on how a given service will evolve over time. Because this stage also encompasses the termination or retirement of a service, these influences can further factor into the service's overall lifespan.

Understanding Scrum Framework in one picture

Agile methodology is a practice that helps continuous iteration of development and testing in the SDLC process. Agile breaks the product into smaller builds.

Scrum is a lightweight agile project management framework mainly used for software development. It describes an iterative and incremental approach for project work.


Scrum can be used in all kinds of software development: for developing complete software packages or  for developing only some parts of bigger systems.

The Scrum Framework implements the cornerstones defined by the Agile Manifesto:

• Individuals and interactions over processes and tools
• Working software over comprehensive documentation
• Customer collaboration over contract negotiation
• Responding to change over following a plan

The Scrum Framework itself is very simple. It defines only some general guidelines with only a few rules, roles, artifacts and events. Nevertheless each of these components is important, serves a specific purpose and is essential for a successful usage of the framework.


 Components of Scrum Framework are:

1. The three roles: Scrum Master, Scrum Product Owner and the Scrum Team
2. A prioritized Backlog containing the end user requirements
3. Sprints
4. Scrum Events: Sprint Planning Meeting (WHAT-Meeting, HOW-Meeting), Daily Scrum Meeting, Sprint Review Meeting, Sprint Retrospective Meeting

Project Manager's Role: In Scrum projects there is no project manager in a classical sense. In the Scrum Framework the Scrum Master and the Scrum Product Owner share his responsibilities. However, in the end the team decides what and how much they can do in a given project iteration (Sprint).

Role of Scrum Master: The Scrum Master is important role in the Scrum Framework as it works as a supervisor-reportee with the Scrum Team. His/her main tasks are to make the Scrum team understand how Scrum operates, to protect the Scrum Team from external interruptions and to remove impediments that hinder the Scrum Team to reach its maximum productivity.

Continuous Improvement: Scrum Framework promote continuous improvement: by inspection & adaption. The Scrum Teams have to frequently inspect and assess their created artifacts and processes in order to adapt and optimize them. In the midterm this will optimize the results, increases predictably and therefore minimize overall project risk.

Handling Change in Requirement: In Scrum, changes and optimizations of product, requirements and processes are an integral part of the whole engineering cycle.The Scrum Framework deal with the fact that the requirements are likely to change or are not completely known at the start of the project. The low-level requirements are only defined at the time when they are going to be really implemented.

Scrum Work Style: Communication is a key aspect of Scrum Framework. The Scrum Product Owner works closely with the Scrum Team to identify and prioritize functionality. This functionality is written down in user stories and stored in a Scrum Product Backlog. The Product Backlog consists everything that needs to be done in order to successfully deliver a working software system.

 

Drawbacks of Classical Waterfall Software Development Model

 
Studies have shown that in over 80% of the investigated and failed software projects, the usage of the Waterfall methodology was one of the key factors of failure. But why?
 


Phases in the Classical Waterfall Software Development Model

As shown above, when deploying the waterfall methodology there is a strict sequential chain of the different project phases. A previous phase has to be completed before starting the next phase. Going back is in most cases difficult, costly, frustrating to the team and time consuming.

The project timeline is planned at the start. A releasable product is delivered only at the end of the project timeline. If one phase is delayed all other phases are also delayed.

To avoid this users of the waterfall methodology normally try to anticipate all possibilities beforehand. This means that in one of the earliest phases of the project they try to define all requirements as fine grained and complete as possible. However, requirement definition in such an early phase of the project is often very difficult and therefore many requirements change (or should change) throughout the project. Studies have shown that in larger and complex projects about 60% of the initial requirements are changed throughout the project. Other requirements are implemented as defined but are not really needed by the customer and consume valuable time that could have been better used to implement functionality with a higher added value for the customer.

The separation into different project phases force users to estimate each phase separately. The problem is that most of these phases are normally not separate. They are working together and in parallel.

The waterfall approach for developing software can be used for implementing small and simple projects. But for bigger and more complex projects it can be said that this approach is highly risky, often more costly and generally less efficient than Scrum Project Management Framework.

The iterative waterfall model provides feedback paths from every phase to its preceding phases, which is the main difference from the classical waterfall model. .A quick look at the various options -


Iterative Waterfall Model: The Iterative Waterfall model is probably the most used software development model. This model is simple to use and understand. But this model is suitable only for well-understood problems and is not suitable for the development of very large projects and projects that suffer from a large number of risks.

Prototyping Model: The Prototyping model is suitable for projects, which either the customer requirements or the technical solutions are not well understood. This risks must be identified before the project starts. This model is especially popular for the development of the user interface part of the project.

Evolutionary Model: The Evolutionary model is suitable for large projects which can be decomposed into a set of modules for incremental development and delivery. This model is widely used in object-oriented development projects. This model is only used if incremental delivery of the system is acceptable to the customer.

Spiral Model: The Spiral model is considered as a meta-model as it includes all other life cycle models. Flexibility and risk handling are the main characteristics of this model. The spiral model is suitable for the development of technically challenging and large software that is prone to various risks that are difficult to anticipate at the start of the project. But this model is very much complex than the other models.

Agile Model: The Agile model was designed to incorporate change requests quickly. In this model, requirements are decomposed into small parts that can be incrementally developed. But the main principle of the Agile model is to deliver an increment to the customer after each Time-box. The end date of an iteration is fixed, it can’t be extended. This agility is achieved by removing unnecessary activities that waste time and effort.
     

Principles of SOA Design - Quick Look

There are 9 design principles to keep in mind when designing a Service-oriented Architecture (SOA) service:

1. Standardized Service Contract
Services adhere to a service-description. The Standardized Service Contract design principle essentially requires that specific considerations be taken into account when designing a service's public technical interface and assessing the nature and quantity of content that will be published as part of a service's official contract.
                                                               

2. Loose Coupling
Services minimize dependencies on each other. Coupling refers to a connection or relationship between two things. This principle advocates the creation of a specific type of relationship within and outside of service boundaries, with a constant emphasis on reducing ("loosening") dependencies between the service contract, its implementation, and its service consumers. The principle of Service Loose Coupling promotes the independent design and evolution of a service's logic and implementation while still guaranteeing baseline interoperability with consumers that have come to rely on the service's capabilities.
                                                                          

3. Service Abstraction
Services hide the logic they encapsulate from the outside world. Abstraction ties into many aspects of service-orientation. On a fundamental level, this principle emphasizes the need to hide as much of the underlying details of a service as possible. Doing so directly enables and preserves the previously described loosely coupled relationship. Service Abstraction also plays a significant role in the positioning and design of service compositions.Various forms of meta data come into the picture when assessing appropriate abstraction levels. The extent of abstraction applied can affect service contract granularity and can further influence the ultimate cost and effort of governing the service.
                                                                          

4. Service Reusability
Logic is divided into services with the intent of maximizing reuse. Reuse is strongly emphasized within service-orientation; so much so, that it becomes a core part of typical service analysis and design processes, and also forms the basis for key service models. The advent of mature, non-proprietary service technology has provided the opportunity to maximize the reuse potential of multi-purpose logic on an unprecedented level.
                                                The principle of Service Reusability emphasizes the positioning of services as enterprise resources with agnostic functional contexts. Numerous design considerations are raised to ensure that individual service capabilities are appropriately defined in relation to an agnostic service context, and to guarantee that they can facilitate the necessary reuse requirements.
                                                                     

5. Service Autonomy
Services should have control over the logic they encapsulate. For services to carry out their capabilities consistently and reliably, their underlying solution logic needs to have a significant degree of control over its environment and resources. The principle of Service Autonomy supports the extent to which other design principles can be effectively realized in real world production environments by fostering design characteristics that increase a service's reliability and behavioral predictability.

This principle raises various issues that pertain to the design of service logic as well as the service's actual implementation environment. Isolation levels and service normalization considerations are taken into account to achieve a suitable measure of autonomy, especially for reusable services that are frequently shared.

                                                                                     

6. Service Statelessness
Ideally, services should be stateless. The management of excessive state information can compromise the availability of a service and undermine its scalability potential. Services are therefore ideally designed to remain stateful only when required. Applying the principle of Service Statelessness requires that measures of realistically attainable statelessness be assessed, based on the adequacy of the surrounding technology architecture to provide state management delegation and deferral options.
                                                                       

7. Service Discoverability
Services can be discovered (usually in a service registry). For services to be positioned as IT assets with repeatable ROI, they need to be easily identified and understood when opportunities for reuse present themselves. The service design therefore needs to take the "communications quality" of the service and its individual capabilities into account, regardless of whether a discovery mechanism (such as a service registry) is an immediate part of the environment.

8. Service Composability
Services decomposition break big problems into little problems.As the sophistication of service-oriented solutions continues to grow, so does the complexity of underlying service composition configurations. The ability to effectively compose services is a critical requirement for achieving some of the most fundamental goals of service-oriented computing.

Complex service compositions place demands on service design that need to be anticipated to avoid massive retro-fitting efforts. Services are expected to be capable of participating as effective composition members, regardless of whether they need to be immediately enlisted in a composition. The principle of Service Composability addresses this requirement by ensuring that a variety of considerations are taken into account.

9. Service Interoperability
Services should use standards that allow diverse subscribers to use the service. Interoperability is considered so obvious these days that it is often dropped as a principle. A fundamental goal of applying service-orientation is for interoperability to become a natural by-product, ideally to the extent that a level of intrinsic interoperability is established as a common and expected service design characteristic.