Understanding the SOA Reference Architecture

What is SOA?

A quick refresh of  SOA definition, a service-oriented architecture is essentially a collection of services. These services communicate with each other. The communication can involve either simple data passing or it could involve two or more services coordinating some activity.  Service-oriented architectures are not a new thing. The first service-oriented architecture for many people in the past was with the use DCOM or Object Request Brokers (ORBs) based on the CORBA specification 

Wikipedia defines SOA (Service-oriented architecture) as a style of software design where services are provided to the other components by application components, through a communication protocol over a network. The basic principles of service-oriented architecture are independent of vendors, products and technologies.[1] A service is a discrete unit of functionality that can be accessed remotely and acted upon and updated independently, such as retrieving a credit card statement online.

Key points to note are that a service has four properties according to one of many definitions of SOA:

•     It logically represents a business activity with a specified outcome.
•     It is self-contained.
•     It is a black box for its consumers.
•     It may consist of other underlying services

Key Business Benefits of SOA

As a flexible and extensible architectural framework, SOA has the following defining capabilities:

1.  Reducing Cost:Through providing the opportunity to consolidate redundant application functionality and decouple functionality from obsolete and increasingly costly applications while leveraging existing investments.
2. Agility: Structure business solutions based on a set of business and IT services in such as way as to facilitate the rapid restructuring and reconfiguration of the business processes and solutions that consume them.
3. Increasing Competitive Advantage: Provide the opportunity to enter into new markets and leverage existing business capabilities in new and innovative ways using a set of loosely-coupled IT services. Potentially increase market share and business value by offering new and better business services.
4. Time-to-Market :Deliver business-aligned solutions faster by allowing the business to decide on the key drivers of a solution and allowing IT to rapidly support and implement that direction.
5. Consolidation :Integrate across silo’ed solutions and organizations, reduce the physical number of systems, and enable consolidation of platforms under a program of “graceful transition” from legacy spaghetti dependencies to a more organized and integrated set of coexisting systems.
6. Alignment : SOA enables organizations to better align IT to business goals, enabling the business to associate IT with capabilities that an organization wants to achieve in alignment with its strategic plan, leading to both sustained agility and re-use over time.

However, significant challenges in creating an SOA solution still remain:

•     Service identification,
•     Service selection,
•     Service design,
•     Solution element selection and combination,
•     Service modelling
•     Service governance,
•     Interoperability and the ability to identify different components key to the effective design, usage, and evolution of SOA.

For example, from a technical perspective, the architect needs to answer questions such as:

1.     What are the considerations and criteria for producing an SOA solution?
2.     How can an SOA solution be organized as an architectural framework with inter-connected architectures and transformation capabilities?
3.     How can an SOA solution be designed in a manner that maximizes asset re-use?
4.     How can automated tools take the guesswork out of architecture validation and capacity planning?

In order to address these issues, there is need for a SOA Reference Architecture for SOA-based solutions. SOA reference architecture provides a high-level abstraction of an SOA partitioned and factored into layers, each of which provides a set of capabilities required to enable the working of an SOA. Each layer addresses a specific set of characteristics and responsibilities that relate to unique value propositions within an SOA. As mentioned above, underlying this layered architecture is a meta-model consisting of layers, capabilities, Architecture Building Blocks (ABB) or ABB, interactions, patterns, options, and architectural decisions and the relation between capabilities, ABB, and layers. These will guide the architect in the creation and evaluation of the architecture.  Likewise, an ABB represents a basic element of re-usable functionality, providing support for one or more capabilities, that can be realized by one or more components or products; examples of the responsibilities of an ABB include: service definition, mediation, routing, etc.

Addo Agnio Award(A3):Congratulaons Ajay Barve! You have been recognized! Certificate

Addo Agnio Award(A3):Congratulaons Ajay Barve! You have been recognized!

What are the key steps for SOA implementation ?

Planning Key phases of SOA implementation

1.     SOA Adoption Planning
2.     Service Inventory Analysis
3.     Service-Oriented Design (Service Contract)
4.     Service Logic Design
5.     Service Development
6.     Service Testing
7.     Service Deployment and Maintenance
8.     Service Usage and Monitoring
9.     Service Discovery
10.     Service Versioning and Retirement

 

SOA Adoption Planning

It is during this initial stage that foundational planning decisions are made. These decisions will shape the entire project, which is why this is considered a critical stage that may require separately allocated funding and time in order to carry out significant studies required to assess and determine a range of factors, including:

• scope of planned service inventory and the ultimate target state
• milestones representing intermediate target states
• timeline for the completion of milestones and the overall adoption effort
• available funding and suitable funding model
• governance system
• management system
• methodology
• risk assessment

Additionally, prerequisite requirements need to be defined in order to establish criteria used to determine the overall viability of the SOA adoption. The basis of these requirements typically originate with the four pillars of service-orientation described in the SOA Planning Section of this Website.

 Service Inventory Analysis

A service inventory represents a collection of independently standardized, owned, and governed services. The scope of a service inventory is expected to be meaningfully "cross-silo," which generally implies that it encompasses multiple business processes or operational areas within an organization.

Service-Oriented Analysis (Service Modeling)

Service-Oriented Analysis represents one of the early stages in an SOA initiative and the first phase in the service delivery cycle. It is a process that begins with preparatory information gathering steps that are completed in support of a service modeling a sub-process that results in the creation of conceptual service candidates, service capability candidates, and service composition candidates
The Service-Oriented Analysis process is commonly carried out iteratively, once for each business process. Typically, the delivery of a service inventory determines a scope that represents a meaningful domain of the enterprise, or the enterprise as a whole. All iterations of a Service-Oriented Analysis then pertain to that scope, with each iteration contributing to the service inventory blueprint.
A key success factor of the Service-Oriented Analysis process is the hands-on collaboration of both Business Analysts and Technology Architects. The former group is especially involved in the definition of service candidates within a business-centric functional context because they understand the business processes used as input for the analysis and because service-orientation aims to align business and IT more closely.

Service-Oriented Design (Service Contract)

The Service-Oriented Design phase represents a service delivery lifecycle stage dedicated to producing service contracts in support of the well-established "contract-first" approach to software development. The typical starting point for the Service-Oriented Design process is a service candidate that was produced as a result of completing all required iterations of the Service-Oriented Analysis process (Figure). Service-Oriented Design subjects this service candidate to additional considerations that shape it into a technical service contract in alignment with other service contracts being produced for the same service inventory. Different approaches are used during this stage for the design of REST services, as these types of services share a common universal contract.
As a precursor to the Service Logic Design stage, Service-Oriented Design is comprised of a process that steps Service Architects through a series of considerations to ensure that the service contract being produced fulfills business requirements while representing a normalized functional context that further adheres to service-orientation principles. Part of this process further includes the authoring of the SLA, which may be especially of significance for cloud-based services being offered to a broader consumer base via the SaaS cloud delivery model.

Service Logic Design
 

The Service-Oriented Design phase represents a service delivery lifecycle stage dedicated to producing service contracts in support of the well-established "contract-first" approach to software development.
The typical starting point for the Service-Oriented Design process is a service candidate that was produced as a result of completing all required iterations of the Service-Oriented Analysis process (Figure 1). Service-Oriented Design subjects this service candidate to additional considerations that shape it into a technical service contract in alignment with other service contracts being produced for the same service inventory. Different approaches are used during this stage for the design of REST services, as these types of services share a common universal contract.
As a precursor to the Service Logic Design stage, Service-Oriented Design is comprised of a process that steps Service Architects through a series of considerations to ensure that the service contract being produced fulfills business requirements while representing a normalized functional context that further adheres to service-orientation principles.
Part of this process further includes the authoring of the SLA, which may be especially of significance for cloud-based services being offered to a broader consumer base via the SaaS cloud delivery model.

Service Development 

After all design specifications have been completed, the actual programming of the service can begin. Because the service architecture will already have been well-defined as a result of the previous stages and the involvement of custom design standards, service developers will generally have clear direction as to how to build the various parts of the service architecture.
For organizations employing the PaaS cloud delivery model, the service development platform itself may be offered by a ready-made environment hosted by virtual servers and geared toward the development and maintenance of cloud-based services and solutions.

Service Testing

Services need to undergo the same types of testing and quality assurance cycles as traditional custom-developed applications. However, in addition, there are new requirements that introduce the need for additional testing methods and effort.
For example, to support the realization of the Service Composability principle, newly delivered services need to be tested individually and as part of service compositions. Agnostic services that provide reusable logic especially require rigorous testing to ensure that they are ready for repeated usage (both concurrently as part of the same service compositions and by different service compositions).
Below are examples of common Service Testing considerations:

1. What types of service consumers could potentially access a service?
2. Will the service need to be deployed in a cloud environment?
3. What types of exception conditions and security threats could a service be potentially subjected to?
4. Are there any security considerations specific to public clouds that need to be taken into account?
5. How well do service contract documents communicate the functional scope and capabilities of a service?
6. Are there SLA guarantees that need to be tested and verified?
7. How easily can the service be composed and recomposed?
8. Can the service be moved between on-premise and cloud environments?
9. How easily can the service be discovered?
10. Is compliance with any industry standards or profiles (such as WS-I profiles) required?
11. If cloud deployed, are there proprietary characteristics being imposed by the cloud provider that are not compatible with on-premise service characteristics?
12. How effective are the validation rules within the service contract and within the service logic?
13. Have all possible service activities and service compositions been mapped out?
14. For service compositions that span on-premise and cloud environments, is the performance and behavior consistent and reliable?

Because services are positioned as IT assets with runtime usage requirements comparable to commercial software products, similar quality assurance processes are generally required.

 

Service Deployment and Maintenance

Service deployment represents the actual implementation of a service into the production environment. This stage can involve numerous inter-dependent parts of the underlying service architecture and supporting infrastructure, such as:

• distributed components
• Web service contract documents
• middleware (such as ESB and orchestration platforms)
• cloud service implementation considerations
• cloud-based IT resources encompassed by an on-premise or cloud-based service
• custom service agents and intermediaries
• system agents and processors
• cloud-based service agents, such as automated scaling listeners and pay-for-use monitors
• on-demand and dynamic scaling and billing configurations
• proprietary runtime platform extensions
• administration and monitoring products

Service maintenance refers to upgrades or changes that need to be made to the deployment environment, either as part of the initial implementation or subsequently. It does not pertain to changes that need to be made to the service contract or the service logic, nor does it relate to any changes that need to be made as part of the environment that would constitute a new version of the service.

Service Usage and Monitoring

The on-going monitoring of the services generates metrics that are necessary to measure service usage for maintenance (such as scalability, reliability, performance etc.), as well as for business assessment reasons (such as when calculating cost of ownership and ROI).
Special considerations regarding this stage apply to cloud-based services, such as:

• the cloud service may be hosted by virtualized IT resources that are further hosted by physical IT resources shared by multiple cloud consumer organizations
• the cloud service usage may be monitored not only for performance, but also for billing purposes when its implementation is based on a per-usage fee license
• the elasticity of the cloud service may be configured to allow for limited or unlimited scalability, thereby increasing the range of behavior (and changing its usage thresholds) when compared to an on-premise implementation

Service monitoring is required for various service governance considerations.

Service Discovery

The primary goal of Service Discovery process is to identify one or more existing agnostic services, such as utility or entity services, within a given service inventory that can fulfill generic requirements for whatever business process the project team is tasked with automating.
The primary mechanism involved in performing Service Discovery is a service registry, which contains relevant metadata about available and upcoming services as well as pointers to the corresponding service contract documents that can include SLAs. The communications quality of the metadata and service contract documents play a significant role in how successful this process can be carried out. This is why one of the eight service-orientation principles, the Service Discoverability principle, is dedicated solely to ensuring that information published about services is highly interpretable and discoverable.

Service Lifecycle,Versioning and Retirement 

After a service has been implemented and used in production environments, the need may arise to make changes to the existing service logic or to increase the functional scope of the service. In cases like this, a new version of the service logic and/or the service contract will likely need to be introduced. To ensure that the versioning of a service can be carried out with minimal impact and disruption to service consumers that have already formed dependencies on the service, a formal service versioning process needs to be in place.
There are different versioning strategies, each of which introduces its own set of rules and priorities when it comes to managing the backwards and forwards compatibilities of services.
The service versioning stage is associated with SOA governance because it can be a recurring and on-going part of the overall service life-cycle. Governance approaches that dictate how service versioning is to be carried out will have a significant influence on how a given service will evolve over time. Because this stage also encompasses the termination or retirement of a service, these influences can further factor into the service's overall lifespan.

Understanding Scrum Framework in one picture

Agile methodology is a practice that helps continuous iteration of development and testing in the SDLC process. Agile breaks the product into smaller builds.

Scrum is a lightweight agile project management framework mainly used for software development. It describes an iterative and incremental approach for project work.


Scrum can be used in all kinds of software development: for developing complete software packages or  for developing only some parts of bigger systems.

The Scrum Framework implements the cornerstones defined by the Agile Manifesto:

• Individuals and interactions over processes and tools
• Working software over comprehensive documentation
• Customer collaboration over contract negotiation
• Responding to change over following a plan

The Scrum Framework itself is very simple. It defines only some general guidelines with only a few rules, roles, artifacts and events. Nevertheless each of these components is important, serves a specific purpose and is essential for a successful usage of the framework.


 Components of Scrum Framework are:

1. The three roles: Scrum Master, Scrum Product Owner and the Scrum Team
2. A prioritized Backlog containing the end user requirements
3. Sprints
4. Scrum Events: Sprint Planning Meeting (WHAT-Meeting, HOW-Meeting), Daily Scrum Meeting, Sprint Review Meeting, Sprint Retrospective Meeting

Project Manager's Role: In Scrum projects there is no project manager in a classical sense. In the Scrum Framework the Scrum Master and the Scrum Product Owner share his responsibilities. However, in the end the team decides what and how much they can do in a given project iteration (Sprint).

Role of Scrum Master: The Scrum Master is important role in the Scrum Framework as it works as a supervisor-reportee with the Scrum Team. His/her main tasks are to make the Scrum team understand how Scrum operates, to protect the Scrum Team from external interruptions and to remove impediments that hinder the Scrum Team to reach its maximum productivity.

Continuous Improvement: Scrum Framework promote continuous improvement: by inspection & adaption. The Scrum Teams have to frequently inspect and assess their created artifacts and processes in order to adapt and optimize them. In the midterm this will optimize the results, increases predictably and therefore minimize overall project risk.

Handling Change in Requirement: In Scrum, changes and optimizations of product, requirements and processes are an integral part of the whole engineering cycle.The Scrum Framework deal with the fact that the requirements are likely to change or are not completely known at the start of the project. The low-level requirements are only defined at the time when they are going to be really implemented.

Scrum Work Style: Communication is a key aspect of Scrum Framework. The Scrum Product Owner works closely with the Scrum Team to identify and prioritize functionality. This functionality is written down in user stories and stored in a Scrum Product Backlog. The Product Backlog consists everything that needs to be done in order to successfully deliver a working software system.

 

Drawbacks of Classical Waterfall Software Development Model

 
Studies have shown that in over 80% of the investigated and failed software projects, the usage of the Waterfall methodology was one of the key factors of failure. But why?
 


Phases in the Classical Waterfall Software Development Model

As shown above, when deploying the waterfall methodology there is a strict sequential chain of the different project phases. A previous phase has to be completed before starting the next phase. Going back is in most cases difficult, costly, frustrating to the team and time consuming.

The project timeline is planned at the start. A releasable product is delivered only at the end of the project timeline. If one phase is delayed all other phases are also delayed.

To avoid this users of the waterfall methodology normally try to anticipate all possibilities beforehand. This means that in one of the earliest phases of the project they try to define all requirements as fine grained and complete as possible. However, requirement definition in such an early phase of the project is often very difficult and therefore many requirements change (or should change) throughout the project. Studies have shown that in larger and complex projects about 60% of the initial requirements are changed throughout the project. Other requirements are implemented as defined but are not really needed by the customer and consume valuable time that could have been better used to implement functionality with a higher added value for the customer.

The separation into different project phases force users to estimate each phase separately. The problem is that most of these phases are normally not separate. They are working together and in parallel.

The waterfall approach for developing software can be used for implementing small and simple projects. But for bigger and more complex projects it can be said that this approach is highly risky, often more costly and generally less efficient than Scrum Project Management Framework.

The iterative waterfall model provides feedback paths from every phase to its preceding phases, which is the main difference from the classical waterfall model. .A quick look at the various options -


Iterative Waterfall Model: The Iterative Waterfall model is probably the most used software development model. This model is simple to use and understand. But this model is suitable only for well-understood problems and is not suitable for the development of very large projects and projects that suffer from a large number of risks.

Prototyping Model: The Prototyping model is suitable for projects, which either the customer requirements or the technical solutions are not well understood. This risks must be identified before the project starts. This model is especially popular for the development of the user interface part of the project.

Evolutionary Model: The Evolutionary model is suitable for large projects which can be decomposed into a set of modules for incremental development and delivery. This model is widely used in object-oriented development projects. This model is only used if incremental delivery of the system is acceptable to the customer.

Spiral Model: The Spiral model is considered as a meta-model as it includes all other life cycle models. Flexibility and risk handling are the main characteristics of this model. The spiral model is suitable for the development of technically challenging and large software that is prone to various risks that are difficult to anticipate at the start of the project. But this model is very much complex than the other models.

Agile Model: The Agile model was designed to incorporate change requests quickly. In this model, requirements are decomposed into small parts that can be incrementally developed. But the main principle of the Agile model is to deliver an increment to the customer after each Time-box. The end date of an iteration is fixed, it can’t be extended. This agility is achieved by removing unnecessary activities that waste time and effort.
     

Principles of SOA Design - Quick Look

There are 9 design principles to keep in mind when designing a Service-oriented Architecture (SOA) service:

1. Standardized Service Contract
Services adhere to a service-description. The Standardized Service Contract design principle essentially requires that specific considerations be taken into account when designing a service's public technical interface and assessing the nature and quantity of content that will be published as part of a service's official contract.
                                                               

2. Loose Coupling
Services minimize dependencies on each other. Coupling refers to a connection or relationship between two things. This principle advocates the creation of a specific type of relationship within and outside of service boundaries, with a constant emphasis on reducing ("loosening") dependencies between the service contract, its implementation, and its service consumers. The principle of Service Loose Coupling promotes the independent design and evolution of a service's logic and implementation while still guaranteeing baseline interoperability with consumers that have come to rely on the service's capabilities.
                                                                          

3. Service Abstraction
Services hide the logic they encapsulate from the outside world. Abstraction ties into many aspects of service-orientation. On a fundamental level, this principle emphasizes the need to hide as much of the underlying details of a service as possible. Doing so directly enables and preserves the previously described loosely coupled relationship. Service Abstraction also plays a significant role in the positioning and design of service compositions.Various forms of meta data come into the picture when assessing appropriate abstraction levels. The extent of abstraction applied can affect service contract granularity and can further influence the ultimate cost and effort of governing the service.
                                                                          

4. Service Reusability
Logic is divided into services with the intent of maximizing reuse. Reuse is strongly emphasized within service-orientation; so much so, that it becomes a core part of typical service analysis and design processes, and also forms the basis for key service models. The advent of mature, non-proprietary service technology has provided the opportunity to maximize the reuse potential of multi-purpose logic on an unprecedented level.
                                                The principle of Service Reusability emphasizes the positioning of services as enterprise resources with agnostic functional contexts. Numerous design considerations are raised to ensure that individual service capabilities are appropriately defined in relation to an agnostic service context, and to guarantee that they can facilitate the necessary reuse requirements.
                                                                     

5. Service Autonomy
Services should have control over the logic they encapsulate. For services to carry out their capabilities consistently and reliably, their underlying solution logic needs to have a significant degree of control over its environment and resources. The principle of Service Autonomy supports the extent to which other design principles can be effectively realized in real world production environments by fostering design characteristics that increase a service's reliability and behavioral predictability.

This principle raises various issues that pertain to the design of service logic as well as the service's actual implementation environment. Isolation levels and service normalization considerations are taken into account to achieve a suitable measure of autonomy, especially for reusable services that are frequently shared.

                                                                                     

6. Service Statelessness
Ideally, services should be stateless. The management of excessive state information can compromise the availability of a service and undermine its scalability potential. Services are therefore ideally designed to remain stateful only when required. Applying the principle of Service Statelessness requires that measures of realistically attainable statelessness be assessed, based on the adequacy of the surrounding technology architecture to provide state management delegation and deferral options.
                                                                       

7. Service Discoverability
Services can be discovered (usually in a service registry). For services to be positioned as IT assets with repeatable ROI, they need to be easily identified and understood when opportunities for reuse present themselves. The service design therefore needs to take the "communications quality" of the service and its individual capabilities into account, regardless of whether a discovery mechanism (such as a service registry) is an immediate part of the environment.

8. Service Composability
Services decomposition break big problems into little problems.As the sophistication of service-oriented solutions continues to grow, so does the complexity of underlying service composition configurations. The ability to effectively compose services is a critical requirement for achieving some of the most fundamental goals of service-oriented computing.

Complex service compositions place demands on service design that need to be anticipated to avoid massive retro-fitting efforts. Services are expected to be capable of participating as effective composition members, regardless of whether they need to be immediately enlisted in a composition. The principle of Service Composability addresses this requirement by ensuring that a variety of considerations are taken into account.

9. Service Interoperability
Services should use standards that allow diverse subscribers to use the service. Interoperability is considered so obvious these days that it is often dropped as a principle. A fundamental goal of applying service-orientation is for interoperability to become a natural by-product, ideally to the extent that a level of intrinsic interoperability is established as a common and expected service design characteristic.

Logo

Useful links -1

Hello World

https://developers.google.com/codelabs/tensorflow-lab2-computervision#0

Build a computer vision model with TensorFlow

https://codelabs.developers.google.com/codelabs/tensorflow-lab1-helloworld#0

 

https://www.datacamp.com/community/tutorials/tensorflow-tutorial

 

https://pythonprogramming.net/machine-learning-tutorial-python-introduction/

 

https://medium.com/ai-india/hello-world-tensorflow-6ce3f5bcbb6b

 

https://www.pythonprogramming.in/learn-tensorflow-series/hello-world.html