Showing posts with label Architecture. Show all posts
Showing posts with label Architecture. Show all posts

Monday, August 9, 2010

Gartner: What they say about the place for Enterprise Architecture 2010 forward

The following article :
http://biztech2.in.com/india/news/enterprise-solutions/enterprise-architecture-enters-trough-of-disillusionment-gartner/88972/0 argues to the point of how EA when mature should be the bridge to help Business Strategy become IT Strategy ... not vice versa. Business Strategies should always drive Technology. You don't start with a Technological solution or Tool and try to mold the business around it.

Here is an intresting quote from the Article:

"The artificial walls between business and IT are crashing down, and EA is the bridge to integrate business and IT," said Philip Allega, Research Vice President, Gartner. "EA's original promise was its ability to provide future safe guidance given the desires and vision of an organisation's senior leadership team. As IT roles shift away from technology management to enterprise management, EA is suited to bring clarity to these blurred boundaries, and, by 2015, increased adoption of EA processes and uses by business will further IT's alignment with the organisation's culture, future-state vision and delivery of business value outcomes."


I agree with this assessment.

Tuesday, July 13, 2010

SOA, The Good - The Bad - The Ugly


Jens Andexer Executive IT Architect @ IBM and Willem Bekker, Head of Architecture Initiatives @ Standard Bank created this comparision chart that does a great job to outlining the benefits, challenges, and pitfalls of moving toward a Service Oriented Architecture within your company.

From my experience I have found that this information is very true.

Here is a link to the original article. I have included the matrix in my blog for my personal future reference. The credit for the article goes to Jens and Willem.

Good Business Implications of SOABad Business Implications of SOAUgly Business Implications of SOA
Agility -
SOAs allow business processes to be developed more quickly and it allows them to be changed more easily. This allows organizations to adapt to changes in their business environment more rapidly. This translates into a real market advantage since it allows products and services to be brought to market more quickly than competitors.
Change in the organization structure -
At the heart of every SOA is a Centre of Excellence (COE). The COE is a new entity that controls the technical development of the SOA and provides expertise to the rest of the organization. The SOA COE is a new addition to any organization and hence its introduction can lead to conflict when it is resourced and when decisions made here effect the rest of the organization.
It's not easy - Transforming and organization to become services oriented is not easy. The transition is to a SOA is characterized by an evolution and not a revolution. Organisations that have traditionally been organised as silos my need to change their structure to take full advantage of becoming services oriented. This shift can be complex, expensive and there is no shortage of opponents to the change.
Alignment -
A closer cooperative relationship between business and IT removes traditional barriers the hindered the IT implementation of business requirements. The footprint of a service in the business domain is a business function and it is described in business terms. The details of its implementation are hidden.
Change in organizational power structure
Placing the ownership and control of services into the domain of business changes the power structure in organizations. This is typically met with resistance from those who have a vested interest in keeping the status quo.
Change in culture -
A SOA represents more than a change in technology. Attendant to this type of transformation comes a change in an organization's culture as it becomes value driven. The organization must learn what it means t be agile and how best to exploit this agility for itself. The ugly truth is that this is one of the most difficult lessons to learn.
Business Process Improvements -
Typically any SOA is associated with a rethinking of business processes. This business process reengineering is an opportunity to optimize how the organization operationally is going about its business. Doing a good job of this reengineering can make a significant improvement to a business's operational efficiency.
New challenges for business -
Business must get more involved in giving direction to IT. Business lines must take ownership and responsibility for services and their enhancements hence initiating the development and change cycle as they will drive this process. This is not a role typically filled by business lines and will make for an uncomfortable change.
Flexibility -
The adherence to good software engineering practice in SOA allows IT to be more responsive to business needs. Time to market for products and services is shortened and the cost of development and change process is reduced.
IT landscape becomes more complex before it can become simpler -

SOA is enabled with a set of technologies like a business process execution engine and ESB. Adding technology to an IT landscape does not make it simpler even when the advantages out way the costs.

But just because the IT landscape is more complex does not mean it can not appear simpler. The introduction of the service allows the complexities in the IT to be a secret. Consumers of the services need to know nothing about what goes on inside a service. As a result any rationalization that takes place in the back end can be hidden behind the service interface.

Technology alone will probably not show value -
A SOA that focuses on infrastructure and technology is likely to fail. A SOA initiative is founded on the promise of delivering business value more quickly and more cheaply than before. A SOA that is too technology focused is unlikely to deliver on that promise since they will not show value in terms business people want to see it. Flexibility is only recognised as business value when it accelerates operationalising business requirements or reduces the cost of operational systems by allowing their rationalisation. Technology focused initiatives typically do not do this.
Data Unification -
The service interface provides an opportunity to unify characteristics of the data so that service interfaces consume data that adheres to a unified data model. Common here means common:


  • Structure - the structural relationships between elements is the same so that for example an address consistently consists of a house number, street, city, region and post code.

  • Semantics - semantic refers to the meaning and use of the data. Data must have a consistent meaning and must not be used in a way that can be misleading. For example the concept of a customer might be hits on a web site in contrast to a set of owners of accounts.

  • Format - how data is represented is important. Dates that are formatted DDMMYYYY must not be confused MMDDYYYY.

  • Type - A type is determined by the representation of data and the set of behavior that can be performed on it.

  • Timing - Timing refers to when an attribute is updated. In some cases attributes are updated by the front end system in real time. However, some attributes are only updated periodically by batch operations.

  • Life cycle - under what circumstances data is add to data bases and when it is updated and when and how it is finally deleted from data bases
No One view of data -
Standard interfaces for services require a common view of the data. This common view typically does not exist and trying to develop it often shows how disparate views in the organisation may be.
Unification may not be possible -
To get all the characteristics of data consistent is seldom possible. Beyond the issues above are the ever present problems associated with "dirty data". Dealing with the inconsistencies is one of the great challenges of designing service interfaces. The ugly truth is that a uniform service interface is very difficult to build.
Operational Monitoring -
The technologies and principles used to enable a SOA make monitoring business processes easier. This type of monitoring allows feedback from daily operations. This feedback can be used to measure how well the organization is doing against its strategic goals
Traditionally business processes (BP) and presentation logic where all written into the same programs that contained business logic. The best one could hope for was that the different types of logic where at least grouped together but even this is often not the case. The upshot of this is that it makes monitoring of categories of logic very difficult. For example business processes could only be monitored as part of the application since it could not be separated out.
SOAs, typically come with a business process execution engine. The introduction of this technology encourages business process logic to be located in one spot. Having the BP logic in one spot makes BP monitoring possible without the need to have the BP logic in the application logic. This is not unique to a SOA but the technology used to enable a SOA along with the discipline that comes from SOA's good software engineering practices make it easier in a SOA.
Monitoring complexity -
Developing a monitoring model for business processes that feeds back to organizational goals is a significant piece of work that requires specialist skills.
Leveraging operational systems -
A SOA uses operational systems to supply the business functionality for services. This means that the investment in existing systems can be used by repackaging it into services.
Technical misfits -
In some cases operational systems do not relent easily to being repackaged as is the case
when the structure of business functions does not match the requirements. For example if a transaction to add a new customer has a commit point after the name and address has been added to the customer data base and a second commit point after the security data has been added then these two operations are inexorably linked.
Changes may be needed -
In some cases operational systems may need to be changed. When the changes need to be made in ERPs vendors are reluctant to make these changes. If the organization decides to make the changes in house the maintenance cost must be considered in the funding of the service.

Tuesday, June 1, 2010

Doing Business without and EA is like building a chain of "open" links

This is directly repeated from this article. No credit to me on this but very profound and true .. so I am repeating it here so that I have a copy for myself if the link dies :) The credit for this insight goes to Nick Malik who is an Enterprise Architect with Microsoft Corporation.

"There are a thousand hacking at the branches of evil to one who is striking at the root." – Thoreau

Doing Business without and EA is like building a chain of "open" links

I had the occasion today to have a long chat with a senior architect about Enterprise Architecture. The conversation turned to the “underlying problem” that EA is supposed to solve. In other words, if we use terms like “alignment” or “simplification,” what is the problem that causes misalignment and complexity?
To paraphrase Thoreau, I asked “what is the root of the evil we are striking at?”
The answer was simple, and glaringly obvious. Communication.


To whit:


•When a leader communicates, no one under that leader wants to admit that they cannot (or did not) completely understand every word.

•When a project is planned, few people want to admit that the project is (or will be) incapable of producing the desired effect. This is especially true if their leader is the one who thought the project up.
As long as communication is flowing in one direction, from a leader to his direct report, then it is easy for the next person down the line to “drift” a little, performing work that may or may not produce the result that the leader expected. Such a system needs to be controlled in order to prevent that drift.

It would be like manufacturing a product on an assembly line without ever checking, along the way, that the intermediate parts are meeting quality control checks. When a part consistently fails a quality control check, you would go back up the line to find the systemic causes of the defect, fix the root cause, verify the result, and continue with manufacturing. This “quality feedback loop” is essential to producing a good product on a consistent basis by building quality into a product rather than trying to insert quality through massive inspection regimens.

The assembly line that EA addresses is the line that starts at strategy and ends at execution.
There is no question that being able to execute on good strategies is a required competency of a successful company. Execution is everything. And what is the mechanism that controls the quality of that execution, reducing defects and building quality into it along the way? Enterprise Architecture.

I’m going to jump to another metaphor here… this one is more visual. This is the metaphor that was going through my head during the conversation.

The chain metaphor


A company can operate entirely without enterprise architecture, just as you can build a chain composed entirely of “open” (or incomplete) links (left).
Such a chain will hold some weight, and it may lighter than a full chain. Potentially such a chain may be inexpensive to create. Sounds compelling, doesn’t it?
But look at the real world. How many “open link” chains have you actually seen? Why is that?
A chain of open links cannot absorb changes easily. It is weak and frail. A sudden change in the weight bearing load, or a strong push in one direction or another, and the chain fails.
For this reason, chains have closed links (right). Stronger, responsive to change, easier to handle.


Let’s apply this metaphor to the way a business operates. The chain in question is the chain of planning activities that starts with the formulation of strategies and culminates in a series of key changes in the way a business operates. The desired outcome of these changes, all summed up, is the realization of that business strategy.




Enterprise Architecture is in the unique position of addressing these “open” links by creating a quality feedback loop at every step of the way. Using Enterprise Architecture, Business Architecture, and Solution Architecture, the chain becomes stronger, more flexible, and less expensive to own.



This role of Enterprise Architecture is interesting, and completely distinct from that of other roles in the company. Just as a developer who tests his own code is prone to bugs, and a lawyer who represents himself has a fool for a client, a business executive who both articulates strategy and tries to rationalize the subsequent goals is betting against himself.
Enterprise architecture does not create the strategy. But it is essential for helping to insure that execution follows effectively from it.

One day Alice came to a fork in the road and saw a Cheshire cat in a tree. Which road do I take? she asked. Where do you want to go? was his response. I don't know, Alice answered. Then, said the cat, it doesn't matter.—Lewis Carroll

Thursday, April 29, 2010

Not A Bad Year Afterall

Its easy as busy as we stay during a year to feel overwhelmed and think that you aren't accomplishing as much as you would like to. In preparation for my Annual Review I had to prepare and Accomplishments document to present to my manager. Preparing the document made me feel better ... it hasn't been a bad year after all :) Read below to see into a year of my professional life:

Accomplishments --


As a Systems Architect

Created the Software Architecture, Data Architecture, and Infrastructure Architecture to enable the TeleVox Notifications Platform to progress toward the goal to support 10X growth while upgrading to a new overall Platform Architecture utilizing new technologies and hardware to ensure Customer Trust in our ability to deliver their notifications and never miss a call (currently over 5 million distinct notification per week and growing). Championed and led strategic initiatives centered on platform reliability and scale, capacity and port expansion, business intelligence, enterprise application integration, business process improvement, customer data centralization, and Notifications Platform enhancement to support growth and position the company for agility in the future.

Software Architecture

Service Oriented Architecture

Designed an architecture to expose WCF and Web Services through a load balanced Services Tier to enable service composition to define business processes. The Services Tier exposes business functionality to processes and applications through a reusable component library that is maintained and versioned centrally within servers in the farm. The goal of this is to speed the development and deployment of future development initiatives through reuse and will help prevent component versioning issues across multiple tiers and servers within our enterprise.

This architecture also integrates WCF and Web Services with Microsoft UDDI 3.0 to enable service virtualization. Service Virtualization will allow us to maintain service configuration information in a central repository that is discoverable at runtime instead of being hard coded in multiple app.config and web.config files scattered throughout the enterprise.

Enterprise Service Bus

Designed an Architectural approach to utilize BizTalk 2009 and Microsoft ESB ToolKit 2.0 to enable BizTalk 2009 to serve as an Enterprise Service Bus that is at the core of our emerging redesigned Notifications Platform. Utilizing BizTalk in a middleware bus architecture will enable cross process data visibility and enrichment and will enable a development model where subsystems and services can be connected to the bus’s data flow in a decoupled asynchronous manner. The bus’s itinerary based messaging allows for the composition of multiple processes and services into a business or process workflow that can be modified based on changing business requirements at the message routing level dynamically. Also, the Pub/Sub architecture behind BizTalk will allow the company to develop solutions based on Event Driven Architectures and remove the multitude of  legacy heavy polling based applications that we have written to try and chain business processes across multiple tasks, executables, and servers.

Designed and specified the requirements for a two node BizTalk Cluster to enable both HTTP load balancing and host clustering to enable high availability and failover. I have developed near term and future looking designs to horizontally scale the current two node ESB to support near and long term projected growth.

Designed and specified requirements for three SQL 2008 database clusters to support the BizTalk OLTP and DSS/OLAP databases in a high availability manner.

Data Architecture

Physical Data Architecture

Cluster Design

Specified the Architecture, design and requirements for three new Windows 2008/ SQL 2008 64-Bit Clusters to support anticipated and ongoing Notifications Platform reliability and performance needs. These new SQL Clusters are dedicated to supporting OLTP, DSS/Reporting, and OLAP respectively. The new SQL Clusters will enable us to migrate high transactional, decision support, and reporting related tables from our web site supporting customer/product databases to a more appropriate cluster and will increase the responsiveness of Product Websites.

Storage Recommendations and Requirements (SAN)

Specified LUN Size, Data and Transaction File location, and RAID Levels for the new databases being deployed in the VXML Project. I worked with our DBA to ensure the accuracy of these upfront requirements and to share the vision for the ongoing Data Architecture so that upcoming storage requirements could be anticipated and delivered.

Logical Data Architecture

Database segmentation across functionality types (OLTP, DSS, OLAP)

Began the implementation of a strategy to move away from the current monolithic SQL Server design that places databases with different user targets and transactional focus on the same physical database cluster. The ongoing Data Architecture will organize databases according to type within separate physical database clusters (Transactional, Decision Support, Report, and Business Intelligence). This segmentation by type will allow specific database types and database hardware and clusters to be tuned for maximum performance and will provide the hardware infrastructure to support growing capacity and enable horizontal scalability across cluster nodes. Segmenting the databases across purpose specific clusters will also allow for a better security model ensuring that PHI and private information can be stored and accessed securely.

Business Intelligence Strategy

Began the implementation of a BI strategy that will leverage BizTalk’s Business Activity Monitoring subsystem to enable OLAP Data warehouse enrichment based on in flight messages moving through our ESB. OLAP based storage of data flowing through our business processes will enrich the Company’s ability to make educated decisions on platform and customer trends and will allow the systems within the Platform to make intelligent runtime decisions during processing.

Data Centralization Strategy

Customer Key Synchronization

Worked across departments to design a strategy to centralize our customer data across physical data boundries. The Customer Key Synchronization project deliverable allows us to identify our customers across both product and backend systems (CRM, ERP, Accounting, and Billing) based on a matrix of key information that links the customer’s identity across the disparate platforms. This common SQL structure and Services model will allow for data synchronization and customer visibility into data events and business processes that occur from the creation of the customer in our CRM to the accounting and billing process. This ability to understand who our customer is across platforms will feed improvement initiatives around implementation, support, reporting, business analysis, and business intelligence.

Customer Profile Data Store

Designed the data architecture for a Common Customer data store that will centralize the Customer Profile and Preference data at a higher level of abstraction than the current siloed domain data structures within our Products and Backend Systems. Taking this level of abstraction will position us to enable future business intelligence, reporting, and cross datacenter notifications processing initiatives.

Infrastructure Architecture

Designed the Infrastructure Architecture and defined specifications to support several new Platform Tiers. I specified the hardware, networking, configuration and storage requirements to deliver a load balanced Services Tier, a Clustered BizTalk based Middleware Tier, and three new database clusters.

Enterprise Application Integration Strategy (EAI)

Backend and LOB Systems Integration

Developed a strategy for linking our backend systems (CRM, ERP, and Billing) so that business events and processes can trigger data synchronization that can flow in a circular fashion between the systems. This will be accomplished using BizTalk in a Hub and Spoke Architecture to connect the systems and provide a data gateway between the various data stores to allow critical customer data to remain identical between the disparate systems.

Service Virtualization Strategy
Integrated UDDI 3.0 to enable runtime discovery of WCF and ASMX Service URI and service metadata. Utilizing the UDDI 3.0 platform enable our company to remove the reliance on application configuration files for storing this information and allowed our applications and processes to discover the information needed to interact with our services library dynamically. This allows for portability for our services and multiple service reconfiguration based on runtime discovery.

As a Lead Developer/Team Lead

Lead the Platform design and development efforts for the HouseCalls Redesign VXML Project. Mentored team members in Service Oriented design, Object Oriented design, use of Design Patterns, Best Practices for coding and naming conventions, and alignment with the overall Platform Architecture goals.

BizTalk Development

Designed and developed many of the core BizTalk Orchestrations that are involved in the VXML Calling process. Mentored other team members in the development of other orchestrations. Designed the physical architecture for high availability deployment of BizTalk applications.

Core Services Development

Continued development of the Core Services that manage our frameworks and expose enterprise resources through managed best practices based implementation.

Core Components Development

Continued development of the Core Components that manage our frameworks and expose enterprise resources through managed best practices based implementation.

Logging Framework

Expanded the Logging Framework that I developed for the Project Genesis deployment to support asynchronous communications through MSMQ with the Logging Database.

Configuration Framework

Designed and developed a Configuration Framework composed of WCF Services, .Net Component libraries, and a new Database to allow for application and process configuration to be stored a managed centrally in a database. This framework removes the need to update configuration files across multiple services and servers when a configuration value needs to be added, removed, or modified.

As a Systems/Infrastructure resource

Production and PreProd Infrastructure deployment and configuration for VXML Project

Worked independently and with Service Delivery and EIT to build out the hardware infrastructure in the Pre Prod and Production environments to support the VXML Project. I configured VMWare virtual machines, Windows 2008 clusters, administered databases, created Active Directory OU Structures, accounts and groups, created DNS entries for services, and configured Operating System and Server Roles. I also defined the overall Architecture, configuration, and security model for these environments to reliably support the VXML Integration Project application deployment.

Friday, April 23, 2010

The Role of the Enterprise Service Bus

I ran across a good architectural overview of where an ESB fits into a Service Oriented Architecture.  Check it out here: http://www.infoq.com/presentations/Enterprise-Service-Bus

Monday, March 15, 2010

Why I like being a BizTalk Architect/Developer

So why is BizTalk so much fun? Well, on an average day a BizTalk Architect/Developer gets to play in some of the following.
  • Database Design and performance tuning
  • Web servers and Load Balancing strategies
  • Core operating system features such as Windows Clustering
  • Line-of-business systems (CRM, ERP .. etc..)
  • Communication channels and protocols
  • File formats and data transformation
  • XML, XSD
  • Advanced design patterns.
  • Storage Solutions and recommendations
  • Business Process design and orchestration
  • Secure Messaging
  • Systems and Application Integration
  • Network and Infrastructure design and configuration
These are things that a front-end web developer, SharePoint developer or DBA may never have or need exposure to. So, a good BizTalk Developer has to dig in deep and be able to wear many hats in a lot of specific areas.

In my opinion a good BizTalk developer needs to be equally effective in both Software and Infrastructure disciplines which means that you need lots of good tools in the toolkit and it provides a lot of nice toys for your toybox :-)

Thursday, March 11, 2010

SOA 2.0

Finally it seems like the ideas for creating Service Oriented Architectures and Event Driven Architectures are centering around a common theme. SOA 2.0 appears to be the marriage between the two.

I don't know about you but as a BizTalk Architect/Developer, I have been thinking in the "SOA 2.0" style for a while.  The introduction of the Microsoft ESB ToolKit 2.0 opened up BizTalk 2009 as a real contender for a very usable Enterprise Service Bus implementation.

BizTalk has always been event based due to its Publish/Subscribe pattern for processing messages. So, its not a far step for any BizTalker or developer with EAI Hub and Spoke Architecture development experience to understand how this fits in with the Event Driven Architecture model.

I think that SOA 2.0 appears to be a very logical step and a very complimentary joining of what used to be considered two differing approaches in Software Architecture.