C H A P T E R 6

Enterprise Application
Integration

ENTERPRISE information systems—the collection of relational and legacy data-

base systems, enterprise resource planning (ERP) systems, and mainframe transac-
tion processing systems—provide the critical information infrastructure for an
enterprise’s business processes. These varied systems hold the information that an
enterprise needs to carry out its daily operations. It is essential that new applications
developed for an enterprise be able to integrate with these enterprise information
systems (EIS).
EIS integration has always been of great importance, and this has given rise to
enterprise application integration, or EAI. EAI enables an enterprise to integrate
its existing applications and systems, plus it enables the addition of new technolo-
gies and applications. Enterprises must leverage their existing systems and
resources even as they adopt new technologies. Considering the cost already
invested in these existing systems, no business can afford to discard them. Plus,
since these systems often contain valuable data needed by the enterprise, the
enterprise is not likely to disrupt them. Yet, at the same time, enterprises continu-
ally grow and require new applications. To keep their businesses growing and to
remain cost effective, enterprises must integrate their existing systems with these
new applications and not replace existing systems with new applications written
from scratch. The emergence of Web-based architectures and Web services adds
impetus for enterprises to integrate their EISs and expose them to the Web.
The emergence of the Web and Web services is not the only factor driving the
need for integration. More and more, enterprises are either merging or acquiring
other enterprises. Such mergers and acquisitions usually entail merging and com-

247
248 Integration Requirements and Scenarios

bining two divergent information technology (IT) systems. Not only are the IT
systems different, but, as a further challenge, they each may have standardized on
using different integration technologies within their respective environments.
Emerging Web services standards are another factor driving Web services-based
EAI. These standards are making it possible to integrate heterogeneous systems
easily and cost effectively.
In today’s environment, a typical enterprise has a multitude of existing appli-
cations running on diverse platforms and operating systems. Although these appli-
cations may very well rely on the same or similar data, they keep that data in
different formats. Thus, the integration problem encompasses both data and
system platforms.
These are but a few examples of the complexities that enterprise application
integration must address. Not only must EAI handle integrating applications, it
must also address integrating data and technologies so that enterprises can easily
share business processes and data.
Using several scenarios, this chapter illustrates key integration considerations.
It describes the J2EE 1.4 platform technologies that help with integration and pre-
sents some integration design approaches and guidelines.

6.1 Integration Requirements and Scenarios

Before delving in the details of the technologies, it is helpful to illustrate the major
concerns for integrating enterprise information systems and to get a sense of the
extent of the EAI problem. We discuss three types of integration scenarios: data
integration, application integration, and business process integration. Often, an
enterprise’s integration needs span these different types.

6.1.1 Typical Integration Scenarios

Data integration involves integrating existing data living in different enterprise sys-
tems, and it often occurs when an enterprise relies on multiple types of database sys-
tems. For example, some database systems may be relational, others may be
hierarchical or based on objects, and still others may be file based or even legacy
stores. As an illustration, a newly developed Web-based order management system
might have to integrate with an existing customer order database. Data integration
involves not only integrating different data systems, but it also entails integrating
different informational or data models.
Chapter 6 Enterprise Application Integration 249

Application integration involves integrating new applications with existing or

legacy applications. Since an enterprise’s business relies on the continuity of its
existing applications, it is important to integrate the new applications with
minimal disruptions. Integrating with home-grown legacy applications presents a
bigger challenge, since these systems have no vendor-provided or off-the-shelf
adapter layers. Not only do you have to write the adapter layers yourself, these
home-grown applications may be more idiosyncratic, with an architecture that is
more opaque and difficult to understand.
Business process integration involves integrating an enterprise’s existing
systems to support a set of business processes. A business process is a series of
(often asynchronous) steps that together complete a business task or function. For
example, the adventure builder enterprise has a business process for fulfilling pur-
chase orders submitted to its order processing center. The order fulfillment busi-
ness process includes such steps as validating a customer’s credit card,
communicating with various suppliers to fill different parts of an order, and noti-
fying the customer of the order status at various stages of processing.
Integration is often accomplished by exchanging documents, which more and
more are XML documents, among business processes according to defined busi-
ness rules. The different processes transform the documents by applying their
individual business rules, and then they route the documents to other processes.
Good examples are business processes that do such things as handle purchase
orders and invoices, or incorporate supplier catalogs. Each such business process
runs a workflow that interacts with other entities, either internal or external.

6.1.2 Example Integration Scenarios

Examining some typical scenarios helps to bring these integration requirements into
better perspective. We use the scenarios present in our adventure builder enterprise
example for illustration. One common integration scenario is that of the adventure
builder enterprise, which may want to make its inventory available online to expand
its customer reach. The enterprise may have existing applications and databases—
for example, catalog and inventory databases, along with order processing and cus-
tomer relationship management (CRM) systems—for its business. These systems
and databases need to be enhanced to accommodate the e-store.
Adventure builder enterprise purchases and customizes the CRM package, but
its order processing system may be homegrown. As much as possible, the enter-
prise wants to reduce software duplication and keep its infrastructure costs to a
minimum. To that end, it may want to use these same systems to handle the e-store
250 Integration Requirements and Scenarios

business, especially since its existing customers may also buy products through
the online store. Adventure builder enterprise also wants to leverage its customer
service department and have these same specialists service both types of custom-
ers: store front as well as online. Adventure builder enterprise expects to include
additional databases relevant for the Web site only.
This scenario illustrates an application and data integration problem. The
enterprise’s existing databases store information needed by the e-store, which may
need to update these databases. The databases have existing security settings, plus
protocols for transactions. The vendor for the CRM system may have provided a
J2EE connector that could be used to plug the CRM system into the enterprise’s
J2EE application server. However, the order processing system, since it is home-
grown, may have no such support.
Figure 6.1 illustrates the scenario of this application with respect to the EISs
that it uses. This example scenario touches upon many of the integration require-
ments that pertain to all enterprises. Because it is open to anyone on the Internet,
the adventure builder Web site may potentially have a large number of users,
making scalability and performance important issues. Security is an important
consideration, since adventure builder’s Web site handles customer data that it
must keep private. The enterprise has the further challenge of ensuring that its
legacy systems are not stretched beyond their capabilities and that heterogeneous
platforms may host the EIS systems.
The enterprise relies on an order fulfillment center to process orders placed
through the e-store Web site. A separate department owns the order fulfillment
center, and it uses its own set of databases separate from the e-store’s Web site. To
keep the two data models decoupled, orders flowing between the e-store and the
order fulfillment center are kept in XML format. Communication is also asyn-
chronous, allowing clients to continue their own work while an order is processed.
In essence, the order fulfillment center initiates a business process whenever it
receives an order. The business process, following a set of automated rules, inter-
acts with several systems in a workflow to complete or fulfill the order with no
human intervention. Part of the workflow includes sending confirming e-mails to
customers and keeping records for administrative reports.
Chapter 6 Enterprise Application Integration 251

Order processing
database

Customer
Web site Order processing
system
(legacy)

Adventure
Builder travel
agency

Suppliers

CRM system
(legacy)

Web site
database

CRM database

Figure 6.1 An Application Scenario

The order fulfillment center interacts with external business partners to com-
plete its workflow. For example, the order fulfillment center might rely on a sepa-
rate credit card billing service to process its customer billing. Some of its products
may be supplied directly by another vendor. The order fulfillment process initiates
multiple business processes for each external business with which it interacts.
Let’s look at a different integration scenario. A human resources application,
designed for internal use only, may have a similar scenario to an e-store, since it
concerns a new application that uses existing enterprise assets. However, the
application’s internal use limits the scalability, performance, and security con-
cerns. The principal concerns with internal applications such as this are fast deliv-
ery time, platform heterogeneity, and the capacity to grow the application and
support multiple types of clients as the enterprise expands. The application may
252 J2EE Integration Technologies

also want single sign-on for users across various security domains, but, at the
same time, permit access only to employees whose proper access privileges
adhere to company-wide rules.
Internal applications may also provide some limited mobile functionality; that
is, they need to be accessible from mobile devices such as PDAs and cellphones.
For example, a travel expense tracking application may want to allow employees
who are on the road to keep track of their expense records. Likewise, enterprises
whose departments are geographically scattered may find it more efficient for
employees to use the Internet for internal communications. Web service interfaces
are particularly useful in these situations, although there are the additional con-
cerns of distributed and multiple security domains.
Often, systems developed for internal use rely on home-grown legacy sys-
tems. While systems bought from third-party vendors may be integrated using
standard connectors supplied by their vendors or third parties, this is not true for
homegrown systems. These homegrown, or one-off, applications must still be
integrated with other applications in the enterprise and accessed from a Web
browser.

6.2 J2EE Integration Technologies

Now that we have examined some common integration scenarios, let’s look at the
technologies that are available on the J2EE 1.4 platform to help with these issues.
The J2EE platform provides a set of EIS integration technologies that address the
EIS integration problem. These include relational database integration technologies
(such as JDBC, Enterprise JavaBeans technology container-managed persistence,
and Java Data Objects), messaging technologies (such as Java Message Service and
message-driven beans), EIS access technologies (particularly the J2EE Connector
architecture), Web services, and XML technologies for manipulating documents.
Let’s briefly examine some of the available integration technologies. The
section “Integration Design Approaches” on page 263 maps these technologies to
different integration problems, illustrating when and how to use them most
effectively.
Chapter 6 Enterprise Application Integration 253

6.2.1 Relational Database Integration Technologies

Relational database management systems (RDBMS) are the most prevalent form of
enterprise data store. The J2EE platform provides three technologies for integrating
data in RDBMS:

• JDBC—Developers can use the JDBC APIs to access relational data in a tab-
ular form.
• Enterprise JavaBeans container-managed persistence (CMP)—Developers use
container-managed persistence to do object-relational (O/R) mapping. By
mapping database table data to Java objects, developers can deal with an object
view of the data rather than a tabular, relational view. CMP also encapsulates
the objects into higher-level managed components with transactional and secu-
rity features.
• Java Data Objects (JDO)—An O/R mapping technology that generates Java
classes as opposed to components. Note that JDO is optional in the J2EE 1.4
platform. Since it is optional, your application server may not support JDO, or
it may support JDO in a nonstandard manner.

“Data Mapping in EAI Applications” on page 274 explains when it is better to

use JDBC, enterprise beans, or JDO.

6.2.1.1 JDBC
The JDBC API defines a standard Java API for integration with relational data-
base systems. A Java application uses the JDBC API for obtaining a database con-
nection, retrieving database records, executing database queries and stored
procedures, and performing other database functions.
Many application component providers use the JDBC API for accessing rela-
tional databases to manage persistent data for their applications.
The JDBC API has two parts: a client API for direct use by developers to
access relational databases and a standard, system-level contract between J2EE
servers and JDBC drivers for supporting connection pooling and transactions.
Developers do not use the contract between J2EE servers and JDBC drivers
directly. Rather, J2EE server vendors use this contract to provide pooling and
transaction services to J2EE components automatically. Note that, according to
the JDBC 3.0 specification, the JDBC system-level contracts can be the same as
the Connector architecture system contracts. Conceptually, JDBC drivers are
254 J2EE Integration Technologies

pluggable resource adapters and may be packaged as J2EE Connector resource

adapters.

6.2.1.2 EJB Container-Managed Persistence

Container-managed persistence (CMP), which is a resource manager–independent
data access API for entity beans, has been expanded and enhanced in the J2EE 1.4
platform. CMP technology enables applications to be easily integrated with various
databases or resource managers, plus it enhances portability.
CMP shields the developer from the details of the underlying data store. The
developer does not need to know how to persist or retrieve data to or from a par-
ticular data store, since the EJB container handles these tasks. Instead, the devel-
oper need only indicate what data or state needs to be stored persistently.
In addition, a developer uses the same API—that is, the EJB CMP methods—
regardless of the underlying type of resource manager. The same entity bean can
thus be used with any type of resource manager or database schema. The technol-
ogy makes it possible to develop enterprise beans that can be customized at
deployment to work with existing data. That is, the same bean implementation can
be deployed to work with many different customer data schemes. The mapping
done at deployment may vary for each customer set up, but the bean itself is the
same. Since the EJB container generates suitable data access code for each situa-
tion, the bean developer does not have to know or care about the underlying
resource manager-specific code. Furthermore, since it has complete control over
managing persistence, the container can optimize database access for better per-
formance.
The J2EE 1.4 platform includes the most up-to-date EJB specification and
CMP technology. Rather than declaring persistent variables in a bean’s implemen-
tation, developers include abstract get and set accessor methods for persistent
variables. Persistent variables are thus treated similarly to JavaBeans properties.
Developers do not provide an implementation for these accessor methods since
they are abstract methods; instead, the EJB container provides the method imple-
mentations.
The CMP architecture also includes container-managed relationships, which
allows multiple entity beans to have relationships among themselves. Container-
managed relationships are handled in much the same way as container-managed
persistence. The bean implementation merely provides get and set accessor
methods for these fields, and the container provides the method implementations.
Similarly, the developer specifies the relationships in the deployment descriptor.
Chapter 6 Enterprise Application Integration 255

6.2.1.3 Java Data Objects

Java Data Objects (JDO) is an API that provides a standard, interface-based Java
model abstraction of persistence. Application developers can use the JDO API to
directly store Java domain model instances into a persistent store, such as a data-
base. You may consider JDO as one alternative to using JDBC or enterprise beans
with container-managed persistence. Keep in mind, however, that JDO is not stan-
dard in the J2EE 1.4 platform.
There are some benefits to using JDO. Since JDO keeps applications indepen-
dent or insulated from the underlying database, application developers can focus
on their domain object model and not have to be concerned with the persistence
details, such as the field-by-field storage of objects. JDO also ensures that the
application uses the optimal data access strategies for best performance.
It is not unusual to compare JDO to enterprise beans with container-managed
persistence, since both provide object-relational mapping capabilities. The princi-
pal difference is that JDO maps database relationships to plain Java objects, while
EJB CMP maps relationships to transactional components managed by a con-
tainer. EJB CMP essentially provides a higher layer of service than JDO. Some
J2EE application servers, such as the J2EE 1.4 platform SDK, internally use JDO
to implement enterprise bean container-managed persistence.

6.2.2 Messaging Technologies

Messaging systems allow unrelated applications to communicate asynchronously
and reliably. Not only do the communicating parties not have to be closely tied to
each other, they can also remain relatively anonymous.
The J2EE platform provides the Java Message Service (JMS) API, which is a
standard Java API defined for enterprise messaging systems. Along with JMS, the
J2EE platform also provides message-driven beans. Message-driven beans are
EJB components that consume and process asynchronous messages delivered via
JMS or some other messaging system.
Let’s take a look at messaging technologies in general, then examine two
J2EE-specific technologies: Java Message Service and message-driven beans.

6.2.2.1 Overview of Messaging Technologies

Prior to the advent of Web services, developers often chose messaging systems
(called MOM for Message Oriented Middleware) to create an integration architec-
ture. With a messaging system, two systems can communicate with each other by
256 J2EE Integration Technologies

sending messages. Such messages, which are delivered asynchronously, typically

consist of two parts: one part—the message body—contains the business data and
the other part—the message header—contains routing information. Since messages
are sent asynchronously, the sender does not have to wait for the message to be
delivered to the receiver.
There are two common messaging styles: point-to-point, and publish and sub-
scribe. A point-to-point messaging style is used when messages are sent to only
one receiver. The recipient receives messages sent to it through a queue specifi-
cally set up for the receiver. A message sender sends messages to this queue, and
the recipient retrieves (and removes) its messages from the queue. Publish and
subscribe, on the other hand, is intended to be used when there can be multiple
recipients of a message. Rather than a queue, this style uses a topic. Messages are
sent—or, more correctly, published—to the topic, and all receivers interested in
these messages subscribe to the topic. Any message published to a topic can be
received by any receiver that has subscribed to the topic. See Figure 6.2.
In addition, a typical MOM system has a message router that is responsible
for ensuring message delivery (according to the agreed-upon quality of service) to
the receiver. The message router uses the message header information to deter-
mine where and how to route the message contents.

message A Client
Topic
<Subscriber>

me
Publish-Subscribe ssa
ge
sh A Messaging System A
bli ge
pu ssa
me Client
<Subscriber>

Client
<Sender>
se
m nd
es
sa
g eB

message B Client
Queue
<Receiver>

Figure 6.2 Messaging System Queues and Topics

Chapter 6 Enterprise Application Integration 257

When used for integration, an enterprise very likely requires that all partici-
pating EIS systems communicate with each other by sending messages via the
messaging system. As a result, enterprises typically standardize on one vendor for
their MOM system, and they use that vendor’s adapters to accommodate their
various EISs. In return, enterprises using messaging systems gain the benefits of
asynchronous messaging calls: Messages are queued and delivered when the
target system is available without constraining the sending system. The asynchro-
nicity of messaging systems means that communicating applications need not be
currently running to receive messages. This protects the communicating applica-
tions from system failures and other types of partial outages, conditions that are
not uncommon in network situations.
Messaging systems also bring a dynamic quality to EIS systems. Components
can be added or removed to the network without affecting other systems. Systems
do not need to have their throughputs perfectly match since they can interact with
the messaging system at their own pace. For example, if one application sends
messages more rapidly than the receiving application can retrieve these messages,
the messaging system keeps these messages in the queue indefinitely. As a result,
good overall throughput is achieved since each part of the system can work at its
optimum capacity.
Furthermore, messaging technology is considered a fairly mature technology.
Most MOM systems provide a number of quality of service features, such as reli-
able once and only once delivery, load balancing, scalability, fault tolerance, and
transactional support.
However, the proprietary nature of MOM systems results in some significant
disadvantages. Since they use proprietary protocols over the network, it is usually
more difficult to mix and match MOM products from different vendors. Although
messaging systems decouple the sender and receiver and permit communicating
parties to run on different hardware and software platforms, they fall short of
achieving true interoperability because they tie developers to their proprietary
APIs to send and receive messages. As a result, application developers must cus-
tomize their applications for different MOM systems.

6.2.2.2 Java Message Service

The Java Message Service (JMS) API, a standard Java API defined for enterprise
messaging systems, can be used across different types of messaging systems. A Java
application uses the JMS API to connect to an enterprise messaging system. Once
connected, the application uses the facilities of the underlying enterprise messaging
258 J2EE Integration Technologies

system (through the API) to create messages and to communicate asynchronously

with one or more peer applications.
For the J2EE 1.4 platform, JMS includes some enhancements. In particular,
the addition of common interfaces enables you to use the JMS API so that it is not
specific to either a point-to-point or publish-subscribe domain. A JMS provider
may also use the J2EE Connector architecture to integrate more closely with an
application server. (The section “EIS Access Technologies” on page 259 discusses
the J2EE Connector architecture.)
In many ways, JMS is to messaging systems what JDBC is to database sys-
tems. Just as JDBC provides a standard interface to many database systems, JMS
provides a standard API for MOM systems. In fact, JMS changed the proprietary
nature of MOM systems by providing a standard Java API that interfaces with any
MOM system. The developer now writes to this standard API rather than to indi-
vidual, proprietary APIs. The J2EE platform further simplified—and made more
portable—the integration of a MOM system with a J2EE application server.
Similar to its support for JDBC, the J2EE platform has added support to JMS
for a connection-oriented operational style: Developers can look up a factory and
a connection in the same way. Like JDBC, JMS supports transactions and can
continue JTA transactions started by either a Web or EJB component.

6.2.2.3 Message-Driven Beans

Message-driven beans, which are EJB components that receive incoming enterprise
messages from a messaging provider, contain the logic for processing these mes-
sages. The business logic of a message-driven bean—which may include initiating a
workflow, performing a computation, or sending a message—may be driven by the
contents of the message itself or merely the receipt of the message.
Message-driven beans are particularly useful in situations where messages
need to be automatically delivered and asynchronous messaging is desired. They
enable applications to be integrated in a loosely coupled, but still reliable, fashion.
Message-driven beans are also useful when the delivery of a message should be
the event initiating a workflow process or when a specific message must trigger a
subsequent action.
When an application produces and sends a message to a particular message
destination queue or topic, the EJB container activates the corresponding
message-driven bean (from the pool of available message-driven beans). The acti-
vated bean instance consumes the message from the message destination. Since
Chapter 6 Enterprise Application Integration 259

they are stateless, any instance of a matching type of message-driven bean can
handle the message.
Implementing a message-driven bean is straightforward. The bean developer
extends the javax.ejb.MessageDrivenBean interface and a message listener inter-
face for the bean—the message listener interface corresponds to the enterprise’s
messaging system. For enterprises using JMS, the developer extends the
javax.jms.MessageListener interface. A developer may embed business logic to
handle particular messages within the MessageListener methods. For example,
when a message-driven bean consumes JMS messages, the developer codes the
message-handling business logic within the onMessage method. When a message
appropriate for this bean arrives at a message destination, the EJB container
invokes the message-driven bean’s onMessage method.
With message-driven beans, you can make the invocation of the bean part of a
transaction. That is, you can ensure that the message delivery from the message
destination to the message-driven bean is part of any subsequent transactional
work initiated by the bean’s logic. If problems occur with the subsequent logic
causing the transaction to roll back, the message delivery also rolls back—and the
message is redelivered to another message-driven bean instance.

6.2.3 EIS Access Technologies

The J2EE 1.4 platform includes the J2EE Connector architecture, a technology
designed specifically for accessing enterprise information systems (EISs). The Con-
nector architecture simplifies integrating diverse EISs into the platform, since
each EIS can use the same, single resource adapter to integrate with all compliant
J2EE servers.
The J2EE Connector architecture provides a standard architecture for integrat-
ing J2EE applications with existing EISs and applications, and particularly for data
integration with non-relational databases. The Connector architecture enables
adapters for external EISs to be plugged into the J2EE application server. Enter-
prise applications can use these adapters to support and manage secure, transac-
tional, and scalable, bi-directional communication with EISs. The EIS vendor
knows that its adapter will work with all J2EE-compliant application servers, and
the compliant J2EE server can connect to multiple EISs. See Figure 6.3.
260 J2EE Integration Technologies

Container-
component
contracts Application
Container
Component

Connection
pool manager Transaction Application
manager contract

Security Message System

service manager pluggability contracts Resource
manager Adapter

Lifecycle
manager Work
manager
EIS-specific
interface

Enterprise
J2EE Application Server Information
System

Figure 6.3 Connector Architecture System and Contracts

The earlier version of the Connector architecture focused on synchronous

integration with EISs, while the current version that is part of J2EE 1.4 extends
this core functionality to support asynchronous integration. That is, it supports
both outbound and inbound message-driven integration that is protocol
independent.
A resource adapter is a J2EE component that implements the Connector API
for a specific EIS. Communication between a J2EE application and an EIS occurs
through this resource adapter. In a sense, think of a resource adapter as analogous
to a JDBC driver, since it provides a standard API for access between the J2EE
server and the external resource.
The J2EE Connector architecture, through its contracts, establishes a set of
rules for EIS integration. The J2EE application server and an EIS resource adapter
collaborate, through a set of system-level contracts, to keep security, transaction,
and connection mechanisms transparent to the application components. Applica-
tion components and resource adapters rely on the application-level contract for
their communication.
Chapter 6 Enterprise Application Integration 261

The application-level contract defines the API used by a client to access a

resource adapter for an EIS. This API may be the Common Client Interface (CCI),
which is a generic API for accessing multiple heterogeneous EISs, or it may be a
resource adapter-specific API.
The initial release of the Connector architecture, which is part of the J2EE 1.3
platform, established three system-level contracts, as follows:

• Connection management contract—Supports connection pooling to an under-

lying EIS, an important requirement for scalable applications.
• Transaction management contract—Supports local and global transactions,
and enables management of (global) transactions across multiple EISs.
• Security management contract—Enables secure interchanges between an EIS
and a J2EE application server and protects EIS-managed resources.

The most recent release (version 1.5) of the Connector architecture expanded
the capabilities of resource adapters. This release expanded the transaction
support for a resource adapter. Previously, a transaction had to start from an enter-
prise bean on the J2EE application server, and it remained in force during opera-
tions on the EIS. Now, transactions can start on the EIS and be imported to the
application server. The J2EE Connector architecture specifies how to propagate
the transaction context from the EIS to the application server. The Connector
architecture also specifies the container’s role in completing a transaction and
how it should handle data recovery after a crash.
The current version also specifies additional system-level contracts to the
initial three contracts just noted. These new contracts are:

• Messaging “pluggability” contract—Extends the capabilities of message-

driven beans so that they can handle messages from any message provider rath-
er than being limited to handling only JMS messages. By following the APIs
for message handling and delivery specified in this contract, an EIS or message
provider can plug its own custom or proprietary message provider into a J2EE
container. JAXM is a good example of this type of message provider.
• Work management contract—Enables the J2EE application server to manage
threads for resource adapter, ensuring that resource adapters use threads prop-
erly. When a resource adapter improperly uses threads, such as when it creates
too many threads or fails to release threads, it can cause problems for the entire
application server environment. Poor thread handling by a resource adapter can
262 J2EE Integration Technologies

inhibit server shutdown and impact performance. To alleviate this problem, the
work management contract enables the application server to pool and reuse
threads, similar to pooling and reusing connections.
In addition, the work management contract gives resource adapters more flex-
ibility for using threads. The resource adapter can specify the execution con-
text for a thread. The contract allows a requesting thread to block—stop its own
execution—until a work thread completes. Or, a requesting thread can block
while it waits to get a work thread; when the application server provides a work
thread, both the requesting and work threads execute in parallel. Yet another
option, a resource adapter can submit the work for the thread to a queue and
have it execute at some later point; the adapter continues its own execution
without waiting further. Thus, a resource adapter and a thread may execute in
conjunction with each other or independently, using a listener mechanism, if
need be, to notify the resource adapter that the work thread has completed.
• Lifecycle management contract—Enables an application server to manage the
lifecycle of a resource adapter. With this contract, the application server has a
mechanism to bootstrap a resource adapter instance at deployment or at server
startup, and to notify the adapter instance when it is undeployed or when the
server is shutting down.

6.2.4 Web Service and XML Technologies

Although messaging systems provide many of the same EAI advantages as Web ser-
vices, Web services go a step further. Principally, Web services support multiple
vendors and the ability to go through firewalls using Internet standards. Web ser-
vices also support a flexible XML format.
The J2EE 1.4 platform provides a rich set of Web service APIs and XML doc-
ument-manipulating technologies. The Web service APIs—JAX-RPC, JAXR,
SAAJ, JAXB, and JAXP—provide standard Java APIs for integrating applica-
tions and other systems. A Java application can use these APIs to obtain and use a
Web service. These APIs are particularly useful when an application or system
exposes a Web service layer explicitly for integration purposes.
There are also a number of XML-related APIs and facilities in the Java lan-
guage (and the J2EE 1.4 platform) that can be applied to integration problems.
They permit you to define an interface data model, manipulate disparate docu-
ments, and perform transformations between data types. These XML APIs give
you the ability to structure a data model for passing data among different systems.
Chapter 2 discusses these technologies in greater depth.
Chapter 6 Enterprise Application Integration 263

6.3 Integration Design Approaches

The J2EE 1.4 platform provides several technologies for integration, and you can
combine them to fit your integration requirements. These technologies, together
with the other platform capabilities, give you a rich platform for designing an inte-
gration solution. IT architects face the challenge of combining the technologies in
the most effective and flexible manner possible to create an integration architec-
ture that can adapt to changing business needs and strategies.
Often, enterprises attempt integration as a “one-off” solution—that is, they
treat each integration problem as a unique instance with its own special solution.
With this type of approach, an enterprise integrates each system in an individual,
custom manner—not the most efficient or effective integration approach. It is far
better to have an integration architecture that grows with the needs of the enter-
prise. The J2EE platform, with its extensive support for Web services, makes such
an architecture possible. A J2EE application server acts as the universal connector
between the EIS systems and new EAI applications. (This chapter uses the term
EAI applications to refer to applications specially designed to solve the problems
of integrating enterprise applications.) In a sense, the J2EE application server is
the integration hub. The data model in the application server becomes the canoni-
cal agreed-upon data model to which all others—EISs and new EAI applica-
tions—adapt. Figure 6.4 shows how this might look from a high level.
A good integration architecture consists of a set of “integration layers” each
of which provide certain quality of services. By integration layer we mean the
interface or endpoint at which distinct systems intersect. Enterprise architects and
software developers must decide how to implement these various integration
layers in the most suitable manner. Generally, the requirements of each situation
drive the choice of integration layer. For example, the adventure builder enterprise
uses Web services as the integration layer for its supply chain. As a result, all
interactions with its suppliers must happen through the Web service interfaces.

❐ The integration layer defines the interface between the EIS systems and new
applications, thus exposing an EIS system to an application.
❐ An integration layer needs to be the stable point in your system—the fixed
standard against which new applications are programmed. While both the new
application and the existing EIS may change, the integration layer should re-
main the same. Otherwise, changes in one place will require changes to numer-
ous dependent systems.
264 Integration Design Approaches

Application 1 EIS A

Application 2 J2EE
Application
Server Relational
database
Application 3
EIS B

Figure 6.4 J2EE Application Server as Integration Hub

Integration approaches vary, and often the situation dictates the best approach.
In some situations, the integration approach is fairly obvious. For example, when
a client requires a tightly coupled interaction with a relational database, clearly the
JDBC API is the design strategy to use. Similarly, another design strategy uses the
J2EE Connector architecture for data integration with non-relational databases.
These two design strategies are particularly appropriate when the client requires
an API for connecting, querying, and updating a database with transactional
semantics. Other situations require you to combine technologies to meet the inte-
gration requirements. You might use connectors on top of a non-relational data-
base, but then, based on the client requirements, you might add a Web service
layer on top of the connector.
It is helpful to view the process of formulating an integration approach as
three separate tasks (each requiring a design and implementation phase):

• Decide on the integration layer, including where it should be located and what
form it should take. With the J2EE platform, various integration layers are pos-
sible, including connectors, enterprise beans, JMS, and Web services. Remem-
ber that you want the integration layer to be such that other applications or
systems can evolve easily.
• Decide how to adapt each EIS to the integration layer.
• Decide how to write new applications against these integration layers.
Chapter 6 Enterprise Application Integration 265

Keep in mind that an important reason for using an integration layer is the ser-
vices it provides for distribution. Usually, EISs are located on different physical
machines, or they may be run as separate processes. To integrate these EISs, you
need to use a distributed technology. J2EE application servers are an excellent
choice for integration because they can handle multiple protocols for distributed
computing: Internet Inter-ORB Protocol (IIOP), messaging, HTTP and Web ser-
vices. In addition, J2EE application servers can handle native EIS protocols
through connectors. These native protocols include proprietary protocols used by
ERP and database vendors, among others.
Figure 6.5 shows the integration design approaches that are possible with the
J2EE technologies.
Note: This figure uses dashed lines to represent asynchronous calls, such as
sending a JMS message. The solid lines show synchronous calls, which are made
either through EJB or Web service APIs. The three applications, which are shown
on the left side of the diagram, are new EAI applications that you might be writ-
ing. The right side of the diagram shows some of the existing EIS systems that
need to be integrated with the EAI applications.
.

Connector as integration layer

Application 1
EIS A
MDB

Connector

EJB/JMS as
EJB

Application 2 J2EE
integration layer
Application
Service

Server
Web

JDBC/CMP

Relational
database
MDB

Application 3 Web services

as integration
layer

EIS B

Figure 6.5 EIS Integration Design Approaches

266 Integration Design Approaches

Since it requires very tight coupling with EIS A, EAI application 1 bypasses
the J2EE application server but reuses the connector layer. Application 2 needs to
use some business logic present in the application server. As a result, it does not
want to access the EIS directly. Application 2 instead uses a combination of syn-
chronous and asynchronous calls to the EJB tier. These calls (to enterprise beans
and message-driven beans) access EIS A and the relational database through
JDBC, EJB container-managed persistence, or connectors. EIS B also generates
asynchronous messages for implementing the business logic, and these messages
must be delivered to a message-driven bean. Application 3 uses the least tightly
coupled interface, Web services, and it accesses available functionality through a
Web service interface. The Web service internally uses the business logic stored in
the J2EE application server. (In the next sections we take a closer look at these
different EAI design approaches and then provide some guidelines for implement-
ing these approaches.)
Let’s see how you might apply this strategy to the adventure builder enter-
prise. The adventure builder enterprise decides to use several layers for
integration:

1. Web services as an integration layer for its supply chain.

2. Web services as an integration layer for communicating among different de-
partments. For example, the adventure builder Web site uses a Web service to
send an order to the order processing center.
3. EJB/JMS components as the integration layer with EISs. The order processing
center integrates EISs within its department using JMS. Hence, the order pro-
cessing center fulfills an order using JMS and EJB technologies for integrating
its various EIS systems, customer relations management, billing systems, and
so forth.

6.3.1 Web Services Approach

One approach for EAI is to use a Web service as the integration layer. With this
approach, an enterprise’s EIS systems expose their functionality by implementing
Web services. They make their Web service interfaces available to other applica-
tions by providing WSDL descriptions of them. In addition, the integration layer
may also include XML schemas for the documents used as parameters and return
values. Essentially, the WSDL description of the service interface and document
schemas becomes the integration layer, or the point of stability.
Chapter 6 Enterprise Application Integration 267

Developers who use the Web service approach for integration can leverage the
advantages of Web services. Developers can write new enterprise applications
with any technology that supports Web services, and the applications may be run
on a variety of hardware and software platforms. However, this approach falls
short when the new applications have certain additional requirements, such as
transactional semantics or stringent security requirements. We discuss how to
handle these issues in subsequent sections.
The adventure builder application is a good example of the Web services
approach. As noted, the adventure builder enterprise uses Web services for inte-
grating its supply chain. (Chapter 8 discuss the exact structure of the application,
but here we highlight those details that pertain to integration.) The adventure
builder architects decide, in consultation with the suppliers, on the schemas for the
documents that they intend to exchange. (See “Designing Domain-Specific XML
Schemas” on page 131.) Since their business depends on this exchange of docu-
ments—the adventure builder application sends purchase orders to various suppli-
ers who fulfill adventure requests, and, in turn, the suppliers invoice adventure
builder—the adventure builder enterprise and the suppliers need to agree on
schemas that describe the content of these documents. For example, a lodging
supplier might use an invoice document schema. Code Example 6.1 shows an
example document corresponding to that schema. Similar schemas are defined for
other kinds of suppliers, such as activity and airline suppliers.

<?xml version="1.0" encoding="UTF-8"?>

<Invoice xmlns="http://java.sun.com/blueprints/ns/invoice"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://java.sun.com/blueprints/ns/invoice
http://java.sun.com/blueprints/schemas/invoice-lodging.xsd">
<ID>1234</ID>
<OPCPoId>AB-j2ee-1069278832687</OPCPoId>
<SupplierId>LODGING_INVOICE</SupplierId>
<status>COMPLETED</status>
<HotelId>LODG-6</HotelId>
<HotelAddress>1234 Main Street, Sometown 12345, USA
</HotelAddress>
<CancelPolicy>No Cancelations 24 hours prior</CancelPolicy>
</Invoice>

Code Example 6.1 Lodging Supplier Invoice

268 Integration Design Approaches

Similarly, the architects standardize on the WSDL to use for invoices when
fulfilling an order. Suppliers, such as airlines, hotels, and activity providers, use
these schemas and WSDL descriptions when interacting with the adventure
builder enterprise. Code Example 6.2 shows the WSDL for the Web service end-
point to which the suppliers send their invoices.
The WSDL shown in Code Example 6.2 provides a single operation,
submitDocument, which has a single xsd:anyType parameter representing the
invoice XML document and which returns a single string value indicating a con-
firmation. The use of xsd:anyType enables an endpoint to receive multiple types
of invoice documents with one method. See “Exchanging XML Documents” on
page 107. A WSDL document normally begins with the type definitions that it
uses, such as the exception type InvalidDocumentException. (For brevity, the
code sample omits these type definitions.) The WSDL code then describes the
SOAP messages used in the SOAP request and reply calls that correspond to the
submitDocument method. It also describes a WSDL port corresponding to the
JAX-RPC interface for receiving invoices, plus it binds the port to the HTTP pro-
tocol and indicates the RPC-literal binding. Finally, the WSDL code defines the
name of the service and the URL at which the service is available. (For a more
detailed explanation of a WSDL document, see “Web Services Description Lan-
guage” on page 36.)

<?xml version="1.0" encoding="UTF-8"?>

<definitions name="WebServiceBroker" targetNamespace= ...>
...
<message name="BrokerServiceIntf_submitDocument">
<part name="Invoice" type="xsd:anyType"/></message>
<message name="BrokerServiceIntf_submitDocumentResponse">
<part name="result" type="xsd:string"/></message>
<message name="InvalidDocumentException"><part name=
"InvalidDocumentException" element=
"tns:InvalidDocumentException"/>
</message>
<portType name="BrokerServiceIntf">
<operation name="submitDocument" parameterOrder="Invoice">
<input message="tns:BrokerServiceIntf_submitDocument"/>
<output message
="tns:BrokerServiceIntf_submitDocumentResponse"/>
<fault name="InvalidDocumentException" message=
"tns:InvalidDocumentException"/>
Chapter 6 Enterprise Application Integration 269

</operation>
</portType>
<binding name="BrokerServiceIntfBinding" type=
"tns:BrokerServiceIntf">
...
</binding>
<service name="WebServiceBroker">
...
</service>
</definitions>

Code Example 6.2 WSDL for Receiving the Invoices

Although you can modify the EIS systems that participate in Web service
interactions to export the Web service interfaces, a better approach uses a J2EE
connector to plug the EIS system into the J2EE application server. The connector
by its nature does not intrude on the EIS. This latter strategy takes advantage of
the capabilities of the J2EE server. Among other advantages, using connectors
allows the transactional and security contexts of a J2EE application server to carry
forward to the EIS application. Plus, the EIS can use the thread pools of the appli-
cation server. In conjunction with connectors, you use JAX-RPC from the J2EE
application server to expose a Web service interface. See Figure 6.6.

❐ To expose a Web service interface for an EIS, use JAX-RPC in front of a con-
nector.

J2EE
Application
Server

Web or EJB
JAX-RPC
endpoint

Connector

Client EIS
component
(business logic)

Figure 6.6 Using JAX-RPC and Connectors

270 Integration Design Approaches

For example, the adventure builder enterprise wants to use Web services to
integrate its existing CRM system, which provides services to manage customer
relations, to process orders. The department that owns the CRM module not only
wants to maintain control of the software, it wants to use a generic interface that
can handle user requests from multiple sources (Web site, telephone, OEM chan-
nels, and so forth). Web services are the best way to create such a generic inter-
face, for these reasons:

• Web services establish clear, defined boundaries. Since Web services can pro-
vide an interface with clear, defined boundaries of responsibilities, the CRM
department has the responsibility to only maintain the endpoint and publish a
WSDL describing the endpoint.
• Web services provide controlled access. Outside requests to the CRM must
come in through the service interface, and the CRM department can apply its
access control parameters to limit access, plus it can log each access.
• Web services support multiple platforms. Because it does not control the hard-
ware and software platforms other departments use, the CRM department can
accommodate any platform by using Web services.
• The current generation of Web services is best suited for applications with a
limited need for transactions and security. The main purpose of the CRM sys-
tem is to allow status queries on existing orders. As such, it has little need for
transactions. It also has limited need for security, since all access to the module
happens within the corporate firewall.

Developers have a choice of endpoint types for implementing the Web service
representing the EIS: either an EJB service endpoint or a Web-based JAX-RPC
service endpoint. Solely from an integration point of view, either type of endpoint
works well since Web and EJB components can both directly use JDBC and J2EE
connectors. For example, the adventure builder enterprise receives invoices using
an EJB service endpoint, as well as JAX-RPC service endpoints for other func-
tions.
Returning to the CRM problem, the adventure builder enterprise could use a
small J2EE application that uses a J2EE connector to connect to the CRM system.
Such an application needs to have a JAX-RPC endpoint to expose the required
Web service functionality.
Today, since many EIS vendors are providing built-in Web services support,
developers can use connectors and leverage this support directly to avoid writing a
Chapter 6 Enterprise Application Integration 271

JAX-RPC wrapper. If you choose to follow this path, ensure that the EIS vendor
provides a WS-I Basic Profile-compatible Web service. If the vendor does not,
consider writing an adapter layer yourself to handle any differences. This
approach may require upgrading your EIS system to a new software version, a
risk you may not want to take. In such cases, consider using the recommended
connectors and JAX-RPC approach.

❐ Consider using the built-in Web services support provided by the EIS vendors
to avoid writing additional interfaces.

6.3.2 Enterprise Beans and JMS Approach

You can use enterprise beans and JMS layers, separately or combined, to develop an
integration layer. With this approach, you use a J2EE application server to provide
an enterprise bean layer for synchronous interactions. For asynchronous interac-
tions, you place a message bus in the enterprise and require that applications use it.
The adventure builder enterprise uses this strategy for integrating applications
within one department. For example, a single department owns the order process-
ing module. Within that department, different groups handle various aspects of
order processing, such as credit card payments, supply chain interactions, cus-
tomer relations, and so on. In this workflow arrangement, interactions among
these departmental groups are handled in a loosely coupled, asynchronous manner
using JMS. When it needs to provide synchronous access, a group may use a
remote enterprise bean interface. (See Figure 8.3 on page 345.)
Using enterprise beans and JMS does not preclude the use of XML, which is
an additional benefit. For example, the adventure builder enterprise uses the same
invoice document listed in Code Example 6.1 when sending a JMS message
within its order processing center. The message-driven bean that receives the JMS
message applies XML validation and translation logic just like any Web service
endpoint.

6.3.3 The Connector Approach

When using connectors, the new EAI application is programmed directly against an
interface provided by the EIS. For new applications that are also J2EE applications,
you use a connector to access the EIS. You can either buy an off-the-shelf connector
or write your own.
272 Integration Design Approaches

If you want to provide a simple isolation layer, you should consider writing a
data access object, or DAO class, to hide the internal details of the EIS. A data
access object encapsulates access to persistent data, such as that stored in a data
management system. It decouples the user of the object from the programming
mechanism for accessing the underlying data. It can also expose an easier-to-use
API. Typically, you develop a data access class for a specific business function or
set of functions, then client applications use these classes. Another common use is
to layer data access objects on top of connectors, or resource adapters, thus isolat-
ing applications from the details of the connector interactions. Often, each
resource adapter interaction has a corresponding data access object, whose
methods pass input records to the adapter as parameters and return output records
to the client application as return values.
Using a connector reduces the complexity of the integration layer. Note that
when using the connector approach, the EAI application is tightly coupled with
the EIS since the application directly uses the data model and functionality of the
EIS. Since there is minimal layering, it also increases performance. However, it is
an approach that works best when the integration problem is small in scope. Since
it does not put into place a true integration architecture, this approach may limit
the ability to scale the integration layer as the enterprise grows. Given these
advantages and disadvantages, consider using this approach as a basic building
block for other strategies.

6.3.4 Combining Approaches into an Integration Architecture

In many cases, architects combine these various integration layers into a single inte-
gration architecture. The end result—the mix of integration layer types—is in large
measure driven by the requirements of each enterprise’s situation. Architects need to
consider the realities of the current state of the technologies and weigh that against
future promises.
For example, Web services, as they exist today, have some shortcomings:
They do not deliver the heavy-duty process integration, data transformation, and
security capabilities required by many EAI scenarios. Similarly, services for trans-
actional integrity and reliable messaging are not yet in place. Since security and
transactional context propagation are critical business requirements, these are
important factors to consider when using a Web services approach.
Data binding is another issue to consider. Web services are a good solution
when data binding requirements are straightforward, such as mapping simple data
types to Java data types. However, when it is necessary to manipulate complex
Chapter 6 Enterprise Application Integration 273

relational or binary from an EIS, you may want to consider other solutions, such
as using the J2EE Connector architecture, which provides a metadata facility to
dynamically discover the data format in the EIS.
Table 6.1 compares these approaches for different integration problems and
indicates the approaches best suited for these integration problems. The adventure
builder enterprise uses Web services for partner and inter-department interactions,
and the enterprise beans with JMS (EJB/JMS) approach for intra-department
interactions.

Table 6.1 Comparing Different Integration Approaches

Operation Connector EJB/JMS Approach Web Services

Approach Approach

Coupling Tight coupling. Can add a layer of No hardware/software plat-

with EIS Uses EIS data abstraction in the form coupling.
model directly. EJB/JMS layer. Can add multiple layers of
abstractions and transla-
tions.

Transac- Available Declarative, automatic Global transaction propa-

tional context propagation gation is not currently
support available. The endpoint
implementation can use
transactions for the busi-
ness logic.

Supporting J2EE 1.4 platform Message-driven beans Currently no asynchro-

asynchro- adds asynchronous provide an easy-to-use nous support. WSDL
nous capabilities to con- abstraction for receiv- provides a primitive mech-
operations nectors. ing asynchronous events anism for one-way calls,
from EISs. although the quality of
service is low.

Performance Highest Overheads because of Significant overheads

remote calls, and because of remote calls,
requirements of running requirements of running a
a server. server, and of XML pro-
cessing and validation.

Heteroge- Requires that the Requires a J2EE appli- Supported on a variety of

neous client is cation server (available hardware/software plat-
platform programmed in on a broad range of forms.
support Java. hardware/software plat-
forms).
274 Data Integration Guidelines

Table 6.1 Comparing Different Integration Approaches (continued)

Operation Connector EJB/JMS Approach Web Services

Approach Approach

Security Can directly inte- Provides application Limited. HTTPS is sup-

features grate with the EISs server security mecha- ported.
security model. nisms.

6.4 Data Integration Guidelines

Recall that enterprise data may be kept in various types of data stores, including
relational databases, non-relational databases, directory servers, and so forth. Some
common strategies that help to integrate data from these different types of data
stores are data mapping, data transformation, and data filtering.

6.4.1 Data Mapping in EAI Applications

Enterprises use various types of data stores to hold data, and each such data store
persists the data according to its own layout and using its own data types. This is
true even within a single enterprise. Thus, there are many varied external representa-
tions of data. Often, external data representations are relational in nature because
most commonly data is stored in a relational database. EAI applications need to
know how to access this data. It is important to map external data to the data struc-
tures used within an EAI application.
An important issue to consider when integrating data sources is to decide on a
mapping layer. Generally, you have these options for relational data:

❐ Create a formal object-oriented data model.

❐ Create a generic data-holder layer.

The formal object-oriented data model relies on object-relational mapping

technologies to map data from relational data sources to an object-oriented
format. You may use such mapping technologies as Enterprise JavaBeans con-
tainer-managed persistence, Java Data Objects, or even the data access object
strategy. There are a number of advantages to this option. For one, you can reap
the traditional advantages of an object-oriented approach, notably reusability,
since you establish a mapping layer that can be reused by other applications.
Chapter 6 Enterprise Application Integration 275

When you use the EJB container-managed persistence technology, you also can
rely on the EJB container’s security features to control access to the data. You also
leverage the performance benefits of the EJB container-managed persistence
engine, which uses data caching to improve performance. Finally, by using these
technologies, you can take advantage of the mapping tools that come with them.
The other option for representing relational data is to create a generic layer to
hold the data. For this approach, you use JDBC APIs to handle data from rela-
tional data sources. The JDBC RowSet technology, in particular, makes it easy to
access relational data in an efficient manner. (Keep in mind, however, that JDBC
RowSet technology is not yet standard in the J2EE platform.) The RowSet technol-
ogy, through the WebRowSet feature, gives you an XML view of the data source. Its
CachedRowSet capabilities lets you access data in a disconnected fashion, while
the FilteredRowSet functions give you the ability to manage data in a discon-
nected manner, without requiring heavyweight DBMS support. By using a generic
layer to hold data, you have a simpler design, since there is no real layering, and
you avoid the conceptual weight of a formal data model. This approach may have
better performance, particularly when access is tabular in nature (such as when
selecting several attributes from all rows that match a particular condition).

❐ To access data stored in non-relational data sources, it is best to use connectors.

Connectors let you plug in non-relational data sources to the J2EE environ-
ment. For non-relational systems that do not have ready-made connectors avail-
able, such as home-grown systems, you need to write your own connector. See
“Integrating Custom Legacy Systems” on page 283.

6.4.2 Data Transformation

Data transformation—the ability to convert data to an acceptable application
format—is a common requirement for newer applications that need to access legacy
data. Such transformation functionality is necessary because most legacy systems
were not designed to handle the requirements of these subsequent applications.
To illustrate, a legacy system may store dates using an eight-digit integer
format; for example, storing the date September 23, 2003 as 20030923. Another
application that accesses this legacy system needs the same date formatted as
either MM/DD/YYYY or 09/23/2003. The application needs to access the eight-digit
date from the legacy system and transform it to a usable format.
276 Data Integration Guidelines

Another example of data transformation might involve customer data. Cus-

tomer data spans a range of information, and might include identity and address
information as well as credit and past ordering information. Different systems
may be interested in different parts of this customer data, and hence each system
may have a different notion of a customer.
Even schemas, including industry-standard schemas such as Electronic Data
Interchange For Administration, Commerce, and Transport (EDIFact), Universal
Business Language (UBL), and RosettaNet, must be transformed to each other.
Often enterprises need to use these industry-standard formats for external commu-
nications while at the same time using proprietary formats for internal communi-
cations.
One way you might solve the data transformation problem is to require that all
systems use the same standard data format. Unfortunately, this solution is unreal-
istic and impractical, as illustrated by the Y2K problem of converting the repre-
sentation of a calendar year from two digits to four digits. Although going from
two to four digits should be a minor change, the cost to fix this problem was enor-
mous. System architects must live with the reality that data transformations are
here to stay, since different systems will inevitably have different representations
of the same information.

❐ A good strategy for data transformation is to use the canonical data model.

An enterprise might set up one canonical data model for the entire enterprise
or separate models for different aspects of its business. Essentially, a canonical
data model is a data model independent of any application. For example, a canon-
ical model might use a standard format for dates, such as MM/DD/YYYY. Rather than
transforming data from one application’s format directly to another application’s
format, you transform the data from the various communicating applications to
this common canonical model. You write new applications to use this common
format and adapt legacy systems to the same format.

❐ Use XML to represent a canonical data model.

XML provides a good means to represent this canonical data model, for a
number of reasons:

• XML, through a schema language, can rigorously represent types. By using

XML to represent your canonical model, you can write various schemas that
Chapter 6 Enterprise Application Integration 277

unambiguously define the data model. Before XML, many times canonical
data models were established but never documented by their developers. If you
were lucky, developers might have described the model in a text document. It
was easy for such a document to get out of sync with the actual types used by
the model.
• XML schemas are enforceable. You can validate an XML document to ensure
that it conforms to the schema of the canonical data model. A text document
cannot enforce use of the proper types.
• XML is easier to convert to an alternate form, by using declarative means such
as style sheets.
• XML is both platform and programming-language neutral. Hence, you can
have a variety of systems use the same canonical data model without requiring
a specific programming language or platform.

This is not meant to imply that XML is perfect, since there are some disadvan-
tages to using XML to represent a canonical model. For one, to use XML you
need to either learn the XML schema language (XSL) or use a good tool. Often,
transforming XML requires you to use XSL, which is not an easy language to
learn, especially since XSL is different from traditional programming languages.
There are also performance overheads with XML. However, for many enterprise
settings (and especially for integration purposes) the benefits of using XML out-
weigh its disadvantages, and the benefits of XML become more significant when
you factor in future maintenance costs.
Code Example 6.3 shows an XML document representing invoice informa-
tion. This might be the canonical model of an invoice used by the adventure
builder enterprise. Since this document is published internally, all new applica-
tions requiring this data type can make use of it.

<?xml version="1.0" encoding="UTF-8"?>

<bpi:Invoice
xmlns:bpi="http://java.sun.com/blueprints/ns/invoice"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://java.sun.com/blueprints/ns/
invoice http://java.sun.com/blueprints/schemas/
invoice.xsd">
<bpi: InvoiceId>1234</bpi:InvoiceId>
<bpi:OPCPoId>AB-j2ee-1069278832687</bpi: OPCPoId>
278 Data Integration Guidelines

<bpi:SupplierId>LODGING_INVOICE</bpi:SupplierId>
<bpi: status>COMPLETED</bpi:status>
</bpi:Invoice>

Code Example 6.3 XML Document With Invoice Information

It is usually a good idea to provide the schema for the canonical data model.
By having the schema available, you can validate the translated documents
against it and newer applications can use the schema to define their own models.
Code Example 6.4 shows the XSD schema file for this invoice information.

<?xml version="1.0" encoding="UTF-8"?>

<xsd:schema
xmlns:xsd="http://www.w3.org/2001/XMLSchema"
targetNamespace="http://java.sun.com/blueprints/ns/invoice"
xmlns="http://java.sun.com/blueprints/ns/invoice"
elementFormDefault="qualified">
<xsd:element name="Invoice" type="InvoiceType"/>
<xsd:complexType name="InvoiceType">
<xsd:sequence>
<xsd:element name="InvoiceId" type="xsd:string"/>
<xsd:element name="OPCPoId" type="xsd:string"/>
<xsd:element name="SupplierId" type="xsd:string"/>
<xsd:element name="status" type="xsd:string"/>
</xsd:sequence>
</xsd:complexType>
</xsd:schema>

Code Example 6.4 XSD Schema for Invoice Information

In addition to the XML form, the canonical model representation may be

needed in the Java Object form. Often, the Java Object form is needed, for exam-
ple, when a substantial amount of business logic is written in the J2EE application
server. To use the canonical form for new code, you convert the canonical data
types (from an XML schema to Java or vice versa) in Java objects. That is, you
use the XML documents or their Java object representations as the stable integra-
tion point. (See “Web Services Approach” on page 266.) For example, Code
Chapter 6 Enterprise Application Integration 279

Example 6.5 shows the equivalent Java classes for the canonical model of invoice
information.

public class Invoice {

public string getInvoiceId();
public Address getOPCPoId();
public string getSupplierId();
public string getStatus();
// class implementation ...
}

Code Example 6.5 Java Class Equivalents for Canonical Invoice Information

Once the XML is defined, you can also use tools such as JAXB to generate the Java
classes. (See “Flexible Mapping” on page 148.)
After establishing a canonical data model, you must devise a strategy to
convert any alternate data representations to this model. Because it plugs its enter-
prise systems—billing, order processing, and CRM—in via an application server,
the adventure builder enterprise exposes the canonical data model only through
the external interfaces exposed by the application server. That is, only those com-
ponents with external interfaces—Web service applications, remote enterprise
beans, and so forth—expose the canonical data model. Since the external world
needs (and sees) only the canonical data model, the adventure builder enterprise
must transform its internal data representations—which have their own data
model devised by its various EISs—to this same canonical model.
The data translation between the internal and external representations can be
done before the data goes from the EIS into the application server—that is, the
application server internally uses the canonical data model, which is generally
recommended. Or, the data translation can take place just prior to sending the data
out to the external world—that is, the application server uses the various EISs’
native data representations. Sometimes the business logic necessitates this latter
approach because the logic needs to know the precise native format. The transla-
tion is accomplished in one of two ways:

1. Use XSL style sheets to transform these alternate data representations, either
when data comes in or when data goes out. XSL style sheets work for XML-
based interfaces. In this approach, the application server internally uses the EIS
280 Data Integration Guidelines

native formats and the translation happens just before the data either goes out
to the external world or comes in from the outside.
2. Use a façade approach and do programmatic object mapping in the DAO layer.
That is, you set up a DAO to connect an EIS to the application server. Write
the DAO so that it exposes only the canonical data model and have it map any
incoming data to the appropriate internal data model. Since this approach con-
verts incoming data to the canonical form when the data arrives at the applica-
tion server, the business logic internally uses the canonical data representation.

To understand the XSL style sheet approach, let’s consider how the adventure
builder enterprise receives invoices from various suppliers. In adventure builder’s
case, various suppliers submit invoices, and each supplier may have a different
representation (that is, a different format) of the invoice information. Furthermore,
adventure builder’s various EISs may each have a different representation of the
same invoice information. Code Example 6.6 shows an example of a typical sup-
plier’s invoice.

<?xml version="1.0" encoding="UTF-8"?>

<Invoice xmlns="http://java.sun.com/blueprints/ns/invoice"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation=
"http://java.sun.com/blueprints/ns/invoice
http://java.sun.com/blueprints/schemas/
invoice-lodging.xsd">
<ID>1234</ID>
<OPCPoId>AB-j2ee-1069278832687</OPCPoId>
<SupplierId>LODGING_INVOICE</SupplierId>
<status>COMPLETED</status>
<HotelId>LODG-6</HotelId>
<HotelAddress>1234 Main Street, Sometown 12345, USA
</HotelAddress>
<CancelPolicy>No Cancelations 24 hours prior</CancelPolicy>
</Invoice>

Code Example 6.6 Example of Supplier Invoice Information

Chapter 6 Enterprise Application Integration 281

Compare this listing of invoice information with that of adventure builder’s

canonical model, shown in Code Example 6.3. adventure builder can convert an
invoice from this supplier to its canonical data model by applying in the interac-
tion layer of the Web service the style sheet shown in Code Example 6.7. The
style sheet defines rules, such as those for matching templates (<xsl:template
match-...>), that are applied to the supplier invoice and corresponding XML
fragment when the canonical model is generated.

<xsl:stylesheet version='1.0'
xmlns:xsl='http://www.w3.org/1999/XSL/Transform'
xmlns:bpi='http://java.sun.com/blueprints/ns/invoice'>
<xsl:template match="text()"/>
<xsl:template match="@*"/>
<xsl:template match="bpi:Invoice">
<bpi:Invoice xmlns:xsi=
"http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation=
"http://java.sun.com/blueprints/ns/invoice
http://java.sun.com/blueprints/schemas/invoice.xsd">
<xsl:apply-templates/>
</bpi:Invoice>
</xsl:template>
<xsl:template match="bpi:InvoiceRef">
<bpi:InvoiceId><xsl:value-of select="text()"/>
</bpi:InvoiceId>
</xsl:template>
<xsl:template match="bpi:OPCPoId">
<bpi:OPCPoId><xsl:value-of select="text()"/></bpi:OPCPoId>
</xsl:template>
<xsl:template match="bpi:SupplierId">
<bpi:SupplierId><xsl:value-of select="text()"/>
</bpi:SupplierId>
</xsl:template>
<xsl:template match="bpi:Status">
<bpi:status><xsl:value-of select="text()"/></bpi:status>
</xsl:template>
</xsl:stylesheet>

Code Example 6.7 Style Sheet to Convert Supplier Invoice to Canonical Model
282 Data Integration Guidelines

The adventure builder application applies the style sheet when an invoice is
sent to a supplier. See “Reuse and Pool Parsers and Style Sheets” on page 188 for
more information about pooling style sheets.
Transition façades, which can apply to any object representation of data, are a
more general solution for transformations. You can use transition façades to hide
extra information or to provide simple mappings. To use a façade, you write a
Java class within which you do manual data transformation and mappings.
You should also consider using one of the many tools available for data trans-
formations. These tools simplify the data transformation task and make it easier to
maintain.

❐ When data is in an XML document, it is easier to write XSL style sheets that
do the required transformations.
❐ When you access data using EJB container-managed persistence, you can ei-
ther directly modify the container-managed persistent classes or write façades
to do the required transformations.

6.4.3 Data Filtering

When you do not have access to an application’s code, such as for off-the-shelf
packaged applications or for applications that cannot be modified because they are
critical to a working business system, you might consider using filtering. That is,
you construct a filter that sits in front of the application and does all necessary
data translations.
Generally, data filtering goes hand-in-hand with data transformations. The
canonical data model, because it must support all use cases within an enterprise, is
often a good candidate for filtering. Since many applications do not need access to
all data fields, you can filter data and simplify application development and
improve performance by reducing the amount of data that is exchanged.
There are two types of filtering, and each has its own use cases:

1. Filtering that hides information but saves it for later use

2. Filtering that outputs only needed information

For example, in the adventure builder enterprise, a workflow manager

receives the invoices sent by the different suppliers. Since it needs only an identi-
fier to identify the workflow associated with the invoice, the workflow manager
Chapter 6 Enterprise Application Integration 283

can filter the document to retrieve only this information. However, since it may
need to pass the entire invoice—all the data fields in the invoice—to the next step
in the workflow, the workflow manager must preserve the entire document. The
workflow manager can accomplish this using flexible mapping. (See “Flexible
Mapping” on page 148.)
On the other hand, you may want to apply data filtering before sending infor-
mation. For example, a credit card processing component of a workflow may need
to send credit information. The component should send only information required
for privacy protection and should use data filtering to remove information that
need not be passed to another application.
Filtering can be applied at the database level, too. Using filtering, you can
obtain a simplified view of the data tailored to a particular application. In addition,
using EJB container-managed persistence for data transformation makes it easier
to filter data. The container-managed persistence mapping tools let you select a
subset of the database schema, and this is analogous to filtering. If you are using
the JDBC RowSet approach, you need only select the columns for the data that
you care about.
You can also write façades that appropriately filter the data. In this case, the
client code accesses data only through these façades.

6.5 Guidelines for Integration

Now that we’ve examined integration design approaches and techniques, let’s look
at some guidelines for integrating enterprise applications with EIS systems.
We examine guidelines for integrating legacy systems, particularly home-
grown systems, and guidelines for using command beans. Although not as
common, we mention guidelines for achieving integration by using metadata, reg-
istries, and versioning and evolution.

6.5.1 Integrating Custom Legacy Systems

Most enterprises have home-grown or custom EISs, also referred to as one-off sys-
tems, and these systems need to be integrated. As we have already seen, connectors
are the best solution for integrating EISs in a J2EE environment. However, because
these are custom systems, there are no off-the-shelf or vendor-provided connectors
available for them as there are for well-known EISs. As a result, you may have to
write your own connector for a home-grown EIS. The recommended way to write a
connector is to use a connector builder tool. A tool such as this not only generates a
284 Guidelines for Integration

connector, but (with the help of other J2EE application server services) it may also
create a Web services layer.
Generally, there are two scenarios involving the integration of custom sys-
tems. In one case, two EIS systems need to interact with each other. The connector
approach works well with this type of scenario: You merely write Java classes
implementing the business logic for one EIS and use a connector to communicate
to the other EIS system. In the second scenario, you intend to have several appli-
cations use a custom EIS. For this case, it is better to plug the custom EIS into the
application server and expose the EIS’s reusable business logic through the appli-
cation server. You can expose the logic either by using enterprise beans with JMS
or by using a Web services approach. If the situation warrants it, you can use both
enterprise beans with JMS and a Web services approach together.

6.5.2 Using Screen Scraping for Integration

There are times when you may want to integrate an EIS that has no programming
interface. Often, this happens with legacy mainframe applications. In these cases,
you can resort to screen scraping to create a programming interface. With screen
scraping, you write an adapter layer that acts as an end user entering data into the
mainframe application, and this adapter layer serves as the programming interface.
You then write a connector that uses this programming interface to accomplish its
integration actions.
When resorting to screen scraping, be sure to keep in mind the limitations of
the legacy system. Although you can integrate a new application to a legacy
system, the original system has limitations that may make it unable to handle the
new functionality. For example, suppose you used screen scraping to integrate a
front-end application with a mainframe application designed to be interactive. The
front-end application, since it has no human constraints, can suddenly pump in a
high volume of requests to the mainframe application, which may not have been
designed to handle such a load. It is hard to detect a problem such as this until
runtime, when the system is suddenly overwhelmed with a high volume of
requests. By this point in the development cycle, it may be quite expensive to fix
the application.
Chapter 6 Enterprise Application Integration 285

6.5.3 Metadata
Metadata is information that describes the characteristics of a data set. For an EIS
system, metadata describes the content, quality, and condition (among other charac-
teristics) of the EIS’s data.
EISs provide a range of access to this metadata. Some EISs provide program-
matic access to their metadata, while others provide the metadata information in
text form only, as a form of documentation. Sometimes, the metadata information
is available through database system support tools. For example, some databases
allow queries to learn its table names and schemas. Similarly, an EIS may give
you access to its various quality of service parameters.
Metadata is often relevant to integration. Metadata allows you to use tools to
discover properties of enterprise systems, and from this discovery to create appro-
priate, easy-to-use façades to the systems. By having access to metadata, tools can
generate more meaningful classes.
Web services are designed to support metadata. Web services rely on WSDL
files, which essentially provide metadata describing the services, the operations
offered by the services, parameters for these operations, and so forth. Since these
WSDL files are XML documents, they are accessible to tools and other programs.
(See “Web Services Description Language” on page 36.)
Using an integration layer helps make metadata more explicit. When metadata
is implicitly associated with EIS data, tools may have a difficult time discovering
the metadata. For example, a database table column may represent distances from
a certain point, and these distances are implicitly measured in miles. A tool that
accesses the data from this table column may not be able to determine whether the
distance is measured in miles or kilometers. By creating an integration layer, you
can make this information explicit. You can name the methods that access the data
in such a way as to indicate the associated metadata. For example, rather than
have a getDistance method, you can call the method getDistanceInMiles. For
Web services, the standard document formats that describe the service are
designed to make the metadata more explicit.
Another way to make metadata explicit and accessible, especially for medium
to large enterprises, is to store the metadata in a central location, such as a tradi-
tional LDAP directory. (LDAP, which stands for Lightweight Directory Access
Protocol, is an Internet protocol that programs can use to look up information
from a server.) You can make your EIS’s metadata available to others in several
ways. You can enable metadata support in your enterprise systems. The WSDL
file that accompanies a Web service already defines metadata for the service. You
286 Guidelines for Integration

may also provide specific methods that retrieve metadata related to quality of ser-
vices, parameter constraints, and so forth. Publishing a schema for a document
that a service accepts is another way to provide metadata.
When deciding what information to store in a directory, try to store data that is
useful to more than one application. Storing metadata in a directory also helps if
you need to analyze the impact of any changes to the application.
Also, consider implementing directory-based data sharing, since this enables
applications to operate collectively. Such data sharing reduces management costs
and improves an enterprise’s overall responsiveness to business change. If you use
this approach, be sure to set up authorization policies to control access to the data.
It is also important when using a directory service to locate authoritative or
reliable sources of the data. A major reason for not using a directory service is
outdated or suspect data. Users quickly learn that they cannot rely on the service
and hence stop using it. Keep in mind, however, that a single application or data-
base is rarely authoritative for all required data, and this may cause a data integra-
tion problem of its own.

6.5.4 Using Registries for Integration

You may want to consider using registries to integrate an application with an exist-
ing EISs, especially when you want a loose binding between the application and the
systems with which it is integrated. For example, in the Web services approach, you
can store the URL and the WSDL for the Web service in a UDDI registry. However,
using a registry comes with the additional overhead of running and maintaining a
registry server and the added programming complexity to have your application use
the registry. The registry server may also be a single point of failure for your system.
Generally, it is not worthwhile to use registries for a small enterprise whose
applications integrate with just a few EISs. Registries make more sense for
medium and large integration architectures. You want to maximize the use of the
registry among as many applications as possible.

6.5.5 Versioning and Evolution

Applications integrated with EISs change over time, as do the EISs. As these
systems and applications evolve over their lifetimes, new versions of them emerge.
Integration of the applications and EISs must be able to handle this evolution and
versioning process.
Chapter 6 Enterprise Application Integration 287

Many enterprise applications change because of changes to the underlying

business requirements. EISs change for similar reasons, too, but they may also
change because of hardware and software upgrades and bug fixes. New function-
ality may be added to applications and existing functionality retired. Sometimes
policies change and, as a result, applications must access EISs in a different
manner.
Generally, it is easier to evolve the various components—the enterprise appli-
cations and EISs—than it is to evolve the integration layer, since it is the point of
stability. Let’s first look at some strategies to evolve EISs, and then discuss how to
evolve the different types of integration layers.
Sometimes as it evolves, the changes to an EIS are internal only. The external
functionality—the functionality visible to other applications—remains the same.
When this is the case, you should strive to keep unchanged the original external
interface specified in the integration layer. To do this, make the sort of changes to
the internal implementation of the EIS that do not impact the external interface;
that is, the internal implementation should be changed so that it adapts to the orig-
inal external interface.
It is different when the integration layer evolves. One way to handle this is to
use a transformation layer, which is an additional layer added to enable older
applications to continue to work. A transformation layer accepts messages in the
older style used by these applications and transforms them to the current format.
Often, because of their looser coupling, it is easier to evolve integration layers that
are Web services than other types of integration layers.
When you control both ends of the integration point (the Web services end-
point and the client), it is often easier to upgrade them both at the same time.
When just the internal implementation of the Web service changes—and these
changes have no affect on the external interface—you merely plug in the new
implementation.
Internal implementation changes to the Web service that cause changes to the
external interface can be relatively easier to handle if these changes enhance the
service but do not modify the original service contract. In these cases, you can
provide two sets of WSDL files, one describing the original service and the other
describing the new, enhanced service.
However, more complex scenarios exist, and you cannot expect clients to
immediately migrate to a new service implementation. Some clients may never
migrate to the new service. For more complex scenarios, a good strategy is to
publish a separate Web service endpoint that provides the new service version.
Clients can migrate to the new service when convenient for them. You can keep
288 Conclusion

the old service interface but re-implement the existing endpoint to use the new
implementation of the service. You may even be able to publish the new endpoint
under the same URL without breaking the existing clients. For example, if you are
just adding some additional ports to your WSDL, you should be able to update the
old WSDL with the new WSDL description and then existing clients should con-
tinue to work with no problems.

6.5.6 Writing Applications for Integration

One area in which you have quite a bit of flexibility involves how you code your
new EAI application. Since the application is new, you have the maximum amount
of choice for implementing how the application will access the various resources
that it requires. The J2EE platform, which is moving in this direction, provides some
facilities to help with this task.
The Java Business Integration (JBI) Systems Programming Interface, based
on JSR 208, extends the J2EE platform with a pluggable integration infrastructure
using WSDL-based message exchange. JBI is of most interest to integration-
independent software vendors (ISVs) rather than enterprise developers. JBI
enables ISVs to write integration modules that support business protocols, such as
Business Process Execution Language (BPEL), and plug them into a J2EE appli-
cation server using JBI mechanisms. JBI also provides these integration modules
with useful services such as threading, context management, security, lifecycle
management, connection pooling, timers, and so forth. As a developer, you deal
with the Web services for the various JBI-based integration services provided by
these ISVs.

6.6 Conclusion
This chapter examined some of the key issues for enterprise application integration.
It began by illustrating typical integration problems encountered by enterprises, and
then described the various integration technologies available with the J2EE 1.4 plat-
form. These J2EE technologies are not limited to Web services only, though Web
services represent one valid integration approach most useful in certain situations. In
addition to Web services, the J2EE platform has technologies for integrating with
relational databases and other types of data stores, messaging technologies, and EIS
access technologies.
The chapter described the major J2EE application integration approaches—
including using Web services, enterprise beans and JMS, connectors, and data
Chapter 6 Enterprise Application Integration 289

access objects—and detailed the situations when it was most appropriate to use
these different approaches. It also showed how the various approaches could be
combined.
The chapter concluded with a set of guidelines for achieving data, application,
and business process integration. It also presented some guidelines for integrating
home-grown systems, along with how to leverage the capabilities of registries for
integration purposes.
Chapter 7 discusses the approaches for implementing secure Web services.