Tibco Enterprise Message Service™ Installation On Red Hat Openshift Container Platform
Tibco Enterprise Message Service™ Installation On Red Hat Openshift Container Platform
Contents
About this Product . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
Fault Tolerance and Shared Folder Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
Shared Storage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
Control Access to NFS Shared Folders . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
Setting Up the Shared Folder . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
OpenShift Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
Creating a Project . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
Provisioning the NFS Shared Folder . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
Enabling Images to Run with the USER Specified in the Dockerfile . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
Modifying the Services NodePort range (Optional) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
EMS Server Template . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
Parameters Objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
Service Objects and EMS Client URLs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
Deployment Object . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
Health Checks: Liveness and Readiness Probes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
Creating a Deployment and Service . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
Stopping or Deleting an EMS Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
EMS Server Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
Central Administration Server Template . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .14
Creating a Deployment and Service through the Web Console . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
TLS Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
Creating a Secret . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
Modifying the Template . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .17
Modifying the tibemscreateimage EMS Docker Image Build Script . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .18
Applying the Modifications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
TIBCO Enterprise Message Service™ Installation on Red Hat OpenShift Container Platform
3
Product Editions
● Users may run up to 100 application instances or 1000 web/mobile instances in a production
environment.
● Users do not have access to TIBCO Support, but you can use TIBCO Community as a resource (https://
community.tibco.com).
● Available on Red Hat Enterprise Linux Server, Microsoft Windows & Windows Server and Apple
macOS.
TIBCO Messaging - Community Edition has the following additional limitations and exclusions:
TIBCO Enterprise Message Service™ Installation on Red Hat OpenShift Container Platform
4
Overview
Running TIBCO Enterprise Message Service on Red Hat OpenShift Container Platform involves:
● Docker concepts.
● Red Hat OpenShift Container Platform administration.
● TIBCO EMS configuration.
● NFSv4.
Before you continue, make sure you have the following infrastructure in place:
TIBCO Enterprise Message Service™ Installation on Red Hat OpenShift Container Platform
5
Shared Storage
A traditional EMS server configured for fault tolerance relies on its state being shared by a primary and a
secondary instance, one being in the active state while the other is in standby, ready to take over. The
shared state relies on the server store and configuration files to be located on a shared storage such as a
SAN or a NAS using NFS.
By contrast, the fault tolerance model used by EMS on OpenShift relies on OpenShift restart mechanisms.
Only one EMS server instance is running and, in case of a server failure, will be restarted inside its
container. In case of a failure of the container or of the corresponding cluster node, the cluster will recreate
the container, possibly on a different node, and restart the EMS server there.
Within the container, the health of the EMS server is monitored by two health check probes:
● liveness
● readiness
For more information on the probes, see Health Checks: Liveness and Readiness Probes.
The server requires its state to be shared. The shared storage required by EMS on OpenShift is NFSv4.
1. Log on to a machine that can access the NFS shared folder with the user account meant to be used by
the EMS server.
2. Create the shared folder.
For example, ~/OpenShift/shared.
3. Modify the permissions to your requirements.
For example, 750 (rwxr-x---)
Example
> mkdir -p ~/OpenShift/shared
> chmod -R 750 ~/OpenShift/shared
TIBCO Enterprise Message Service™ Installation on Red Hat OpenShift Container Platform
6
The following examples illustrate how you can experiment with that Docker image:
● This following command creates a sample EMS server folder hierarchy and configuration in the current
directory and starts the corresponding server:
> docker run -p 7222:7222 -v `pwd`:/shared ems:8.6.0 tibemsd
● To create a sample Central Administration server folder hierarchy and configuration in the current
directory and starts the corresponding server:
> docker run -p 8080:8080 -v `pwd`:/shared ems:8.6.0 tibemsca
● You can override the creation and use of the sample configuration with your own setup. The following
example starts an EMS server using the <path to shared location>/<your server config file>
configuration.
> docker run -p 7222:7222 -v <path to shared location>:/shared \
ems:8.6.0 tibemsd -config /shared/<your server config file>
TIBCO Enterprise Message Service™ Installation on Red Hat OpenShift Container Platform
7
2. From the temporary folder, use a Dockerfile based on the example given below to copy these files into
the base Docker image:
FROM ems:8.6.0
COPY --chown=tibuser:tibgroup . /opt/tibco/ems/docker/ftl
3. After customizing your EMS configuration, include /opt/tibco/ems/docker/ftl in the Module Path
property.
1. Copy your custom JAAS or JACI plugin files, including the static configuration files they may rely on, to
a temporary folder.
2. From the temporary folder, use a Dockerfile based on the example given below to copy these files into
the base Docker image:
FROM ems:8.6.0
COPY --chown=tibuser:tibgroup . /opt/tibco/ems/docker/security
3. After customizing your EMS configuration, make sure you include the relevant paths to those files in
the Security Classpath property.
The other required files are in their usual location /opt/tibco/ems/version/bin and /opt/
tibco/ems/version/lib.
TIBCO Enterprise Message Service™ Installation on Red Hat OpenShift Container Platform
8
OpenShift Setup
Creating a Project
Create a project with an OpenShift user that has the required permission.
For example:
> oc login -u <user>
> oc new-project ems-project
Procedure
(1):Optional comma-separated list of NFS mount options used when the PV is mounted on a cluster
node.
(2): The path that is exported by the NFS server. In this example, we want the patch to match the ~/
OpenShift/shared folder created in Setting Up the Shared Folder.
(3): The host name or IP address of the NFS server.
(4): This needs to match the name of the project created previously.
TIBCO Enterprise Message Service™ Installation on Red Hat OpenShift Container Platform
9
2. Switch to the system:admin cluster administrator and create the PV and PVC:
> oc login -u system:admin
> oc create -n ems-project -f nfs-pv-pvc.yaml
The same PV/PVC can be used by multiple pods within the same project.
Creating the PV/PVC is done once for the lifetime of the project.
Procedure
2. Edit this file and give this SCC a new name by changing restricted into nfs-scc and give it a higher
priority.
3. Add a uidRangeMax and a uidRangeMin field to the runAsUser entry. These define a range of allowed
uid values and should match the values you expect your pods to use for accessing NFS. For example:
…
metadata:
…
name: nfs-scc (1)
priority: 9 (2)
…
runAsUser:
type: MustRunAsRange
uidRangeMax: 13000 (3)
uidRangeMin: 12000 (3)
…
You create the SSC once for the lifetime of the cluster.
TIBCO Enterprise Message Service™ Installation on Red Hat OpenShift Container Platform
10
To use port numbers outside this range for the EMS server or Central Administration server, you can alter
the range in the Kubernetes Master Configuration:
Procedure
1. Locate the file master/master-config.yaml on each OpenShift Master and edit it to set the value of
servicesNodePortRange to the range of your choice.
For example:
serviceNodePortRange: 7000-8000
Parameters Objects
The parameters let you configure the aspects of the container and service that can be adjusted at creation
time. These include:
Option Description
EMS_INTERNAL_PORT The port number used by the EMS server inside its
container.
All parameters have a default value that can be overridden upon creation.
The uid provided here must match the one used when creating the EMS Docker image.
TIBCO Enterprise Message Service™ Installation on Red Hat OpenShift Container Platform
11
For example, if your cluster runs on three nodes called node1, node2 and node3 that can be addressed by
those host names. If you have exposed your EMS server through a service using port number 8222, EMS
clients running outside the cluster will be able to access it either through the tcp://node1:8222, tcp://
node2:8222 or tcp://node3:8222 URL, regardless of the node where the container is actually running.
This works by virtue of each node proxying port 8222 into the service.
EMS clients running inside the cluster will be able to access the EMS server either in the fashion described
above or through its service name. Assuming the service name is emsdev01 and the port still is 8222, that
amounts to using the tcp://emsdev01:8222 URL.
To ensure EMS client automated fault-tolerance failover, these must connect with FT double URLs. Using
the example above: tcp://node1:8222, tcp://node1:8222 from outside the cluster or tcp://
emsdev01:8222, tcp://emsdev01:8222 from inside the cluster. For the first form, since all nodes will
proxy port 8222 into the service, repeating the same node name twice fits our purpose. The connection
factories in the sample EMS server configuration generated by default upon creating a container illustrate
that pattern. Should the EMS server or its container fail, clients will automatically reconnect to the same
URL once the server has been restarted.
You can use types of service other than NodePort if they fit your requirements.
Deployment Object
A deployment includes the definition of a set of containers and the desired behavior in terms of number of
replicas (underlying ReplicaSet) and deployment strategy.
kind: Deployment
…
spec:
replicas: 1 (1)
…
strategy:
type: Recreate (2)
…
template:
…
spec:
containers:
- name: tibemsd-container
image: ${EMS_IMAGE_LOCATION}
imagePullPolicy: Always (3)
env: (4)
- name: EMS_NODE_NAME
valueFrom:
fieldRef:
fieldPath: spec.nodeName
- name: EMS_PUBLIC_PORT
value: ${EMS_PUBLIC_PORT}
…
args: (5)
- tibemsd
livenessProbe: (6)
…
readinessProbe: (6)
…
ports:
- containerPort: ${{EMS_INTERNAL_PORT}}
name: tibemsd-tcp
protocol: TCP
…
securityContext:
runAsUser: ${{EMS_UID}} (7)
…
volumeMounts:
- mountPath: /shared (8)
name: tibemsd-volume (9)
…
restartPolicy: Always (10)
…
TIBCO Enterprise Message Service™ Installation on Red Hat OpenShift Container Platform
12
volumes:
- name: tibemsd-volume (9)
persistentVolumeClaim:
claimName: ${{EMS_PVC}} (11)
(1): The number of replicated pods: 1, since we want a single instance of the EMS server.
(2): The deployment strategy: Recreate means that an existing pod must be killed before a new one is
created.
(3): Determines if the EMS Docker image should be pulled from the Docker registry prior to starting the
container.
(4): Environment variables that will passed to the container.
(5):Arguments to be passed to the Docker ENTRYPOINT. For more information, see EMS Server
Configuration.
(6): For details on the liveness and readiness probes, see Liveness and Readiness Probes.
(7): The uid the container will run as.
(8): The path where our NFS shared folder will be mounted inside of the container.
(9): The internal reference to the volume defined here.
(10): The pod restart policy: Set such that the kubelet will always try to restart container. If the EMS server
stops or fails, its container will exit and be restarted.
(11): The name of the PVC created by the cluster administrator.
(1): Number of seconds after the container has started before the probe is initiated.
(2): Number of seconds after which the probe times out. Defaults to 1 second. Minimum value is 1.
(3): How often (in seconds) to perform the probe. Defaults to 10 seconds. Minimum value is 1.
TIBCO Enterprise Message Service™ Installation on Red Hat OpenShift Container Platform
13
If you are logged in as a cluster administrator, log out and log back in as a regular user.
Procedure
1. Edit the tibemsd-template.yaml template and override the default parameters as needed.
2. Create a deployment and service with an EMS server using the modified template.
For example,
> oc login -u <user>
> oc process -f tibemsd-template.yaml -p EMS_SERVICE_NAME=emsdev01 \
-p EMS_PUBLIC_PORT="7779" -p EMS_PVC="claim-nfs-ems-project" \
| oc create -f -
To stop an EMS server without deleting it, use the oc scale operation to set its number of replicas to 0.
For example,
> oc scale --replicas=0 deploy emsdev01
To delete an EMS server deployment and service, use the oc delete operation.
For example,
> oc delete --selector name=emsdev01 deploy,svc
TIBCO Enterprise Message Service™ Installation on Red Hat OpenShift Container Platform
14
You can then use a Web browser to connect to http://node1:7080 and add EMS servers to Central
Administration.
Procedure
1. Upload the templates to your project using the OpenShift Command Line Interface (CLI)
> oc create -f tibemsd-template.yaml
> oc create -f tibemsca-template.yaml
2. Log in to the Web Console, select project ems-project, and then Add to Project > Select from Project:
3. The two templates that you just uploaded are available for you to choose from.
TIBCO Enterprise Message Service™ Installation on Red Hat OpenShift Container Platform
15
vailable
4. Select TIBCO EMS and click Next.
The Information screen displays links to reach the TIBCO EMS documentation and TIBCO Support.
5. Click Next.
TIBCO Enterprise Message Service™ Installation on Red Hat OpenShift Container Platform
16
The Configuration screen is displayed. This is where you can modify the template parameters with
values for this particular instance.
TIBCO Enterprise Message Service™ Installation on Red Hat OpenShift Container Platform
17
TLS Configuration
The following topics describe how to modify the EMS server template and the Docker image build script so
that EMS clients can connect to the server through TLS (formerly SSL).
Whether an EMS listen port is configured for TCP or TLS makes no difference in terms of exposing it
through a service. However, you need to decide how to provision the corresponding certificate files.
While these could be placed in the NFS shared folder or embedded in the EMS Docker image, the standard
practice in the OpenShift world consists of using secret objects. These are meant to decouple sensitive
information from the pods and can be mounted into containers as volumes populated with files to be
accessed by programs.
In this example, the EMS server will be authenticated by EMS clients. This involves providing the server
with its certificate, private key and the corresponding password, which we will store inside a secret. We will
mount that secret into the container, point the EMS server configuration to the certificate and private key
files and pass the corresponding password to the server through its -ssl_password command-line option.
Based on the sample certificates that ship with EMS, the files will eventually be made available inside the
container as follows:
/etc/secret/server.cert.pem
/etc/secret/server.key.pem
/etc/secret/ssl_password
Creating a Secret
To store the server certificate, private key, and the corresponding password in a secret, based on the sample
certificates available in the EMS package under ems/version/samples/certs:
> cd …/ems/<version>/samples
> oc create secret generic tibemsd-secret \
--from-file=server.cert.pem=certs/server.cert.pem \
--from-file=server.key.pem=certs/server.key.pem \
--from-literal=ssl_password=password
TIBCO Enterprise Message Service™ Installation on Red Hat OpenShift Container Platform
18
persistentVolumeClaim:
claimName: ${{EMS_PVC}}
- name: tibemsd-secret-volume
secret:
secretName: tibemsd-secret
1. Regenerate the EMS Docker image, tag it, and push it to the Registry.
See Creating the Base Docker Image
2. Create a new deployment and service. See Creating a Deployment and Service.
You can check the result by connecting to the server with one of the EMS TLS sample clients:
> java tibjmsSSL -server ssl://node1:7779 \
-ssl_trusted ../certs/server_root.cert.pem \
-ssl_hostname server
TIBCO Enterprise Message Service™ Installation on Red Hat OpenShift Container Platform
19
Documentation for TIBCO products is available on the TIBCO Product Documentation website, mainly in
HTML and PDF formats.
The TIBCO Product Documentation website is updated frequently and is more current than any other
documentation included with the product. To access the latest documentation, visit https://docs.tibco.com.
The following documents for this product can be found on the TIBCO Enterprise Message Service ™
product documentation page:
● TIBCO Enterprise Message Service User’s Guide Read this manual to gain an overall understanding of the
product, its features, and configuration.
● TIBCO Enterprise Message Service Central Administration Read this manual for information on the central
administration interface.
● TIBCO Enterprise Message Service Installation Read the relevant sections of this manual before installing
this product.
● TIBCO Enterprise Message Service C & COBOL Reference The C API reference is available in HTML and
PDF formats.
● TIBCO Enterprise Message Service Java API Reference The Java API reference can be accessed only through
the HTML documentation interface.
● TIBCO Enterprise Message Service .NET API Reference The .NET API reference can be accessed only
through the HTML documentation interface.
● TIBCO Enterprise Message Service Installation on Red Hat OpenShift Container Platform This manual
®
describes how to run TIBCO Enterprise Message Service servers on the Red Hat OpenShift Container
Platform.
● TIBCO Enterprise Message Service Release Notes Read the release notes for a list of new and changed
features. This document also contains lists of known issues and closed issues for this release. This
document is available only in PDF format.
You may find it useful to read the documentation for the following TIBCO products:
● TIBCO FTL®
● TIBCO Rendezvous®
● TIBCO EMS® Client for z/OS (CICS)
● TIBCO EMS® Client for z/OS (MVS)
● TIBCO EMS® Client for IBM i
Third-Party Documentation
TIBCO Enterprise Message Service™ Installation on Red Hat OpenShift Container Platform
20
● Java™ Authentication and Authorization Service (JAAS) LoginModule Developer's Guide and Reference
Guide, available through http://www.oracle.com/technetwork/java/javase/jaas/index.html.
TIBCO Community is the official channel for TIBCO customers, partners, and employee subject matter
experts to share and access their collective experience. TIBCO Community offers access to Q&A forums,
product wikis, and best practices. It also offers access to extensions, adapters, solution accelerators, and
tools that extend and enable customers to gain full value from TIBCO products. In addition, users can
submit and vote on feature requests from within the TIBCO Ideas Portal. For a free registration, go to
https://community.tibco.com.
TIBCO Enterprise Message Service™ Installation on Red Hat OpenShift Container Platform
21
TIBCO Enterprise Message Service™ Installation on Red Hat OpenShift Container Platform