ElasticSearch in X3

Chris Cobbold
22/09/2021

© 2021 The Sage Group plc, or its licensors. All rights reserved.
Contents

◉ Introduction
Purpose , Usage, Architecture

◉ Installation

◉ Operation in X3
Automation Schedule

Housekeeping

◉ Troubleshooting

◉ Quick Demo

© 2021 The Sage Group plc, or its licensors. All rights reserved. 2
Introduction

© 2021 The Sage Group plc, or its licensors. All rights reserved. 3
 What is Elastic Search

Elastic Search is an Open Source product.

In simple terms, it provides the ability to search through structured and unstructured data in order to find anything
relating to your search criteria. In Sage terms, this could be a customer name or a function name you are searching for.

It is based on the Java full text search library Lucene.

Technically, the search engine creates its own search indexes, which it then uses when a user submits a query to search
for something. The Sage X3 development teams will define what data is indexed out of the box, but customers can
modify the fields that can be indexed and therefore searched, in order to satisfy their own business requirements

It Is part of the X3 Web Stack as seen in the next slide

© 2021 The Sage Group plc, or its licensors. All rights reserved. 4
X3 Web Stack

© 2021 The Sage Group plc, or its licensors. All rights reserved. 5
 When to Use and Why

Sage X3 primarily uses Elasticsearch for the search bar at the top of the dashboard once you have
logged in, and also for help topics not linked to online help.
This gives the user the capability for rapid searches and results of data specific to their business
function.
The associated X3 screens can then be accessed directly from the search output results.

© 2021 The Sage Group plc, or its licensors. All rights reserved. 6
Architecture

© 2021 The Sage Group plc, or its licensors. All rights reserved. 7
Architecture

Elastic Search can be installed on its own server, or alongside any other Sage X3 components. Logically it would
naturally sit on the same server as the Syracuse server, as Syracuse is the process which will be accessing Elastic Search
data.

Alternatively a cluster configuration could be considered where ElasticSearch consists of multiple nodes and data is
organized and distributed between them.

Syracuse can only point to one ElasticSearch host/port , hence if a node in the cluster is unavailable Syracuse would
need to be reconfigured to point to a working node. An external load balancer in front of the ElasticSearch nodes could
be a potential workaround to this.

© 2021 The Sage Group plc, or its licensors. All rights reserved. 8
 Installation

© 2021 The Sage Group plc, or its licensors. All rights reserved. 9
Pre-Reqs

ElasticSearch is no longer delivered alongside other X3 technology components and must be obtained from
the ElasticSearch website.

ElasticSearch is built using Java, a bundled version of OpenJDK is maintained with each distribution and is
located within the jdk subdirectory under ElasticSearch home
ElasticSearch requires Oracle Java Development Kit (JDK) version 8 or higher to run
If extensive use of indexing is to be made it may be worth considering a performing storage tier for hosting
ElasticSearch data. However the use of 10krpm drives in RAID-10 should give adequate performance for
user data queries and indexing.
Ideally in addition ElasticSearch requires at least 2 cores and 4GB of RAM and hosted on a dedicated VM

© 2021 The Sage Group plc, or its licensors. All rights reserved. 10
Pre-Reqs

© 2021 The Sage Group plc, or its licensors. All rights reserved. 11
SageCity Additional Info / KB Install Notes

© 2021 The Sage Group plc, or its licensors. All rights reserved. 12
ElasticSearch Location

© 2021 The Sage Group plc, or its licensors. All rights reserved. 13
ElasticSearch Location

© 2021 The Sage Group plc, or its licensors. All rights reserved. 14
ElasticSearch Enironment Variables

© 2021 The Sage Group plc, or its licensors. All rights reserved. 15
ElasticSearch Installation

© 2021 The Sage Group plc, or its licensors. All rights reserved. 16
ElasticSearch Installation

© 2021 The Sage Group plc, or its licensors. All rights reserved. 17
Operation in X3

© 2021 The Sage Group plc, or its licensors. All rights reserved. 18
ElasticSearch Terminology

1. FIELDS – Smallest unit of data , analogous to Database Column

2. DOCUMENTS – Base unit of storage , json objects , analogous to Database Row
3 INDEX – Largest unit of data, logical partitions of documents, analogous to Database
4 SHARD – A Lucence index, used to split indices across operations and nodes
5 NODE – A single server instance
6 CLUSTER – A group of nodes organized to process data efficiently
7 REPLICA – Copies of Index Shards often used for high availability

© 2021 The Sage Group plc, or its licensors. All rights reserved. 19
Search Index Administration

© 2021 The Sage Group plc, or its licensors. All rights reserved. 20
Housekeeping

As new data is added to the entities, the indexes will become stale. The statistics held for each index will need to be
updated to ensure an up to date status for the data optimizer.

In addition newly entered user data will not be searchable until the indexes are updated.

Schedule regular updates of the search indexes via batch server tasks. This can be achieved via the Automation
scheduler screen which can be seen in the following slides

Check the logging level of ElasticSearch messages in the logging.yml. The default logging level is INFO which
is quite verbose, hence consider setting to WARN.
Regularly archive the logs to reduce diskspace overhead, the ElasticSearch service needs to be shutdown in
order to archive the latest logs

© 2021 The Sage Group plc, or its licensors. All rights reserved. 21
Automation Schedule

© 2021 The Sage Group plc, or its licensors. All rights reserved. 22
Automation Schedule

© 2021 The Sage Group plc, or its licensors. All rights reserved. 23
X3 Search

© 2021 The Sage Group plc, or its licensors. All rights reserved. 24
Additional Indexes
.

1. Go to Development--> Data and Parameters--> Classes--> Classes (GESATB), select the class you wish to make
searchable (e.g. WORKCOST) and tab Properties (check if the field you are interested in, such as "ACCCOD" field is
searchable). If not make it searchable and do a "Global Validation"

2. Go to Development--> Data and Parameters--> Classes--> Representations (GESASW) Select the appropriate name
(e.g. WORKCOST) and ensure "Used for search results" is checked. Also look in the “Available Properties” tab check
"Detail" is set to "Yes" for the field you are interested in. Save any changes then run "Global Validation" to ensure
the representation record is up to date

3. Go to Administration--> Usage--> Search Index Management. Click the "Entities" button and you should now see
the entity you just updated on the list (e.g. WORKCOST) You can now select this entity, then select "Delete index
before update” and launch the "Update Index" for this entity

4. You can then test this new entity is searchable within X3 itself

Demo : TABPLACE

© 2021 The Sage Group plc, or its licensors. All rights reserved. 25
Troubleshooting

© 2021 The Sage Group plc, or its licensors. All rights reserved. 26
Troubleshooting

1. Check for matching port configuration in:-

nodelocal.js value of searchengine: elasticsearch.yml value of http port:
2. Check the port is in use – Powershell command $netstat –anop TCP | sls –pattern “LISTEN”
3. Check the ElasticSearch process is running via the Windows Task manager
4. Check if proxy used and ElasticSearch server is Excluded
5. Obtain additional trace information by configuring logging level in logging.yml and check logs
6 Add the browser plug-in extension ‘Elasticsearch Head’
7. Use of TCP packet analyzer e.g Wireshark
8. Sage Support Investigation Script - mzElasticURL

© 2021 The Sage Group plc, or its licensors. All rights reserved. 27
Troubleshooting – Syracuse Logs

© 2021 The Sage Group plc, or its licensors. All rights reserved. 28
Troubleshooting – ElasticSearch logs

© 2021 The Sage Group plc, or its licensors. All rights reserved. 29
ElasticSearch Head

This is a basic visualization tool for the data stored in ElasticSearch. It is particularly useful for
testing queries, mappings of indices and reviewing data structures of the documents
We can see the Cluster Health RAG status indicating primarily the status of the Shards – Red
(primary shards not available. Yellow – some replica shards not available. Green – all available.
There are details on the index names e.g gb_entities and the number of documents within each
index
Basic CRUD queries can be performed on the indices

© 2021 The Sage Group plc, or its licensors. All rights reserved. 30
Troubleshooting – ElasticSearch Head - Overview

© 2021 The Sage Group plc, or its licensors. All rights reserved. 31
Troubleshooting – ElasticSearch Head - Indices

© 2021 The Sage Group plc, or its licensors. All rights reserved. 32
Troubleshooting – Script mzElasticURL

© 2021 The Sage Group plc, or its licensors. All rights reserved. 33
Quick Demo

© 2021 The Sage Group plc, or its licensors. All rights reserved. 34
Demo Screens – Data Search

© 2021 The Sage Group plc, or its licensors. All rights reserved. 35
Demo Screens - Classes

© 2021 The Sage Group plc, or its licensors. All rights reserved. 36
Demo Screens - Classes

© 2021 The Sage Group plc, or its licensors. All rights reserved. 37
Demo Screens - Classes

© 2021 The Sage Group plc, or its licensors. All rights reserved. 38
Demo Screens - Classes

© 2021 The Sage Group plc, or its licensors. All rights reserved. 39
Demo Screens - Representations

© 2021 The Sage Group plc, or its licensors. All rights reserved. 40
Demo Screens - Representations

© 2021 The Sage Group plc, or its licensors. All rights reserved. 41
Demo Screens – Search Index

© 2021 The Sage Group plc, or its licensors. All rights reserved. 42
Demo Screens – Search Index

© 2021 The Sage Group plc, or its licensors. All rights reserved. 43
Demo Screens – Search Index

© 2021 The Sage Group plc, or its licensors. All rights reserved. 44
Demo Screens – ES Head Indices

© 2021 The Sage Group plc, or its licensors. All rights reserved. 45
Demo Screens – ES Head Structured Query

© 2021 The Sage Group plc, or its licensors. All rights reserved. 46
Demo Screens – X3 Search

© 2021 The Sage Group plc, or its licensors. All rights reserved. 47
Demo Screens – Results

© 2021 The Sage Group plc, or its licensors. All rights reserved. 48
Demo Screens – Automation Schedule

© 2021 The Sage Group plc, or its licensors. All rights reserved. 49
Demo Screens – Search Index Management

© 2021 The Sage Group plc, or its licensors. All rights reserved. 50
Demo Screens – Search Index Management

© 2021 The Sage Group plc, or its licensors. All rights reserved. 51
Demo Screens – Search Index Management

© 2021 The Sage Group plc, or its licensors. All rights reserved. 52
Demo Screens – Search Index Management

© 2021 The Sage Group plc, or its licensors. All rights reserved. 53
Demo Screens – Modified Index

© 2021 The Sage Group plc, or its licensors. All rights reserved. 54
Demo Screens – Modified Index

© 2021 The Sage Group plc, or its licensors. All rights reserved. 55
Demo Screens – Modified Index

© 2021 The Sage Group plc, or its licensors. All rights reserved. 56
More information

© 2021 The Sage Group plc, or its licensors. All rights reserved. 57
 Information Sources

Sage University - targeted training courses at :-

https://sageu.com/access/su/home.html

Sage City – Information/Blogs/Videos etc on Technical specifics at:-

https://www.sagecity.com/gb/sage-x3-uk/

ElasticSearch Documentation at :-
https://www.elastic.co/guide/index.html

© 2021 The Sage Group plc, or its licensors. All rights reserved. 58
 Thank you

©2021 The Sage Group plc or its licensors. All rights reserved. Sage, Sage logos, and Sage product and service names mentioned herein
are the trademarks of Sage Global Services Limited or its licensors. All other trademarks are the property of their respective owners.