The Global Search Service integrates ElasticSearch with Creatio. It performs the following functions:
- Recording:
-
Subscribes clients by creating an index in ElasticSearch and saves the connection between the index and the application.
-
Disconnects clients by removing their index in ElasticSearch.
-
- Transporting:
-
Participates in the indexing process by retrieving data from the database.
-
The sequence of actions during the global search setup depends on the global search version you are going to use. We recommend always using the latest version of the global search service with the latest version of Creatio.
You can deploy the components of global search version 3.0 using Kubernetes orchestrator and Helm package manager or Docker.
We also recommend backing up ElasticSearch daily to ensure the correct operation of the service and to enable the restoration of data after failures, e. g., power outages.
If you have any questions during the setup, we recommend consulting the Global search and deduplication FAQ.
Set up the global search service using Kubernetes
To set up the service, download the source files. Download files.
To install the service:
- Set up the target environment:
- Kubernetes cluster. Learn more about setting up and managing the cluster in the Kubernetes documentation.
- Helm package manager. Learn more about installing the package manager in the Helm documentation.
- Unpack the values-onsite.yaml file from the source file archive and save the file to the same directory as the archive.
- Open the file. View the main parameters the file uses in the table below.
- Specify the public service URL in the format of http://k8s-node:30332 in the global.searchService.url variable.
-
If you have not installed Redis, RabbitMQ, ElasticSearch, and PostgreSQL service database yet, proceed to the next step.
If you have already installed these services, for example, when setting up other containerized components, disable the installation of the services and set up the connection to existing services in the values-onsite.yaml source file of the global search service.
-
For Redis:
-
Disable the service installation. To do this, specify enabled: false in the redis section of the values-onsite.yaml file.
-
Set up the connection to the previously installed Redis. To do this, specify the connection parameters in the global.redis section of the values-onsite.yaml file.
Where
[host] is the address of the Redis server.
[port] is the connection port of the Redis server.
[database] is the name of the Redis database.
-
-
For RabbitMQ:
-
Disable the service installation. To do this, specify enabled: false in the rabbitmq section of the values-onsite.yaml file.
-
Set up the connection to the previously installed RabbitMQ. To do this, specify the connection parameters in the global.rabbitmq section of the values-onsite.yaml file.
Where
[host] is the address of the RabbitMQ service.
[vhost] is the virtual host address of the RabbitMQ service.
[port] is the amqp connection port of RabbitMQ.
[user] is the RabbitMQ user.
[password] is the RabbitMQ user password.
-
-
For PostgreSQL service database:
-
Disable the PostgreSQL installation. To do this, specify enabled: false in the postgresql section of the values-onsite.yaml file.
-
Set up the connection to the PostgreSQL service database. To do this, specify the connection parameters in the global.postgresql section of the values-onsite.yaml file.
Where
[user] is the PostgreSQL user on whose behalf to connect to the database.
[password] is the PostgreSQL user password.
[database] is the PostgreSQL service database.
[host] is the address of the PostgreSQL database.
[port] is the port to connect to the database.
-
-
For ElasticSearch:
-
Disable ElasticSearch installation. To do this, specify enabled: false in the elasticsearch section of the values-onsite.yaml file.
-
Set up the connection to the previously installed ElasticSearch. To do this, specify the connection parameters in the global.elasticsearch section of the values-onsite.yaml file.
Where
[user] is the ElasticSearch user.
[password] is the ElasticSearch user password.
[url] is the ElasticSearch service URL in the format of http://elasticsearch:9200.
-
-
- Run the helm install gs -f values-onsite.yaml globalsearch.tgz command. As a result, Helm will install the global search service and selected dependencies.
The main parameters of the global search service the values.yaml file uses.
Parameter |
Parameter description |
---|---|
scheduler.env.fillQueueInterval |
The run interval for primary indexing, in milliseconds. |
worker.env.indexingCommandTimeout |
The timeout of the Creatio database query upon primary indexing, in seconds. |
workerSingle.env.indexingCommandTimeout |
The timeout of the Creatio database query upon instant indexing, in seconds. |
global.incrementDays |
The number of days within which to index the changed records during a single primary indexing iteration. Affects the indexing speed and Creatio database load. The higher the value, the faster the indexing and higher the load. The lower the value, the longer the indexing and lower the load. |
log4Net |
Logging settings. |
global.indexingContentLength |
The maximum length of text fields upon indexing. |
global.elasticsearch |
ElasticSearch connection parameters. |
global.searchService |
Public URL to the search-service in the format of http://k8s-node:30332. |
global.rabbitmq |
RabbitMQ connection settings. |
global.db |
The connection settings of the global search service's internal service database. |
global.redis |
Redis connection settings. |
Set up the global search service in Docker
The global search requires 2 dedicated physical or virtual servers (“server 1” and “server 2”) with Linux installed. Use the requirements calculator to check the server requirements.
Learn more about the OS versions supported by Docker in the Docker documentation. Depending on your company needs, you can use either Docker Community Edition (CE) or Enterprise Edition (EE). Learn more in the Docker documentation.
Global search components
Deploy on server 1:
-
elasticsearch. The search engine.
Deploy on server 2:
-
postgres. The database for configuring the global search components.
-
rabbitmq. The message broker.
-
redis. The database used for caching and performance improvement.
-
gs-web-api. The web service that configures global search components.
-
gs-web-indexing-service. The web service that processes requests to perform targeted indexing of Creatio data.
-
gs-search-service. An elasticsearch proxy web service for data search.
-
gs-scheduler. The scheduler of Creatio data indexing in ElasticSearch.
-
gs-worker. The component that indexes Creatio data in ElasticSearch as per the scheduler tasks.
-
gs-worker-replay. The component that processes the indexing results (gs-worker results).
-
gs-worker-single. The component that performs the targeted indexing of business process data in ElasticSearch upon a request from the business process.
-
gs-worker-single-replay. The component that processes exceptions as part of the targeted indexing (gs-worker-single results).
-
gs-worker-single-task. The component that schedules tasks for gs-worker-single.
-
gs-worker-queried-single-task. The component that generates tasks for gs-worker-single.
To set up the components, download the source files. Download files.
The list of ports used by global search components:
Component name |
Outgoing port |
Incoming port |
Notes |
---|---|---|---|
gs-web-api |
|
81 |
Configure the incoming port using the WEB_API_PORT variable |
gs-web-indexing-service |
|
82 |
Configure the incoming port using the WEB_INDEXING_SERVICE_PORT variable |
gs-search-service |
9200 |
83 |
Configure the incoming port using the SEARCH_SERVICE_PORT variable |
gs-worker |
9200 |
|
Requires connection to the server where elasticsearch is located. |
gs-worker-single |
9200 |
|
Requires connection to the server where elasticsearch is located. |
elasticsearch |
|
9200 |
|
Global search setup procedure
-
Install Docker on a physical or virtual machine running Linux OS.
-
Install Docker-Compose.
-
Install ElasticSearch.
-
Set up the container variables.
-
Install and run the Global Search Service components.
-
Enable the global search functionality in Creatio.
Install Docker
Install Docker on Linux to deploy global search components. The installation is covered in the Docker documentation.
Run the docker --version command on a Linux machine to verify the installed Docker version.
Install Docker-Compose
The installation of Docker-Compose is covered in the Docker documentation.
Install ElasticSearch
To install ElasticSearch:
-
Go to the ElasticSearch installation server (server 1) and open the /opt directory.
-
Download and unpack the archive with setup files to the directory. Download the archive.
-
Open the /opt/docker-compose/elasticsearch directory (where the components are located) and run the following command:
The command might take up to several minutes to complete.
-
Make sure that the log files do not contain any errors after the command is complete. To do this, run the following command:
Set up the container variables
Configure the global search component containers via the file that contains the environment variables. The variables are stored in the /opt/compose/services/.env file. Edit this file to set the variables.
Variable name |
Details |
Default value |
---|---|---|
GS_ES_URL |
The external ElasticSearch host required for access from Creatio. Specify the IP address of the server where the ElasticSearch is deployed. |
http://elasticsearch-publicip:9200 |
CURRENT_SERVER_IP |
The external IP address of the server where the global search components are deployed (server 2). |
10.0.0.1 |
Additional variables that control the data indexing parameters in ElasticSearch
Variable name |
Details |
Default value |
---|---|---|
GS_DB_INCREMENT_DAYS |
Number of days to index per one scheduler iteration. ModifiedOn columns of Creatio records are used for comparison. |
500 days |
GS_DB_FILL_QUEUE_INTERVAL |
Creatio database data collection interval for the regular scheduler. The lower the variable, the higher the load on Creatio database and the faster the primary indexing. |
30000 (specified in milliseconds) |
Run containers that have Global Search Service components
-
Go to the server that has the global search components (server 2) and open the /opt folder.
-
Download and unpack the archive with setup files to the opened folder. Download the archive.
-
Open the /opt/compose/services folder and run the following command:
Verify that containers ran successfully
To see the running global search containers, use the following console command:
All currently running containers display the Up status.
Logging
By default, the container logging takes place in the stdout and stderr.
Connect the global search service to Creatio
Actions on the server
To connect global search to Creatio, take the following steps on the server where the global search components are located (server 2 for the service deployed in Docker):
-
Install the api-get install curl or yum install curl utility for HTTP queries.
-
Execute the HTTP request to register the site in global search. Specify the following:
-
DATABASE_TYPE: Creatio database type (mssql, postgresql, or oracle).
-
DATABASE_CONNECTION_STRING: Creatio database connection string
-
SITE_NAME: Creatio site name, e. g., my-test-site.
-
SERVER2_IP_ADDRESS (only for Docker): the IP address of the Linux server where the global search components are deployed.
GS_WEB_API_URL (only for Kubernetes): the IP address of the Linux server where the global search components are deployed.
-
-
Execute the HTTP request to connect the search to the site. Specify the following:
-
SITE_NAME: Creatio site name, e. g., my-test-site.
-
TEMPLATE_NAME: the name of the search template used in ElasticSearch. View the available templates in the table below.
-
SERVER2_IP_ADDRESS (only for Docker): the IP address of the Linux server where the global search components are deployed.
GS_WEB_API_URL (only for Kubernetes): the IP address of the Linux server where the global search components are deployed.
-
View the available search templates and their features in the table below:
|
Old template (version 1.6) |
default.json |
ngram_2.json |
ngram_3.json |
without_ngram.json |
---|---|---|---|---|---|
Search by partial match |
+ |
– |
+ |
+ |
– |
Search by misspelled words |
+ |
– |
+ |
+ |
– |
Search communication options by phone number (partial match) |
+ |
+ |
+ |
+ |
– |
Search communication options (partial match) |
+ |
+ |
+ |
+ |
– |
Search by word swapping |
+ |
+ |
+ |
+ |
+ |
Search by exact match |
+ |
+ |
+ |
+ |
+ |
Search by two characters |
– |
– |
+ |
– |
– |
Average search speed (lower is better) |
|
1x |
13x |
7x |
<1x |
Index size at elasticsearch (lower is better) |
|
1x |
4x |
2.5x |
<1x |
Primary indexing time (lower is better) |
|
1x |
1.8x |
1.4x |
<1x |
Settings in Creatio for Microsoft SQL DBMS
-
Toggle the global search (GlobalSearch, GlobalSearch_V2, GlobalSearchRelatedEntityIndexing) feature in Creatio by running the following SQL script:
-
Set the values of the system settings:
-
“GlobalSearchUrl:” the full path to ElasticSearch, including the index. The web-api returns this value if you request it to add site search.
Example string for Docker: http://[SERVER2_IP_ADDRESS]:83/indexname.
Example string for Kubernetes: http://[GS-SEARCH-SERVICE_URL] /indexname.
-
“GlobalSearchConfigServiceURL:” the global search API URL.
Default value for Docker: http://[SERVER2_IP_ADDRESS]:81.
Default value for Kubernetes: http:// [GS_WEB_API_URL].
-
“GlobalSearchIndexingApiUrl:” the instant indexing service URL.
Default value for Docker: http://[SERVER2_IP_ADDRESS]:82.
Default value for Kubernetes: http://[GS-WEB-INDEXING-SERVICE_URL].
-
-
Restart Creatio, flush Redis, and log in to the application.
Settings in Creatio for Oracle DBMS
-
Toggle the global search (GlobalSearch, GlobalSearch_V2, GlobalSearchRelatedEntityIndexing) feature in Creatio by running the following SQL script:
-
Set the values of the system settings:
To do this, run the following script:
-
“GlobalSearchUrl:” the full path to elasticsearch, including the index. The web-api returns this value if you request it to add site search.
Example string for Docker: http://[SERVER2_IP_ADDRESS]:83/indexname.
Example string for Kubernetes: http://[GS-SEARCH-SERVICE_URL] /indexname
-
“GlobalSearchConfigServiceURL:” the global search API URL.
Default value for Docker: http://[SERVER2_IP_ADDRESS]:81.
Default value for Kubernetes: http:// [GS_WEB_API_URL].
-
“GlobalSearchIndexingApiUrl:” the instant indexing service URL.
Default value for Docker: http://[SERVER2_IP_ADDRESS]:82.
Default value for Kubernetes: http://[GS-WEB-INDEXING-SERVICE_URL].
-
-
Restart Creatio, flush Redis, and log in to the application.
Settings in Creatio for PostgreSQL DBMS
-
Toggle the global search (GlobalSearch, GlobalSearch_V2, GlobalSearchRelatedEntityIndexing) feature in Creatio by running the following SQL script:
-
Set the values of the system settings:
-
“GlobalSearchUrl:” the full path to ElasticSearch, including the index. The web-api returns this value if you request it to add site search.
Example string for Docker: http://[SERVER2_IP_ADDRESS]:83/indexname.
Example string for Kubernetes: http://[GS-SEARCH-SERVICE_URL] /indexname.
-
“GlobalSearchConfigServiceURL:” the global search API URL.
Default value for Docker: http://[SERVER2_IP_ADDRESS]:81.
Default value for Kubernetes: http:// [GS_WEB_API_URL].
-
“GlobalSearchIndexingApiUrl:” the instant indexing service URL.
Default value for Docker: http://[SERVER2_IP_ADDRESS]:82.
Default value for Kubernetes: http://[GS-WEB-INDEXING-SERVICE_URL].
To do this, run the following script:
-
-
Restart Creatio, flush Redis, and log in to the application.