Deploy Data Loader

From Genesys Documentation
Jump to: navigation, search
This topic is part of the manual Genesys Predictive Routing Deployment and Operations Guide for version 9.0.0 of Genesys Predictive Routing.

Data Loader uploads and joins data from various sources in your environment to create datasets, which form the basis on which Predictive Routing (GPR) can score agents and improve key performance indicator (KPI) outcomes.

Related documentation:

About Data Loader

What Data Loader does:

  • It connects to Genesys Info Mart to retrieve data for the Agent Profile dataset.
  • It connects to Genesys Info Mart to collect interaction data, which it saves in one or more datasets for use in creating predictors and models.
  • It enables you to upload .csv files containing customer and outcome data, which GPR uses to create the Customer Profile and outcome datasets.
  • It can join agent and customer data to an interaction dataset on-premises, prior to upload to the GPR Core Platform.
  • It enables you to upload datasets to the cloud-based Feature Engineering pipeline (CFEP), which facilitates computation of aggregated features and complex data joins for more broad-based predictive analysis. The CFEP is enabled by default. See Configure for Feature Engineering for the steps required to use feature engineering and, if you do not want to use the CFEP, how to disable it.
  • It anonymizes sensitive data and personally identifiable information (PII). For more information, see Data anonymization.

Deploying Data Loader has the following main phases:

Environment Assumptions

  • You must deploy Data Loader on a host that has no other instances of Local Control Agent installed.
  • The instructions in this section assume that you are creating new Application objects under the Environment folder, in either a single-tenant or multitenant configuration environment.

Download and prepare the Data Loader IP files

The Data Loader IP contains the following items:

  • The gpr folder, which contains subfolder, ai-data-loader, that in turn contains the <version_number>.tar.gz file.
  • The ai-data-loader-scripts.tar.gz zipped archive, which contains initialization scripts Data Loader uses during deployment.

To deploy Data Loader, perform the following steps:

  1. Copy the Data Loader IP files from the Docker Linux IP repository to the desired installation directory.
    • Genesys recommends that you use the gpr home directory as the destination for the Data Loader installation package.
  2. Unzip the ai-data-loader-scripts.tar.gz file. The following files and folders are extracted into your installation directory:
    • docker-compose Linux executable file
    • The /scripts folder, which contains all the installation and other helper files
    • The /conf folder, which contains the dl.env file. The dl.env file contains the environment variables used to configure Data Loader.
    • The /templates folder, which contains the files used to create the Application Template and Application objects for Data Loader.

Configure the Data Loader Application

  1. Import the Application Template
  2. Create a Data Loader Application Object
  3. Configure the General tab
  4. Configure the Server Info tab
  5. Configure the Start Info tab
  6. Configure the Options tab - The configuration options control many aspects of Data Loader behavior, including the password and URLs for connections to the GPR Core Platform, data schemas and upload parameters, and the ability to control threading and timeouts.
  7. Configure the Tenants tab
  8. Configure the Connections tab
  9. Configure the Security tab

Import the Application Template

Before you can configure an Application object for Data Loader, you must import its Application template. The Application template provides most of the configuration options and their default values. You can use this Application template to create as many Application objects of the same type as you need.

Important
For an explanation of how to use Genesys Administrator Extension (GAX) to import the Application template and to create a new Application object, see the Genesys Administrator Extension Help file. The GAX Help is directly available from Genesys Administrator Extension user interface, and from the link given here.
  1. On the Configuration Manager window in Genesys Administrator Extension, select the Environment > Application Templates folder.
  2. From the File menu, select Import Application Template.
  3. In the Look In box, click the down arrow.
  4. Browse to the IP for Data Loader and open the TEMPLATES folder.
  5. Select the template file for Data Loader; it is called DataLoader_900.apd.
  6. Click Open to open the Properties dialog box for the template.
  7. Update the values as needed.
  8. To save the template and close the Properties dialog box, click OK.

The next step is to configure a Data Loader Application object.

Create a Data Loader Application object

After you import the Application template, you can create and configure an Application object for Data Loader by using the Configuration Manager tab in Genesys Administrator.

NOTE: When using a primary-backup high-availability architecture (recommended for all production environments), configure each Data Loader instance in a primary-backup pair with the same configuration option values. If there is a discrepancy between the primary and backup instances, Data Loader uses the values set for the backup instance.

  1. On the Configuration Manager window, select the Environment > Applications folder.
  2. From the File menu, select New > Application.
  3. From the available application templates in the Browse dialog box, select the template that you imported for Data Loader.
  4. On each of the Application tabs, enter the settings appropriate for your environment, as explained in detail in the section that follows.
  5. Click OK to save your changes, then close the Properties dialog box.

Configure the General tab

In the Properties dialog box, click the General tab, and then enter a name for this application.

Configure the Server Info tab

Click the Server Info tab, and then specify the following properties:

  • Host—Select the host for the Docker container containing Data Loader and Local Control Agent (LCA).
    The LCA port is set to 4999 in the Host object.
  • Port—Enter the value 0 (zero) for the port number.
Important
In IPv6 deployments, you cannot set the IP address of the host—only IPv4 addresses can be set for the host. Therefore, enter the name of the host instead.

Configure the Start Info tab

Click the Start Info tab, and then specify the following properties:

  • Working Directory—/dl
  • Command Line—./DataLoader.sh
  • Command Line Arguments—Add the following startup arguments, which are used in the startup script:
    -host <configuration_server_hostname> -port <configuration_server_port> -app <DataLoader_Application_object_name>

Configure the Options tab

Click the Options tab, and then specify or change the values of the configuration options, as suitable for your deployment. Among other configuration, you must configure options to enable the following functionality:

  • The automatic upload of Genesys Info Mart data to Datasets and the Agent Profile. See Configure Data Loader to upload data for specific guidelines.
  • Data Loader access to the GPR Core Services platform. To configure this:
    1. Enter the account information you receive when you start using Predictive Routing in the password and platform-username options.
      • After you set up the initial account access, these options are hidden and Data Loader automatically updates the password every 90 days. Log messages appear in Solution Control Interface indicating when the password is about to expire and when it has been successfully updated.
      • Data Loader sets passwords that comply with Genesys Multicloud CX security standards.
    2. Configure the URL used to access the GPR Core Platform, set in the platform-base-url option.
    3. Configure the API key used to access the GPR Core Platform, set in the platform-api-key option. This API key is available in the Settings > Account window of the GPR web application.
    You must specify the same user name, password, API key, and URL in the corresponding options on the Predictive_Route_DataCfg Transaction List object so the URS Strategy Subroutines component can access the platform.
  • For information about the entire set of Data Loader configuration options, see Data Loader options.

Log Options

  • Configure both Data Loader-specific log options and common log options in the log-related configuration sections. For option descriptions, see Log Options and the Framework Configuration Options Reference Manual.
    Important
    The path to the log file is correctly set by default and should not be edited. Genesys recommends that you retain the default log file name as well.

Configure the Tenants tab

Tip
The Tenants tab is displayed only in a multitenant environment.
  • Click the Tenants tab, and then click Add to add all tenants that this Data Loader application will serve. Data Loader only monitors objects that are associated with the tenants you specify.
  • If this Data Loader instance is required to monitor the objects that are configured under the Environment folder, assign the Environment tenant among the other tenants.

Configure the Connections tab

Click the Connections tab, and then add the following connections:

  • Configuration Server (connection to Configuration Server Proxy is not supported)
  • The Database Access Point (DAP) Application for the Genesys Info Mart Database
  • Message Server

High Availability

In high availability (HA) environments using primary and backup pairs of servers, the servers listed on the Connections tab are handled as primary. To specify the backup servers for any primary servers, open the Application object for the primary server and add the backup server on the primary server’s Server Info tab.

Configure the Security tab

To configure the Security tab settings:

  1. Click Permissions.
  2. Double-click SYSTEM.
  3. Verify that the following are selected for the Change (H) option: Read, Change, Execute, and Read & Execute.
  4. Click OK.
  5. Verify that the Log On As parameter is configured as the SYSTEM account.
  6. Click OK.

When your environment is configured for anonymization on-premises, assign Read, Change, Execute, and Read & Execute permissions to the following configuration objects.

  • The Data Loader Application object.
  • The Predictive_Route_DataCfg Transaction List object, which is used by the GPR URS Strategy Subroutines.
  • All levels of the folder/directory structure within which the Data Loader Application object and the Predictive_Route_DataCfg Transaction List object exist.

This ensures that Data Loader can create and read the options specific to anonymization in the Data Loader Application object and the Predictive_Route_DataCfg Transaction List object.

Install Data Loader

Data Loader is deployed in a Docker container. The container also provides Local Control Agent (LCA), which communicates with Solution Control Server, enabling real-time monitoring of the status of the Data Loader container in Solution Control Interface.

Refer to the following sections of this Guide to make sure you have set up your environment correctly before starting to install Data Loader:

Preliminary Steps: Install and Start Docker

Important
  • Genesys does not include Docker as a part of Data Loader. You must install Docker in your environment before you can load the Data Loader containers. See System Requirements and Required Components/Versions for supported Docker versions.
  • You might need an active Internet connection to download additional libraries when installing Docker.
  1. Install Docker-ee or Docker-ce. Click the desired version to access the relevant deployment instructions on the Docker site.
  2. During Docker deployment, create a Linux group with the name gpr and the group ID of 1010. You will use this group for Data Loader.
  3. During Docker deployment, create a user with the name gpr and the user ID of 1010.
  4. Run the following commands to add the gpr user to the gpr, docker, and systemd-journal groups: 
    $ sudo groupadd -g 1010 gpr
$ sudo useradd -u 1010 gpr
$ sudo usermod -aG gpr gpr
$ sudo usermod -aG docker gpr
$ sudo usermod -aG systemd-journal gpr
$ sudo passwd gpr
    • You must have sudo rights to execute these commands.
  5. Grant SSH access to gpr so that you can copy the Data Loader installation package to the target server. The Data Loader installation package should always be copied to the target server by gpr into the gpr home directory.
  6. To enable the Docker service, execute the following command: 
    $ sudo systemctl enable docker
  7. To start the Docker service, execute the following command: 
    $ sudo service docker start

Install Data Loader and Start the Container

  1. Perform the installation using the gpr user. The gpr user must have sudo rights to perform the installation. Do NOT use the root user to install Data Loader.
  2. Verify that you have created the Data Loader user and assigned it the SERVICE role before starting Data Loader for the first time. For instructions, see Create the Data Loader user in the Predictive Routing Help.
  3. Execute the install.sh script, using the following command from the folder containing the Data Loader IP package: 
    $ cd ./ai-data-loader-scripts/scripts
    $ bash install.sh
    The installation script verifies the following:
    • The target server has sufficient hardware resources. If not, the installation process stops without performing the install.
    • The installed Docker package is docker-ce or docker-ee.
    • The Docker version is supported.
    • The number of cores in the server is sufficient.
    • The root partition free space is sufficient.
    • gpr belongs to the Docker group.
    • gpr belongs to the systemd-journal group (for container log checking).
    • The Docker service is running.
  4. In release 9.0.018.01 and higher, the install.sh process asks you if you would like to start Data Loader when the container starts. If you reply Y/y to the prompt, the script asks you to specify values for the following environment variables:
    • CONFIG_HOST - The host name where Configuration Server is running.
    • CONFIG_PORT - The port number where Configuration Server is running.
    • CONFIG_DL_APP_NAME - The Configuration Server Application object name.
    In release 9.0.017.01 and lower, or if you reply N/n to the prompt, you must start Data Loader using Solution Control Interface after starting the container.
  5. (Optional) To disable the target server verification checks, execute install.sh with the -s parameter:

$ bash install.sh -s

Mapping Between Folders on the Host and in the Container

The following list shows the full path to the various folders on the host machine where the Data Loader container is deployed and the corresponding Docker container folders:

Host Folder Path Docker Container Folder Path
<IP folder>/ai-data-loader-scripts/scripts/datasets_agents /datasets/agents/
<IP folder>/ai-data-loader-scripts/scripts/datasets_customers /datasets/customers/
<IP folder>/ai-data-loader-scripts/scripts/datasets_outcomes /datasets/outcomes/
<IP folder>/ai-data-loader-scripts/scripts/log_dl /log/dl/
<IP folder>/ai-data-loader-scripts/scripts/log_lca /log/lca

See Start and Stop Data Loader for how to start, stop, and restart the Data Loader container and Data Loader itself.

Configure HTTPS proxy

If HTTPS proxy is configured in your environment, configure the JVMPARAMS environment variable, as follows:

  1. Open the <DL installation folder>/ai-data-loader-scripts/conf/dl.env file.
  2. Add the following parameters to the value of the JVMPARAMS environment variable:
    -Dhttps.proxyHost=<proxy_host_name> -Dhttps.proxyPort=<proxy_port>
  3. Save the dl.env file.

Ports for the Data Loader Docker container

Data Loader uses the default ports shown in the following table, which also provides links to the procedures for changing those port numbers, if your environment requires it.

Source Client Server Protocol Direction Port Number Comments
GPR Data Loader Genesys Info Mart DAP TCP To For Microsoft SQL Server, the default communication port is 1433; for Oracle, it is 1521; for PostgreSQL, it is 5432. The database access point (DAP) for the Genesys Info Mart database. For more information about configuring this DAP, see Configuring Required DAPs in the Genesys Info Mart Deployment Guide.
GPR Data Loader Configuration Server TCP To 2020 To change the port number, see Application Parameter Options and Sample Configuration Server Configuration

File on p. 105 of the Framework Configuration Options Reference Manual.

GPR Data Loader Message Server TCP To 2030 To change the port number, see Configuring Client-Side Port Definition on p. 53 of the Framework Configuration Options Reference Manual.
Local Control Agent (LCA) Solution Control Server TCP To 4999 This is the instance of LCA deployed in the Docker container with Data Loader. This port number is not configurable.
GPR Data Loader Local Control Agent (LCA) TCP To 4999 This port number is not configurable.
GPR Data Loader GPR Core Platform HTTPS To 443 This port number is not configurable.
Prometheus GPR Data Loader HTTP To 9105 Prometheus is an optional third-party application which you can use to monitor health and metrics in your environment. To change the port number, see the instructions following this table.

The following steps enable you to change the default port for viewing the Data Loader Spring Boot application metrics, using Prometheus.

  1. Stop the Data Loader application by running the following command: 
    $ bash stop.sh
  2. Edit the scripts/docker-compose.yml file:
    1. Locate the ports label.
    2. Replace the listening port number with the new port number.
    For example, to replace the default Prometheus port, 9105, with port 9090, replace ports: "9105:9105" with ports: "9090:9105".
  3. Restart Data Loader using the following command: 
    $ bash restart.sh

Clean up unnecessary Docker data

Use the following commands to remove unnecessary Docker data: 

$ docker container prune -f
$ docker volume prune -f
$ docker network prune -f

To schedule regular cleanup jobs, use the crontab functionality to execute the appropriate command on every server where GPR is installed. The following example schedules the cleanup job for every Saturday at 1:00 am: 

$ echo "0 1 * * Sat (docker container prune -f; docker volume prune -f; docker network prune -f)" )
Retrieved from "https://all.docs.genesys.com/PE-GPR/9.0.0/Deployment/cfgAsc (2024-04-25 16:12:39)"
Comments or questions about this documentation? Contact us for support!