PEC-REP/Current/GIMPEGuide/ConfigureGSP

Learn how to configure GIM Stream Processor (GSP).

Related documentation:

• How Reporting works
• Suite-level guide
• All services
• Genesys Info Mart Release Notes

RSS:

• For private edition

Override Helm chart values

Download the GSP Helm charts from JFrog using your credentials. You must override certain parameters in the GSP values.yaml file to provide deployment-specific values for certain parameters.

For general information about overriding Helm chart values, see Overriding Helm chart values in the Genesys Multicloud CX Private Edition Guide.

If you want to use arbitrary UIDs in your OpenShift deployment, you must override the securityContext settings in the GSP values.yaml file, so that no user or group IDs are specified. For details, see Configure security, below.

To enable S3-compatible storage to store data that GSP requires during processing, see Configure S3-compatible storage, below.

You can also specify values in the values.yaml file to override the default values of configuration options that control GSP behavior and to customize user data and Outbound field mappings.

At a minimum, you must override the following key entries in the GSP values.yaml file:

• image:

  registry — the registry from which Kubernetes will pull images (pureengage-docker-staging.jfrog.io by default)

  tag — the container image version

• imagePullSecrets:

  pureengage-docker-dev or pureengage-docker-staging — the secret from which Kubernetes will get credentials to pull the image from the registry

• kafka:

  bootstrap — the Kafka address to align with the infrastructure Kafka

If topic names in your Kafka configuration have been customized, you must also modify the kafka:topic parameter values to match. For more details about the required Kafka topics, see Kafka configuration.

Important

Treat your modified values.yaml file as source code, which you are responsible to maintain so that your overrides are preserved and available for reuse when you upgrade.

Configure Kubernetes

Secrets

GSP requires the following secrets:

• docker-registry — Credentials to pull the image from the JFrog repository
• kafka-secrets — Credentials to access Kafka
• gsp-s3 — Credentials to access S3-compatible storage

Except for docker-registry and kafka-secrets, which you must create manually (see the environment setup instructions on Deploy GIM Stream Processor), Helm creates the secrets based on values you specify in the values.yaml file.

Config Maps

Helm creates a number of Config Maps based on option values you specify in the values.yaml file (see Configure GSP behavior). There are no Config Maps you can configure directly.

Kubernetes API requirements

GSP uses Apache Flink as the engine for stateful stream processing, with communications handled via the Kubernetes API.

GSP needs permissions to use the following verbs:

• get, list, watch, and delete on jobs resources in the batch API group. GSP uses these commands during upgrade and for a pre-upgrade hook to ensure that the previous version of GSP is stopped before upgrading to the new version.
• update, get, create, watch, patch, delete, list on configmap resources in the general API group (“”). GSP uses these commands:
  • To enable Flink's Kubernetes high availability (HA) services to function
  • To record the path to the savepoint periodically taken by the take-savepoint cron job and by the upgrade hook

Configure security

The security context settings define the privilege and access control settings for pods and containers.

By default, the user and group IDs are set in the GSP values.yaml file as 500:500:500, meaning the genesys user.

securityContext:
  runAsNonRoot: true
  runAsUser: 500
  runAsGroup: 500
  fsGroup: 500

containerSecurityContext: {}

Arbitrary UIDs in OpenShift

If you want to use arbitrary UIDs in your OpenShift deployment, you must override the securityContext settings in the GSP values.yaml file, so that you do not define any specific IDs.

securityContext:
  runAsNonRoot: true
  runAsUser: null
  runAsGroup: 0
  fsGroup: null

containerSecurityContext: {}

Configure S3-compatible storage

By default, GSP is configured to use Azure Blob Storage as the persistent data store for data used during processing. To use Azure Blob Storage, modify the following entries in the values.yaml file:

• job:

     storage:

        gspPrefix — the URI path prefix under which GSP savepoints, checkpoints, and high-availability data will be stored

        gcaSnapshots — the URI path under which GCA snapshot(s) will be stored

To enable other types of S3-compatible storage, modify the following entries in the values.yaml file:

• azure:

     enabled: false

• job:

     storage:

        gspPrefix — the bucket name where GSP savepoints, checkpoints, and high-availability data will be stored

        gcaSnapshots — the bucket name where the GCA snapshot(s) will be stored

        s3 — the applicable details defined with the OBC or GCP bucket

  Note: The host parameter is ignored.

OpenShift example

azure:
  enabled: false
..
job:
  storage:
    host: gspstate{{.Values.short_location}}{{.Values.environment}}.blob.core.windows.net
    #gspPrefix: wasbs://gsp-state@{{ tpl .Values.job.storage.host . }}/{{ .Release.Name }}/
    gspPrefix: "s3p://gim-3f7ac1ab-03b9-445b-ba12-137d4bbc3c38/{{ .Release.Name }}/"                                                                               
    #gcaSnapshots: wasbs://gca@{{ tpl .Values.job.storage.host . }}/
    gcaSnapshots: "s3p://gim-3f7ac1ab-03b9-445b-ba12-137d4bbc3c38/gca/"                                                                  
    checkpoints: '{{ tpl .Values.job.storage.gspPrefix . }}checkpoints'
    savepoints: '{{ tpl .Values.job.storage.gspPrefix . }}savepoints'
    highAvailability: '{{ tpl .Values.job.storage.gspPrefix . }}ha'
    s3:
      endpoint: "https://s3.openshift-storage.svc:443"
      accessKey: "<access key>"
      secretKey: "<secret key>"
      pathStyleAccess: "true"

GKE example

azure:
  enabled: false
...
job:
  storage:
    host: gspstate{{.Values.short_location}}{{.Values.environment}}.blob.core.windows.net
    #gspPrefix: wasbs://gsp-state@{{ tpl .Values.job.storage.host . }}/{{ .Release.Name }}/
    gspPrefix: "s3p://test-example-bucket-one/{{ .Release.Name }}/"                                                                               
    #gcaSnapshots: wasbs://gca@{{ tpl .Values.job.storage.host . }}/
    gcaSnapshots: "s3p://test-example-bucket-one/gca/"                                                                  
    checkpoints: '{{ tpl .Values.job.storage.gspPrefix . }}checkpoints'
    savepoints: '{{ tpl .Values.job.storage.gspPrefix . }}savepoints'
    highAvailability: '{{ tpl .Values.job.storage.gspPrefix . }}ha'
    s3:
      endpoint: "https://storage.googleapis.com:443"
      accessKey: "<access Key>"
      secretKey: "<secret key>"
      pathStyleAccess: "true"

Configure GSP behavior

Whole section is new

You can override aspects of the default configuration to modify GSP behavior and customize the way data is stored in the Info Mart database.

You can customize the following:

• Configuration options, to modify data-related aspects of GSP behavior
• User data mappings, to map custom key-value pairs (KVPs), which are attached to event and reporting protocol data, to tables and columns in the Info Mart database
• Outbound record field mappings, to map custom CX Contact record field data to Outbound-related tables and columns in the Info Mart database

Customize configuration option settings

For full information about the options you can configure, including the default and valid values, see GSP configuration options.

Alexey/Kostya, I've combined and extrapolated from info in e-mails and Jira tickets. Please esp. confirm the syntax in this subsection and in the extended example at the bottom of the page.

Under the cfgOptions object in the GSP values.yaml file, specify the option and value in JSON format. Note the following:

• Options are separately configurable by tenant and, where applicable, by media type or even at the level of individual queues (DNs or scripts).
• Where an option can be configured at various levels, you can override a value set at a higher level (for example, for a particular media type in general) to set a different value for a particular lower-level object (for example, for that media type for an individual DN).
• See the note about configuration levels for information about the available configuration levels for certain options.

The structure of the entries is:

cfgOptions:
    "<tenant_id>": |
      standard:
        <option 1>: <value>
      media:
        <media type 1>:
          <option 2>: <value 1>
          <option 3>: <value 1>
        <media type 2>:
          <option 2>: <value 2>
          <option 4>: <value 1>
      dn:
        <dn_id>:
          media:
            <media type 1>:
              <option 2>: <value 3>
      script:
        <script_id>:
          media:
            <media type 2>:
              <option 2>: <value 4> 
      ...

For an example, see below.

Reviewers! For the review request relating to GIM-14206, skip the next two sections and jump to the cfgOptions part of the example at the bottom of the page (link). Then please go to the GSP configuration options page.

Customize user data mapping

Genesys Info Mart uses a wide range of user-data KVPs from a number of upstream services to populate data in the Info Mart database. For full information about user data in Genesys Info Mart, see [[PEC-REP/Current/GIMPEGuide/UserData|]].

As described on [link TBD], you can extend storage of user data in the Info Mart database to include additional user-data KVPs you want to capture as custom user-data facts or dimensions.

Under the udeMapping object in the GSP values.yaml file, specify the mapping between your custom KVPs and the custom user-data database table(s) and column(s). Note the following:

• The mapping, which is specified in JSON format, is configured separately by tenant.
• In addition to specifying the database table and column in which the KVP value will be stored, you also specify the propagation rule that Genesys Info Mart will use to determine what value to store if more than one value is extracted for the same key in the same interaction. See Propagation rules for more information.
• You can specify a default value to use if the KVP value is null. You must provide a default value for dimensions.
• You map custom user-data dimensions via the IRF_USER_DATA_KEYS table, where you specify the foreign key reference(s) for the user-data dimension table(s).

Genesys Info Mart provides the following sample tables in the Info Mart database schema for storage of custom KVPs:

• USER_DATA_CUST_DIM_1 — for low-cardinality user data to be stored as dimensions
• IRF_USER_DATA_CUST_1 — for high-cardinality user data to be stored as facts

The structure of the entries to specify in the values.yaml file is:

Questions:

1. Can you customize the table names (USER_DATA_CUST_DIM_1, IRF_USER_DATA_CUST_1) and column names (DIM_ATTRIBUTE_1, CUSTOM_DATA_1, etc.; also CUSTOM_KEY_1 in IRF_USER_DATA_KEYS)? i.e., Is it OK to show these as <variables>?
2. Alexey, the example at the bottom of the page is taken from an example you sent me. Why are these particular IRF_USER_DATA_GEN_1 entries listed? In GIM Classic, we say these particular mappings are predefined by default. Do they need to be overtly specified here, or should we change the example to show non-standard KVPs?

udeMapping:
  "<tenant_id">: |
    IRF_USER_DATA_KEYS:
      columns:
        <CUSTOM_KEY_1>:
          type: dim
          table: <USER_DATA_CUST_DIM_1>
          attributes:
            <DIM_ATTRIBUTE_1>:
              kvp: <KVP>
              rule: <propagation_rule>
              default: <default_value>
            ...
    <IRF_USER_DATA_CUST_1>:
      columns:
        <CUSTOM_DATA_1>:
          type: value
          kvp: <KVP>
          rule: <propagation_rule>
          default: <default_value>
        ...

For an example, see below.

Customize Outbound Contact field mapping

Writer's note: In GIM Classic, we've been religious about using the phrase "Outbound Contact" to refer to OCS data. For PE and Multicloud in general, confirm we should use just "Outbound". And for PE, only CX Contact applies.

Genesys Info Mart stores data about every outbound contact attempt, based on Record Field data it receives from the CX Contact (CXC) service. As described on [[PEC-REP/Current/GIMPEGuide/Outbound|]], some of the mapping between Field data and the Info Mart database tables and columns is predefined, and some is custom.

Under the ocsMapping object in the GSP values.yaml file, specify the mapping between your custom record fields and the tables and columns provided in the Info Mart database for custom record field data, namely:

• In the CONTACT_ATTEMPT_FACT table:
  • 10 floating-point numbers: numeric(14,4)
  • 20 integers: integer
  • 30 strings: varchar(255)
• In the RECORD_FIELD_GROUP_1 and RECORD_FIELD_GROUP_2 tables:
  • 10 strings each: varchar(255)

Placeholder for something to say about rpcValue and conversionValue (see question 2).

Questions:

1. Per the CXC documentation (Create and Manage User-defined Field Labels), by default "all user-defined fields in a contact list are labelled as Other1, Other2, Other3, and so on." In our example below, should I change all instances of "GenRecordField" to "Other"? -- e.g., CAF.RECORD_FIELD_1 populated by field: Other1, RECORD_FIELD_GROUP_1.RECORD_FIELD_1_STRING_1 populated by field: Other1String1, etc.
2. I cannot find any reference to OCS-equivalent right_person and conversion configuration in the CXC documentation. Are rpcValue and conversionValue applicable for PE?

The mapping, which is specified in JSON format, is configured separately by tenant. The structure of the entries is:

ocsMapping:
  "<tenant_id>": |
    CONTACT_ATTEMPT_FACT:
      columns:
        RECORD_FIELD_1:
          field: <numericFieldName>
          rpcValue:
          conversionValue:
        ...
        RECORD_FIELD_11:
          field: <intFieldName>
          rpcValue:
          conversionValue:
        ...
        RECORD_FIELD_31:
          field: <stringFieldName>
          rpcValue:
          conversionValue:
        ...
    RECORD_FIELD_GROUP_1:
      columns:
        RECORD_FIELD_1_STRING_1:
          field: <stringFieldName>
          rpcValue:
          conversionValue:
        ...
    RECORD_FIELD_GROUP_2:
      columns:
        RECORD_FIELD_2_STRING_1:
          field: <stringFieldName>
          rpcValue:
          conversionValue:
        ...

For an example, see below.

Example

cfgOptions:
    "eb0c9f3a-5dca-498f-98d5-7610f5fd1015": |
      standard:
        completed-queues: iWD_Completed,iWD_Processed_ext
        populate-workbin-as-hold: false
      media:
        voice:
          q-answer-threshold: 30
      media:
        email:
          q-short-abandoned-threshold: 20
      dn:
        eb0c9f3a-5dca-498f-98d5-7610f5fd1015_MM_VQ:
          media:
            email:
              q-answer-threshold: 90
              q-short-abandoned-threshold: 40
      script:
        eb0c9f3a-5dca-498f-98d5-7610f5fd1015_ixn_queue:
          media:
            chat:
              q-answer-threshold: 20
      script:
        eb0c9f3a-5dca-498f-98d5-7610f5fd1015_dev.IQ:
          standard:
            populate-ixnqueue-facts: false

udeMapping:
  "eb0c9f3a-5dca-498f-98d5-7610f5fd1015": |
    IRF_USER_DATA_KEYS:
      columns:
        CUSTOM_KEY_1:
          type: dim
          table: USER_DATA_CUST_DIM_1
          attributes:
            DIM_ATTRIBUTE_1:
              kvp: BusinessLine
              rule: CALL
              default: retail
            DIM_ATTRIBUTE_2:
              kvp: RULE_PARTY
              rule: PARTY
              default: none
            DIM_ATTRIBUTE_3:
              kvp: RULE_IRF
              rule: IRF
              default: none
            DIM_ATTRIBUTE_4:
              kvp: RULE_IRF_INITIAL
              rule: IRF_INITIAL
              default: none
            DIM_ATTRIBUTE_5:
              kvp: RULE_IRF_FIRST_UPDATE
              rule: IRF_FIRST_UPDATE
              default: none
    IRF_USER_DATA_CUST_1:
      columns:
        CUSTOM_DATA_1:
          type: value
          kvp: RULE_CALL
          rule: CALL
          default: none
        CUSTOM_DATA_2:
          type: value
          kvp: RULE_PARTY
          rule: PARTY
          default: none
        CUSTOM_DATA_3:
          type: value
          kvp: RULE_IRF
          rule: IRF
        CUSTOM_DATA_4:
          type: value
          kvp: RULE_IRF_FIRST_UPDATE
          rule: IRF_FIRST_UPDATE
        CUSTOM_DATA_5:
          type: value
          kvp: RULE_IRF_INITIAL
          rule: IRF_INITIAL
        CUSTOM_DATA_6:
          type: value
          kvp: RULE_IRF_ROUTE
          rule: IRF_ROUTE
    IRF_USER_DATA_GEN_1:
      columns:
        CASE_ID:
          type: value
          kvp: CaseID
          rule: Call
          default: none
        CUSTOMER_ID:
          type: value
          kvp: GIM_GRP
          rule: Call
          default: none

ocsMapping:
  "eb0c9f3a-5dca-498f-98d5-7610f5fd1015": |
    CONTACT_ATTEMPT_FACT:
      columns:
        RECORD_FIELD_1:
          field: GenRecordField1
          rpcValue:
          conversionValue:
        RECORD_FIELD_2:
          field: GenRecordField2
          rpcValue:
          conversionValue:
        ...
        RECORD_FIELD_60:
          field: GenRecordField60
          rpcValue:
          conversionValue:
    RECORD_FIELD_GROUP_1:
      columns:
        RECORD_FIELD_1_STRING_1:
          field: GenRecordField1String1
          rpcValue:
          conversionValue:
        RECORD_FIELD_1_STRING_2:
          field: GenRecordField1String2
          rpcValue:
          conversionValue:
        RECORD_FIELD_1_STRING_3:
        ...
    RECORD_FIELD_GROUP_2:
      columns:
        RECORD_FIELD_2_STRING_1:
          field: GenRecordField2String1
          rpcValue:
          conversionValue:
        RECORD_FIELD_2_STRING_2:
        ...