Difference between revisions of "PEC-REP/Current/GIMPEGuide/ConfigureGSP"

From Genesys Documentation
Jump to: navigation, search
m (Text replacement - "Genesys Engage [cC]loud" to "Genesys Multicloud CX")
Line 5: Line 5:
 
|ComingSoon=No
 
|ComingSoon=No
 
|Section={{Section
 
|Section={{Section
|sectionHeading=Create an Object Bucket Claim
 
|anchor=S3
 
|alignment=Vertical
 
|structuredtext=To enable storage of data during GSP processing, create an S3 Object Bucket Claim (OBC) if none exists.
 
 
See the '''gsp-obc.yaml''' file:
 
<source lang="bash">apiVersion: objectbucket.io/v1alpha1
 
kind: ObjectBucketClaim
 
metadata:
 
  name: gim
 
  namespace: gsp
 
spec:
 
  generateBucketName: gim
 
  storageClassName: openshift-storage.noobaa.io</source>
 
 
Then execute the command to create the OBC:
 
 
<source lang="text">oc create -f gsp-obc.yaml -n gsp</source>
 
 
The following Kubernetes resources are created automatically:
 
 
*An ObjectBucket (OB), which contains the bucket endpoint information, a reference to the OBC, and a reference to the storage class.
 
*A ConfigMap in the same namespace as the OBC, which contains the endpoint to which applications connect in order to consume the object interface
 
*A Secret in the same namespace as the OBC, which contains the key-pairs needed to access the bucket.
 
 
Note the following:
 
 
*The name of the secret and the configMap are the same as the OBC name.
 
*The bucket name is created with a randomized suffix.
 
 
{{AnchorDiv|S3Data}}
 
===Get S3 data===
 
You need to know details of your S3 object to populate Helm chart override values for GSP and GCA.
 
 
To get the OBC data, execute the following command, where <tt>gim</tt> is the name of the configMap associated with the OBC:
 
<source lang="text">oc get cm gim -n gsp -o yaml -o jsonpath={.data}</source>
 
 
The result shows data such as BUCKET_HOST, BUCKET_NAME, BUCKET_PORT, and so on.
 
 
Execute the following commands to get the values of the keys you require for access, where <tt>gim</tt> is the name of the secret associated with the OBC:
 
 
*To get the value of the access key:
 
*:<source lang="text">oc get secret gim -n gsp -o yaml -o jsonpath={.data.AWS_ACCESS_KEY_ID} | base64 --decode</source>
 
*To get the value of the secret key:
 
*:<source lang="text">oc get secret gim -n gsp -o yaml -o jsonpath={.data.AWS_SECRET_ACCESS_KEY} | base64 --decode</source>
 
 
Use the S3 data to populate the Helm chart override values for GSP and GCA.
 
 
{{NoteFormat|You can also obtain the S3 data from the OpenShift console: Go to the '''Object bucket claims''' section under the '''Storage''' menu, and click on the required OBC resource. The data will be at the bottom of the page.|2}}
 
|Status=No
 
}}{{Section
 
 
|sectionHeading=Override Helm chart values
 
|sectionHeading=Override Helm chart values
 +
|anchor=OverrideValues
 
|alignment=Vertical
 
|alignment=Vertical
 
|structuredtext=Download the GSP Helm charts from JFrog using your credentials. You must override certain parameters in the GSP '''values.yaml''' file to provide deployment-specific values for certain parameters.
 
|structuredtext=Download the GSP Helm charts from JFrog using your credentials. You must override certain parameters in the GSP '''values.yaml''' file to provide deployment-specific values for certain parameters.
Line 63: Line 13:
  
 
If you want to use arbitrary UIDs in your OpenShift deployment, you must override the '''securityContext''' settings in the GSP '''values.yaml''' file, so that no user or group IDs are specified. For details, see [[{{FULLPAGENAME}}#Security|Configure security]], below.
 
If you want to use arbitrary UIDs in your OpenShift deployment, you must override the '''securityContext''' settings in the GSP '''values.yaml''' file, so that no user or group IDs are specified. For details, see [[{{FULLPAGENAME}}#Security|Configure security]], below.
 +
 +
To enable S3-compatible storage to store data that GSP requires during processing, see [[{{FULLPAGENAME}}#Storage|Configure S3-compatible storage]], below.
 +
<!--
 +
{{Editgrn_open}}You can also specify values in the '''values.yaml''' to override the default values of configuration options that control GSP behavior and to customize user data and certain mappings. For more information, see [[{{FULLPAGENAME}}#Options|Configure GSP behavior]].{{Editgrn_close}}-->
  
 
At a minimum, you must override the following key entries in the GSP '''values.yaml''' file:
 
At a minimum, you must override the following key entries in the GSP '''values.yaml''' file:
  
 
*<tt>image:</tt>
 
*<tt>image:</tt>
*:<tt>registry</tt> - the registry from which Kubernetes will pull images (<tt>pureengage-docker-staging.jfrog.io</tt> by default)
+
*:<tt>registry</tt> — ''the registry from which Kubernetes will pull images (''<tt>pureengage-docker-staging.jfrog.io</tt> ''by default)''
*:<tt>tag</tt> - the container image version
+
*:<tt>tag</tt> — ''the container image version''
 
*<tt>imagePullSecrets:</tt>
 
*<tt>imagePullSecrets:</tt>
*:<tt>jfrog-stage-credentials</tt> - the secret from which Kubernetes will get credentials to pull the image from the registry
+
*:<tt>pureengage-docker-dev</tt> or <tt>pureengage-docker-staging</tt> — ''the secret from which Kubernetes will get credentials to pull the image from the registry'' <!--{{Editgrn_open}}<font color=red>'''Writer's note:''' The PAT team instructions show jfrog-stage-credentials but that parameter is not in the values.yaml?</font>{{Editgrn_close}}-->
 
*<tt>kafka:</tt>
 
*<tt>kafka:</tt>
*:<tt>bootstrap</tt> - the Kafka address to align with the infrastructure Kafka
+
*:<tt>bootstrap</tt> — ''the Kafka address to align with the infrastructure Kafka''
*<tt>storage:</tt>
+
<!--{{Editgrn_open}}<font color=red>'''Writer's note:''' Should we add a note that the storage:pvc parameter is not used?</font>{{Editgrn_close}}-->
*:<tt>gspPrefix</tt> - the s3 bucket name
 
*<tt>s3</tt> - the applicable s3 details defined with the OBC (see {{Link-SomewhereInThisVersion|manual=GIMPEGuide|topic=ConfigureGSP|anchor=S3Data|display text=Get S3 data}})
 
{{AnchorDiv|YAML-file}}
 
====The GSP '''values.yaml''' file====
 
The following sample GSP '''values.yaml''' file, which may not be completely up to date, shows the key parameter values you must override.
 
  
<source lang="bash">global:
+
If topic names in your Kafka configuration have been customized, you must also modify the <tt>kafka:topic</tt> parameter values to match. For more details about the required Kafka topics, see {{Link-SomewhereInThisVersion|manual=GIMPEGuide|topic=PlanningGSP|anchor=Kafka|display text=Kafka configuration}}.
  rbac:
+
 
    create: true
+
{{NoteFormat|Treat your modified '''values.yaml''' file as source code, which you are responsible to maintain so that your overrides are preserved and available for reuse when you upgrade.}}
  serviceAccount:
 
    create: true
 
image:
 
  registry: pureengage-docker-staging.jfrog.io
 
  repository: gim/gsp
 
  pullPolicy: IfNotPresent
 
  tag: <image-version>
 
imagePullSecrets:
 
  pureengage-docker-dev: {}
 
  pureengage-docker-staging: {}
 
  jfrog-stage-credentials: {}
 
azure:
 
  enabled: false
 
environment: dev
 
location: eastus2
 
job:
 
  rbac:
 
    create: null
 
  serviceAccount:
 
    create: true
 
    name: gsp
 
  id: '00000000000000000000000000000000'
 
  className: com.genesyslab.gim.fsp.App
 
  savepoint: ''
 
  checkpointing:
 
    mode: AT_LEAST_ONCE
 
    interval: 20 min
 
    timeout: 40 min
 
    minPause: 15 min
 
    unaligned: 'false'
 
    concurrent: '1'
 
    external: ''
 
    tolerableFailed: '300'
 
  parallelism: '2'
 
  autoCreateTopics:
 
    partitions: 1
 
    replicationFactor: 3
 
  dumps: /var/lib/dumps
 
  timeDeviation: PT15S
 
  idleness: PT15M
 
  objectReuse: 'true'
 
  kafkaRateLimit: null
 
  storage:
 
    host: gspstate{{.Values.short_location}}{{.Values.environment}}.blob.core.windows.net
 
    #gspPrefix: wasbs://gsp-state@{{ tpl .Values.job.storage.host . }}/{{ .Release.Name }}/
 
    gspPrefix: "s3p://<bucket-name>/{{ .Release.Name }}/"
 
    #gcaSnapshots: wasbs://gca@{{ tpl .Values.job.storage.host . }}/
 
    gcaSnapshots: "s3p://<bucket-name>/gca/"
 
    checkpoints: '{{ tpl .Values.job.storage.gspPrefix . }}checkpoints'
 
    savepoints: '{{ tpl .Values.job.storage.gspPrefix . }}savepoints'
 
    highAvailability: '{{ tpl .Values.job.storage.gspPrefix . }}ha'
 
    s3:
 
      endpoint: "https://<bucket-host>:<bucket-port>"
 
      accessKey: "<access-key-value>"
 
      secretKey: "<secret-key-value>"
 
      pathStyleAccess: "true"
 
#    pvc:
 
#      create: true
 
#      mountPath: /opt/flink/state
 
#      claim: ''
 
#      claimSize: 10Gi
 
#      storageClass: standard
 
  log:
 
    level: INFO
 
    loggers:
 
      org.apache.kafka: INFO
 
  highAvailability:
 
    high-availability: org.apache.flink.kubernetes.highavailability.KubernetesHaServicesFactory
 
    high-availability.jobmanager.port: '50010'
 
    kubernetes.namespace: '{{ .Release.Namespace }}'
 
    kubernetes.cluster-id: '{{ .Release.Name }}'
 
monitoring:
 
  enabled: true
 
  port: 9249
 
  dashboards:
 
    targetDirectory: /var/lib/grafana/dashboards/{{ .Release.Namespace }}
 
tm:
 
  nameOverride: ''
 
  fullnameOverride: ''
 
  numberOfTaskSlots: '2'
 
  deployment:
 
    replicaCount: 1
 
  port:
 
    rpc: 6122
 
  memory:
 
    jvmOverheadFraction: 0.18
 
    jvmOverheadMin: 220mb
 
    jvmOverheadMax: ''
 
    jvmMetaspace: 256mb
 
    offHeap: 128mb
 
    managed: ''
 
    heap: ''
 
    networkMax: ''
 
  resources:
 
    requests:
 
      memory: 1Gi
 
      cpu: '0.05'
 
    limits:
 
      memory: 3Gi
 
      cpu: '2'
 
  tolerations: []
 
  affinity: {}
 
jm:
 
  nameOverride: ''
 
  fullnameOverride: ''
 
  savepoints: ''
 
  port:
 
    rpc: 6123
 
    blob: 6124
 
    rest: 8081
 
  resources:
 
    requests:
 
      memory: 1Gi
 
      cpu: '0.05'
 
    limits:
 
      memory: 2048Mi
 
      cpu: '1'
 
monitor:
 
  rbac:
 
    create: null
 
  serviceAccount:
 
    create: true
 
    annotations: {}
 
    name: '{{ .Release.Name }}-monitor'
 
podSecurityContext: {}
 
securityContext: {}
 
service:
 
  type: ClusterIP
 
  port: 80
 
ingress:
 
  enabled: false
 
  annotations: {}
 
  hosts: []
 
  tls: []
 
kafka:
 
  bootstrap: 'infra-kafka-cp-kafka.infra.svc.cluster.local:9092'
 
  groupId: null
 
  clientId: gim-gsp
 
  offsets: GROUP_OFFSETS
 
  topic:
 
    out:
 
      interactions: gsp-ixn
 
      agentStates: gsp-sm
 
      outbound: gsp-outbound
 
      custom: gsp-custom
 
      cfg: gsp-cfg
 
    in:
 
      digitalItx: digital-itx
 
      digitalAgentStates: digital-agentstate
 
  maxRequestSize: '4194304'
 
  compressionType: lz4
 
  maxBlockMs: '322000'
 
  metadataMaxAgeMs: 600000
 
  metadataMaxIdleMs: 600000
 
  requestTimeoutMs: 32000
 
schemaRegistry:
 
  enabled: false
 
  url: ''
 
  user: ''
 
  password: ''
 
dnsConfig:
 
  options:
 
  - name: ndots
 
    value: '3'</source>
 
 
|Status=No
 
|Status=No
 
}}{{Section
 
}}{{Section
 
|sectionHeading=Configure Kubernetes
 
|sectionHeading=Configure Kubernetes
 
|alignment=Vertical
 
|alignment=Vertical
|structuredtext={{Notices|Notice=PEComingSoon}}
+
|structuredtext=
 +
{{AnchorDiv|Secrets}}
 +
===Secrets===
 +
GSP requires the following secrets:
 +
* <tt>docker-registry</tt> — Credentials to pull the image from the JFrog repository
 +
* <tt>kafka-secrets</tt> — Credentials to access Kafka
 +
* <tt>gsp-s3</tt> — Credentials to access S3-compatible storage
 +
 
 +
Except for <tt>docker-registry</tt> and <tt>kafka-secrets</tt>, which you must create manually (see the environment setup instructions on {{Link-SomewhereInThisVersion|manual=GIMPEGuide|topic=DeployGSP}}), Helm creates the secrets based on values you specify in the '''values.yaml''' file.
 +
 
 +
{{AnchorDiv|ConfigMaps}}
 +
===Config Maps===
 +
Helm creates a number of Config Maps based on option values you specify in the '''values.yaml''' file<!-- (see [[{{FULLPAGENAME}}#Options|Configure GSP behavior]])-->. There are no Config Maps you can configure directly.
 
|Status=No
 
|Status=No
 
}}{{Section
 
}}{{Section
Line 282: Line 79:
 
</source>
 
</source>
 
|Status=No
 
|Status=No
}}
+
}}{{Section
 +
|sectionHeading=Configure S3-compatible storage
 +
|anchor=Storage
 +
|alignment=Vertical
 +
|structuredtext=<!--{{Editgrn_open}}If you are not using the default Azure Blob Storage{{Editgrn_close}}-->To enable S3-compatible storage, modify the following entries in the '''values.yaml''' file:
 +
* <tt>azure:</tt>
 +
*: <tt>enabled:</tt> false
 +
* <tt>storage:</tt>
 +
*: <tt>gspPrefix</tt> — ''the bucket name''
 +
*: <tt>gcaSnapshots</tt> — ''the bucket name where the GCA snapshot is stored''
 +
*: <tt>s3</tt> — ''the applicable details defined with the OBC or GCP bucket''
 +
*:'''Note:''' The <tt>host</tt> parameter is ignored.
 +
 
 +
====OpenShift example====
 +
<source lang="bash">azure:
 +
  enabled: false
 +
..
 +
storage:
 +
    host: gspstate{{.Values.short_location}}{{.Values.environment}}.blob.core.windows.net
 +
    #gspPrefix: wasbs://gsp-state@{{ tpl .Values.job.storage.host . }}/{{ .Release.Name }}/
 +
    gspPrefix: "s3p://gim-3f7ac1ab-03b9-445b-ba12-137d4bbc3c38/{{ .Release.Name }}/"                                                                             
 +
    #gcaSnapshots: wasbs://gca@{{ tpl .Values.job.storage.host . }}/
 +
    gcaSnapshots: "s3p://gim-3f7ac1ab-03b9-445b-ba12-137d4bbc3c38/gca/"                                                                 
 +
    checkpoints: '{{ tpl .Values.job.storage.gspPrefix . }}checkpoints'
 +
    savepoints: '{{ tpl .Values.job.storage.gspPrefix . }}savepoints'
 +
    highAvailability: '{{ tpl .Values.job.storage.gspPrefix . }}ha'
 +
    s3:
 +
      endpoint: "https://s3.openshift-storage.svc:443"
 +
      accessKey: "<access key>"
 +
      secretKey: "<secret key>"
 +
      pathStyleAccess: "true"</source>
 +
 
 +
====GKE example====
 +
<source lang="bash">azure:
 +
  enabled: false
 +
...
 +
storage:
 +
    host: gspstate{{.Values.short_location}}{{.Values.environment}}.blob.core.windows.net
 +
    #gspPrefix: wasbs://gsp-state@{{ tpl .Values.job.storage.host . }}/{{ .Release.Name }}/
 +
    gspPrefix: "s3p://test-example-bucket-one/{{ .Release.Name }}/"                                                                             
 +
    #gcaSnapshots: wasbs://gca@{{ tpl .Values.job.storage.host . }}/
 +
    gcaSnapshots: "s3p://test-example-bucket-one/gca/"                                                                 
 +
    checkpoints: '{{ tpl .Values.job.storage.gspPrefix . }}checkpoints'
 +
    savepoints: '{{ tpl .Values.job.storage.gspPrefix . }}savepoints'
 +
    highAvailability: '{{ tpl .Values.job.storage.gspPrefix . }}ha'
 +
    s3:
 +
      endpoint: "https://storage.googleapis.com:443"
 +
      accessKey: "<access Key>"
 +
      secretKey: "<secret key>"
 +
      pathStyleAccess: "true"</source>
 +
|Status=No
 +
}}<!--{{Section
 +
|sectionHeading=Configure GSP behavior
 +
|anchor=Options
 +
|alignment=Vertical
 +
|structuredtext=
 +
|Status=No
 +
}}-->
 
|PEPageType=9c3ae89b-4f75-495b-85f8-d8c4afcb3f97
 
|PEPageType=9c3ae89b-4f75-495b-85f8-d8c4afcb3f97
 
}}
 
}}

Revision as of 19:53, March 30, 2022

This topic is part of the manual Genesys Info Mart Private Edition Guide for version Current of Reporting.

Learn how to configure GIM Stream Processor (GSP).

Override Helm chart values

Download the GSP Helm charts from JFrog using your credentials. You must override certain parameters in the GSP values.yaml file to provide deployment-specific values for certain parameters.

For general information about overriding Helm chart values, see Overriding Helm chart values in the Genesys Multicloud CX Private Edition Guide.

If you want to use arbitrary UIDs in your OpenShift deployment, you must override the securityContext settings in the GSP values.yaml file, so that no user or group IDs are specified. For details, see Configure security, below.

To enable S3-compatible storage to store data that GSP requires during processing, see Configure S3-compatible storage, below.

At a minimum, you must override the following key entries in the GSP values.yaml file:

  • image:
    registrythe registry from which Kubernetes will pull images (pureengage-docker-staging.jfrog.io by default)
    tagthe container image version
  • imagePullSecrets:
    pureengage-docker-dev or pureengage-docker-stagingthe secret from which Kubernetes will get credentials to pull the image from the registry
  • kafka:
    bootstrapthe Kafka address to align with the infrastructure Kafka

If topic names in your Kafka configuration have been customized, you must also modify the kafka:topic parameter values to match. For more details about the required Kafka topics, see Kafka configuration.

Important
Treat your modified values.yaml file as source code, which you are responsible to maintain so that your overrides are preserved and available for reuse when you upgrade.

Configure Kubernetes

Secrets

GSP requires the following secrets:

  • docker-registry — Credentials to pull the image from the JFrog repository
  • kafka-secrets — Credentials to access Kafka
  • gsp-s3 — Credentials to access S3-compatible storage

Except for docker-registry and kafka-secrets, which you must create manually (see the environment setup instructions on Deploy GIM Stream Processor), Helm creates the secrets based on values you specify in the values.yaml file.

Config Maps

Helm creates a number of Config Maps based on option values you specify in the values.yaml file. There are no Config Maps you can configure directly.

Configure security

The security context settings define the privilege and access control settings for pods and containers.

By default, the user and group IDs are set in the GSP values.yaml file as 500:500:500, meaning the genesys user.

securityContext:
  runAsNonRoot: true
  runAsUser: 500
  runAsGroup: 500
  fsGroup: 500

containerSecurityContext: {}

Arbitrary UIDs in OpenShift

If you want to use arbitrary UIDs in your OpenShift deployment, you must override the securityContext settings in the GSP values.yaml file, so that you do not define any specific IDs.

securityContext:
  runAsNonRoot: true
  runAsUser: null
  runAsGroup: 0
  fsGroup: null

containerSecurityContext: {}

Configure S3-compatible storage

To enable S3-compatible storage, modify the following entries in the values.yaml file:

  • azure:
    enabled: false
  • storage:
    gspPrefixthe bucket name
    gcaSnapshotsthe bucket name where the GCA snapshot is stored
    s3the applicable details defined with the OBC or GCP bucket
    Note: The host parameter is ignored.

OpenShift example

azure:
  enabled: false
..
storage:
    host: gspstate{{.Values.short_location}}{{.Values.environment}}.blob.core.windows.net
    #gspPrefix: wasbs://gsp-state@{{ tpl .Values.job.storage.host . }}/{{ .Release.Name }}/
    gspPrefix: "s3p://gim-3f7ac1ab-03b9-445b-ba12-137d4bbc3c38/{{ .Release.Name }}/"                                                                               
    #gcaSnapshots: wasbs://gca@{{ tpl .Values.job.storage.host . }}/
    gcaSnapshots: "s3p://gim-3f7ac1ab-03b9-445b-ba12-137d4bbc3c38/gca/"                                                                  
    checkpoints: '{{ tpl .Values.job.storage.gspPrefix . }}checkpoints'
    savepoints: '{{ tpl .Values.job.storage.gspPrefix . }}savepoints'
    highAvailability: '{{ tpl .Values.job.storage.gspPrefix . }}ha'
    s3:
      endpoint: "https://s3.openshift-storage.svc:443"
      accessKey: "<access key>"
      secretKey: "<secret key>"
      pathStyleAccess: "true"

GKE example

azure:
  enabled: false
...
storage:
    host: gspstate{{.Values.short_location}}{{.Values.environment}}.blob.core.windows.net
    #gspPrefix: wasbs://gsp-state@{{ tpl .Values.job.storage.host . }}/{{ .Release.Name }}/
    gspPrefix: "s3p://test-example-bucket-one/{{ .Release.Name }}/"                                                                               
    #gcaSnapshots: wasbs://gca@{{ tpl .Values.job.storage.host . }}/
    gcaSnapshots: "s3p://test-example-bucket-one/gca/"                                                                  
    checkpoints: '{{ tpl .Values.job.storage.gspPrefix . }}checkpoints'
    savepoints: '{{ tpl .Values.job.storage.gspPrefix . }}savepoints'
    highAvailability: '{{ tpl .Values.job.storage.gspPrefix . }}ha'
    s3:
      endpoint: "https://storage.googleapis.com:443"
      accessKey: "<access Key>"
      secretKey: "<secret key>"
      pathStyleAccess: "true"
Comments or questions about this documentation? Contact us for support!