How do I scale and size Workflow Server pods on CP4BA (replicas, JVM heap, database connections)?

Question

On WebSphere we tuned the cluster members' heap and the JDBC pools. On the Cloud Pak the workflow pods run out of memory under load. Where are these settings and how does scaling work?

Answer 1

All of it is in the workflow section of the custom resource; the operator applies it and rolls the pods:

spec:
  baw_configuration:
  - name: bawins1
    replicas: 3                                   # horizontal scale (StatefulSet); session affinity by the route cookie
    resources:
      requests: { cpu: "2", memory: "4Gi" }
      limits:   { cpu: "4", memory: "8Gi" }
    jvm_customize_options: "-Xms2g -Xmx6g -Xmn512m -XX:+UseContainerSupport"   # heap must fit the memory limit (leave ~25% for native)
    database:
      # connection pool of the engine data sources
      cm_max_pool_size: 100
      cm_min_pool_size: 10
    # thread pools and engine caches still come from 100Custom.xml (event manager threads, cache sizes)
  pfs_configuration:
    replicas: 2
    elasticsearch: { replicas: 3, storage: { size: 50Gi } }

Guidelines: one workflow pod handles a load comparable to one WebSphere cluster member with the same heap; scale out before scaling up beyond 8-12 GB heaps (GC pauses); the event manager runs in every replica, so timers / UCAs scale with replicas; the database pool per pod times the replicas must stay under the database's connection limit; keep CPU requests realistic (the operator sizes are conservative). For authoring (Workflow Center) one replica is the norm. Use the starter profile sizes only for trials; the production profile (sc_deployment_profile_size: large) sets sensible defaults for every component that you then override per instance.

Observe: oc adm top pods, the Liberty metrics (heap, thread pool, connection pool via /metrics), and the Operations REST event manager monitor for engine backlog. Horizontal Pod Autoscaling is not supported for the workflow StatefulSet on most releases - scale by changing replicas (a GitOps commit).

References

• CP4BA documentation - Business Automation Workflow configuration parameters (resources, JVM, database)
• CP4BA - deployment profiles and sizing