Client demo strategy

Open Liberty + NGINX Web Tier POC

Recommended demo app

BRAC Digital Onboarding and Loan Eligibility POC

A compact enterprise application that shows Open Liberty as a production-ready MicroProfile runtime behind an NGINX web tier. The app demonstrates canary routing, operational health, JVM metrics, edge hardening, mesh telemetry, and end-to-end traceability in one business-relevant workflow.

Stable Normal traffic routes to Liberty A
Beta Header: version: beta routes to Liberty B
Live spoke-dc OpenShift
OSSM Ambient platform Healthy; namespace enrollment drafted PR #48
Tempo Traces verified; logs/metrics fanout drafted PR #51

Capability story

Why this app fits the client requirement

Business-relevant workflow

The demo models a digital onboarding or loan eligibility journey instead of a generic hello-world endpoint, keeping the story close to banking operations.

Visible canary behavior

Stable users receive the current scoring response from Instance A. Beta users receive the experimental risk model response from Instance B.

Target architecture

Web tier and Open Liberty runtime

Browser / curl Client demo traffic
NGINX Reverse Proxy Routing, HSTS, rate limits, error pages
Open Liberty A Stable application path
Open Liberty B Beta canary path

OpenShift target platform

How the demo maps to the client runtime

spoke-dc only

All runtime, observability, GitOps, and mesh design targets the spoke-dc cluster only.

NGINX web tier

NGINX remains the L7 control point for header canary routing, hardening headers, rate limiting, and custom error pages.

Namespace

The project namespace is brac-poc-openliberty-v5, matching the repo, docs site, and client demo identity.

Platform observability

MicroProfile health and metrics are exposed through NGINX. User workload monitoring, OpenShift Logging, Tempo, OpenTelemetry, and Perses are available on the target cluster.

Current platform status

Deployed and verified on spoke-dc

Area Current state Impact for the POC
Target cluster spoke-dc API reachable at api.spoke-dc.ocp.comptech-lab.com. This remains the only deployment and observability target.
OpenShift base OpenShift 4.20.18 cluster operators are Available, not Progressing, and not Degraded. The platform is healthy for the current demo slice.
Project namespace brac-poc-openliberty-v5 exists and contains the app runtime. The namespace now hosts Liberty A, Liberty B, NGINX, services, route, and ServiceMonitor.
Application route https://brac-poc-openliberty-v5.apps.spoke-dc.ocp.comptech-lab.com Public demo route for canary, health, metrics, and hardening proof.
Runtime pods liberty-a, liberty-b, and nginx are 1/1 available. The client can see two Liberty instances behind one NGINX reverse proxy.
OpenShift GitOps openshift-gitops running. Two Applications on spoke-dc: labops Synced/Healthy, spoke-dc-cluster-config OutOfSync/Healthy. No brac-poc-openliberty-v5 Application yet. Architectural choice made: Option C — the workload now lives at lab-workloads.git/components/apps/brac-poc-openliberty-v5/ and is activated by clusters/spoke-dc/kustomization.yaml (lab-workloads MR !2 merged 2026-05-07). Liberty instances are OpenLibertyApplication CRs (apps.openliberty.io/v1), reconciled by the Open Liberty Operator v1.6.1 already installed on spoke-dc. The Argo Application yaml in this repo (PR #60 merged) targets path: clusters/spoke-dc; applying it is Phase 5 step 3, gated on Phase 4 (Nexus image publish).
OSSM3 Istio is Healthy with ZTunnel v1.28.5; Kiali and istio-cni pods are running. Mesh visibility and policy are available; namespace enrollment + waypoint + AuthorizationPolicy are drafted in PR #48.
Ambient mode ZTunnel/default is Healthy on spoke-dc with 4 ztunnel pods Running across worker nodes; istio-waypoint GatewayClass is Accepted. Platform side is ready. Namespace enrollment (istio.io/dataplane-mode: ambient) and the waypoint Gateway are drafted in PR #48 and apply via Argo sync.
Gateway API istio, istio-remote, and istio-waypoint GatewayClasses are accepted. The app can later use Gateway API patterns if the owner chooses that route; istio-waypoint is consumed by PR #48.
Observability User workload monitoring, Loki, OpenTelemetry Collector, Tempo, and Perses resources are present. Health, metrics, and Tempo traces are verified for the deployed app.
Trace pipeline The previous collector Kafka bridge exporter issue is resolved for the demo trace path. The active collector path is traces to Tempo. A namespace-scoped fanout collector that preserves Tempo/Loki/UWM and adds Splunk + SigNoZ + Kafka exporters is drafted in PR #51.
External Secrets Operator ESO v1.1.0 healthy; SecretStore/rke2-vault Ready=True. Namespace-scoped SecretStore/brac-poc-vault Ready=True and ExternalSecret/nexus-docker-pull SecretSynced=True (PR #45 applied directly to brac-poc-openliberty-v5 namespace 2026-05-06). Vault → nexus-docker-pull sync verified end-to-end. Same shape reused by PR #51 (observability-export) and PR #56 (runtime-sso) once their Vault KV paths are populated.
Vault state (RKE2) HA active (3 raft nodes), v1.18.2, transit auto-unseal. Auth mount kubernetes-spoke-dc/ enabled with role brac-poc-openliberty-v5 bound to SA eso-vault-auth. Policies brac-poc-openliberty-v5-read and brac-poc-openliberty-v5-jenkins-ci exist. KV paths secret/brac-poc-openliberty-v5/ci/{nexus,source} populated. Phase 2 of the rollout is effectively complete. Outstanding KV writes: runtime/observability-export (PR #51 destinations) and runtime/sso (PR #56 mpJwt) — pending real destination/IDP credentials.
Jenkins build worker RKE2 PVCs jenkins-buildah-cache (30Gi) and jenkins-maven-cache (10Gi) on longhorn are Bound and match the PR #46 drafts. JCasC ConfigMap does not yet declare a Kubernetes clouds: block — pod-agent cloud config is the remaining gap before a release rehearsal. Phase 3 step 1 (PVCs) done. Step 2 (JCasC clouds + smoke run with buildah container) is the next piece of real work before Phase 4 image publish to Nexus.

Ambient mesh and traceability

mTLS, waypoint policy, and end-to-end request correlation

OSSM3 ambient mode

The target design uses ambient mode for the POC namespace. ztunnel provides L4 mTLS, while waypoint proxies provide L7 policy, telemetry, retries, timeouts, and circuit breaking for NGINX-to-Liberty calls.

Trace context

NGINX and Open Liberty must preserve W3C trace headers such as traceparent and tracestate so each demo request can be followed across proxy, mesh, and app logs.

ztunnelL4 service identity and mTLS for ambient workloads.
waypointL7 policy point for retries, timeouts, circuit breaking, and richer telemetry.
NGINXHeader-based canary routing remains in the web tier; ztunnel is not used for L7 header steering.
KialiTopology view for NGINX, Liberty A, and Liberty B traffic flow.
traceparentRequired request correlation header propagated from edge to application.
request-idApplication-visible fallback correlation ID for logs and support workflows.

Verified on May 6, 2026: OSSM3 is Healthy on spoke-dc, Kiali is running, istio-cni is running, and the platform-side ambient runtime (ZTunnel/default Healthy with 4 ztunnel pods, istio-waypoint GatewayClass Accepted) is ready. Namespace enrollment, waypoint Gateway, and a baseline AuthorizationPolicy are drafted in PR #48 with the runbook at docs/handoff/ossm3-ambient-runbook.md; owner applies via Argo sync after PR #47, or via the standalone-apply path before.

CI/CD and GitOps

Jenkins builds, Nexus stores, OpenShift GitOps deploys

GitHub Application source, Liberty config, NGINX source config, Dockerfiles, and docs.
Jenkins CI Compile, test, package WAR, build images, scan, and prepare a promotion change.
Nexus Repository Stores Maven artifacts and versioned container images for promotion.
GitLab GitOps Owner-reviewed desired state and promoted image tags in lab-workloads.git.
OpenShift GitOps Syncs Kustomize or Helm manifests into OpenShift using desired state.

Jenkins should not deploy directly to OpenShift in the final flow. It should publish artifacts to Nexus and create or prepare a GitOps promotion change for owner review, leaving OpenShift GitOps to reconcile the cluster after the approved merge.

Verified on May 6, 2026: Jenkins build brac-poc-openliberty-v5-maven-smoke #4 completed successfully. The job read source configuration and a GitHub read-only deploy key from Vault through Kubernetes auth, cloned the private GitHub repository over SSH, compiled 13 Java sources with Java 17, resolved dependencies through Nexus maven-public, and produced brac-poc-openliberty-v5.war.

Build speed policy is cache-first and IPv4-preferred. Use Nexus-hosted or proxied base images, keep Jenkins Maven/container caches, and let the owner pre-pull demo images with crictl on approved nodes before rehearsals. For RKE2 or other containerd targets, use owner-run scp + ctr image archive imports when LAN transfer is faster than registry pulls. Jenkins can also publish an app-specific Liberty feature base image so later runtime builds can skip repeated features.sh execution.

Repository ownership

Where application code, NGINX, and manifests live

GitHub application repository

brac-poc-openliberty-v5 owns the Java application source, Open Liberty runtime config, NGINX source config, Dockerfiles, Jenkinsfile, and this Cloudflare Pages wiki.

GitLab GitOps repository

lab-workloads.git owns the OpenShift desired state: namespace, Deployment, Service, Route or Ingress, ConfigMap references, and promoted image tags.

Asset Source of truth Reason
Open Liberty app code GitHub Owned by the application delivery lifecycle and built by Jenkins.
Open Liberty server.xml GitHub Versioned with the runtime image and MicroProfile feature set.
NGINX reverse proxy config GitHub Part of the app web tier: canary routing, HSTS, rate limiting, and custom errors.
NGINX runtime deployment GitLab GitOps OpenShift desired state references the built NGINX image and runtime objects.
OpenShift manifests GitLab GitOps Already consumed by OpenShift GitOps through lab-workloads.git.
Built artifacts and images Nexus Stores versioned WAR/Maven artifacts and container images for promotion.

Future enterprise integrations

Design extension points to preserve from day one

Verified 2026-05-07: every external destination this section refers to is already running on the lab RKE2 cluster. Splunk (splunk.apps.sub.comptech-lab.com), SigNoZ (signoz.apps.sub.comptech-lab.com), Strimzi/Kafka (kafka namespace, brokers 0/1/2 + bootstrap), Keycloak / SSO IDP (auth.apps.sub.comptech-lab.com), and WSO2 IS / APIM (API gateway candidates) are all up. Per-destination enablement is now a Vault-write + collector-uncomment task, not a provisioning task.

Splunk export

Logs, metrics, and traces forward through the namespace fanout collector to a Splunk HEC endpoint. Drafted in PR #51; owner enables by writing splunk_hec_* keys to Vault and uncommenting the exporter in pipelines.

SigNoZ export

OTLP gRPC export to SigNoZ via the same fanout collector. Drafted in PR #51; owner enables by writing signoz_otlp_endpoint (and optional access token) to Vault.

Kafka topics

OTLP-proto traces, logs, and metrics on per-signal Kafka topics for a deferred consumer. Drafted in PR #51; Strimzi/AMQ Streams provisioning is owner-managed.

SSO and API gateway

Identity contract codified: API gateway owns the trust boundary, NGINX scrubs client-supplied identity headers, Liberty re-verifies JWTs via MicroProfile JWT. See PR #50 / docs/adr/0006-sso-and-api-gateway-integration-points.md.

Vault integration

Vault is the source of truth whenever applicable. The ESO pattern operationalised in PR #45 synthesizes nexus-docker-pull from secret/brac-poc-openliberty-v5/ci/nexus; the same shape is reused by PR #51 (observability-export) and the SSO/gateway paths in ADR-0006. GitHub PAT migration to Vault is drafted in PR #49.

Infrastructure governance

Cluster changes are manual owner actions

Assistant scope

The assistant may inspect cluster state with read-only commands, draft manifests, document required actions, update application or documentation files, and publish this Cloudflare Pages wiki.

Owner scope

The project owner manually performs all OpenShift, RKE2, GitLab GitOps, Jenkins, Nexus, Kafka, Redis, and platform service changes.

Direct cluster mutations are normally avoided. The current OpenShift deployment was done under an explicit owner-approved exception; future promotion should return to Jenkins, Nexus, and GitLab/OpenShift GitOps.

Secret handling is Vault-first. Kubernetes Secrets, OpenShift pull secrets, Jenkins credentials, ExternalSecrets, and CSI mounts are delivery mechanisms only and should be backed by Vault wherever the platform supports it.

Manual owner actions

Platform work needed before the full demo

All design PRs (#45–#58) are merged to main. The architectural choice for the GitOps tree (Option C — OpenLibertyApplication CRs + components/apps/<base> + cluster overlay) is also merged (this repo PR #60, lab-workloads.git MR !2) on 2026-05-07. The remaining work is operational rollout, sequenced in docs/handoff/rollout-sequence.md. Cluster-side state on 2026-05-07:

  • Phase 2 — Vault + ESO bootstrap: done end-to-end on spoke-dc.
  • Phase 3 step 1 — Jenkins K8s pod-agent PVCs on RKE2: done, PVCs match drafts.
  • Phase 5 step 1 — desired state in lab-workloads.git/main: done (MR !2 merged).
  • Phase 3 step 2 — Jenkins JCasC clouds: block: pending (real work).
  • Phase 4 — release rehearsal to Nexus: pending on 3 step 2.
  • Phase 5 step 2 — Argo CD repo credential for lab-workloads.git: pending, owner-applied with PAT.
  • Phase 5 step 3 — apply Argo Application + manual sync: pending on Phase 4 (so pods don't ImagePullBackOff).
  • Phases 6 + 7 — apply via the same Argo sync once 5 lands.
Replace the one-time direct OpenShift deployment with the approved GitLab/OpenShift GitOps promotion path.
Drafted in PR #47 · runbook docs/handoff/gitops-promotion-runbook.md.
Enable or confirm OSSM3 ambient runtime for the namespace, including ztunnel and waypoint readiness.
Platform side already Healthy. Namespace enrollment, waypoint Gateway, and AuthorizationPolicy drafted in PR #48 · runbook docs/handoff/ossm3-ambient-runbook.md.
Register or update the lab-workloads.git path for clusters/spoke-dc/brac-poc-openliberty-v5.
Drafted in PR #47 with the GitLab repo creation + Argo CD repo credential steps in docs/handoff/gitops-promotion-runbook.md.
Store Nexus release credentials in Vault, then create or synchronize Jenkins credentials and the OpenShift nexus-docker-pull image pull secret before full image promotion.
ESO-driven nexus-docker-pull drafted in PR #45 · Jenkins K8s pod agent (maven + buildah + persistent caches) drafted in PR #46 · runbooks docs/handoff/nexus-vault-bootstrap-runbook.md and docs/handoff/jenkins-k8s-agent-runbook.md.
Rotate the GitHub PAT from /home/ze/secrets/github-token; Jenkins source checkout should continue through the Vault-backed read-only deploy key.
Drafted in PR #49 · runbook docs/handoff/github-token-rotation-runbook.md · helper scripts/secrets/refresh-gh-token.sh.
Approve any future SSO, API gateway, Vault, Splunk, or SigNoZ platform integration before it is added to runtime.
Identity contract (trust boundary, header rules, mpJwt re-verification) codified in PR #50 · docs/adr/0006-sso-and-api-gateway-integration-points.md.
Decide the future logs and metrics forwarding path for Splunk, SigNoZ, or Kafka after the base trace demo is accepted.
All three plus the existing OpenShift-native view (Tempo + Loki + UWM + Perses) drafted as a fanout collector in PR #51 · runbook docs/handoff/observability-fanout-runbook.md.

Client requirement map

How the POC proves each requested capability

Requirement Demo implementation Proof point
L7 Load Balancing Two Open Liberty workloads behind one NGINX reverse proxy on OpenShift. Verified on the OpenShift route; all external proof uses NGINX.
Header-based Canary NGINX routes version: beta to Instance B. Verified response body shows instance: B, beta flags, and route decision.
MicroProfile Health Liberty MicroProfile Health exposes /health/live through NGINX. Verified /health/live returns UP.
MicroProfile Metrics Liberty MicroProfile Metrics exposes /metrics through NGINX. Verified thread_count{mp_scope="base"} is exposed.
Hardening NGINX config includes HSTS, custom errors, and rate limiting. Verified /app/login returns custom 429 under burst load.
Enterprise delivery Jenkins smoke build and Nexus Docker Registry API are verified; Vault-backed credentials and GitOps promotion remain open. Evidence pack separates current smoke proof from pending image push and GitOps sync proof.
Traceability and mesh OSSM3 ambient mode, waypoint L7 policy, and propagated trace headers. Tempo traces are verified; Kiali/ambient proof remains a later phase.

Application scope

Small app, clear enterprise signals

/appLanding page showing the active backend instance.
/app/api/applicationsMock onboarding application records.
/app/api/eligibilityLoan eligibility scoring endpoint.
/app/api/instanceReturns instance, version, hostname, and feature flags.
/app/loginProtected context for NGINX rate-limit demonstration.
/health/liveMicroProfile liveness endpoint exposed via NGINX.
/health/readyMicroProfile readiness endpoint exposed via NGINX.
/metricsMicroProfile metrics endpoint for JVM thread count.
/openapiOpenAPI document for API discovery.

Traffic steering

Canary behavior the client can see

Default request
curl -k https://brac-poc-openliberty-v5.apps.spoke-dc.ocp.comptech-lab.com/app/api/instance
Beta canary request
curl -k -H "version: beta" https://brac-poc-openliberty-v5.apps.spoke-dc.ocp.comptech-lab.com/app/api/instance

The beta response should visibly include instance: B, version: beta, and a feature flag such as new-risk-model.

The browser page at /app shows instance, channel, hostname, request_id, trace_id, span_id, latency_ms, and route_decision so the client can see routing and traceability without reading logs first.

MicroProfile observability

Health and JVM metrics through NGINX

Liveness

Enable Liberty MicroProfile Health, expose /health/live through NGINX, and show an aggregate green liveness status in the demo.

JVM thread count

Enable Liberty MicroProfile Metrics, read /metrics, and highlight the JVM thread count as the required runtime signal.

ServiceMonitorScrapes the Open Liberty metrics endpoint for user workload monitoring.
Perses dashboardShows liveness, JVM thread count, canary split, rate-limit responses, and upstream errors.
Loki logsReceives structured app and NGINX logs with request and trace correlation fields.
Tempo tracesReceives OpenTelemetry spans from the deployed Liberty workloads.
MicroProfile TelemetryEnabled in Liberty with otel.sdk.disabled=false; OTLP points to otel-collector.openshift-tracing.svc:4317.

End-to-end traceability contract

Fields every layer must preserve

Signal Required fields Where it appears
Trace headers traceparent, tracestate, and fallback x-request-id. NGINX access logs, Liberty request logs, mesh telemetry, and traces.
Application logs trace_id, span_id, request_id, instance, channel, status, duration_ms. OpenShift Logging, Loki, future Splunk forwarding.
Metrics labels app, component, instance, channel, namespace, route. Prometheus user workload monitoring and Perses dashboards.
Trace spans service.name, service.version, deployment.environment, http.route, net.peer.name. OpenTelemetry Collector, Tempo, and future SigNoZ export.

NGINX hardening

Controls to demonstrate at the web tier

HSTS response header on the HTTPS route for browser-side transport hardening.
Custom error pages for upstream failure and protected routes.
Rate limiting on /app/login and high-risk app contexts.
Brute-force simulation that returns throttling responses through NGINX.

Proof checks

Local and OpenShift validation commands

Start the local stack
docker compose up --build
Run proof checks
BASE_URL=https://brac-poc-openliberty-v5.apps.spoke-dc.ocp.comptech-lab.com tests/nginx/proof.sh
Dockerfile.libertyBuilds the Maven WAR and runs it on Open Liberty Java 17 for local/container builds.
Dockerfile.liberty.runtimeBuilds the Liberty runtime image from the Jenkins-packaged WAR.
Dockerfile.nginxRuns the hardened NGINX reverse proxy on port 8080.
docker-compose.ymlStarts Liberty A, Liberty B, and NGINX for local validation.
scripts/image-cache/prepare-ctr-cache.shBuilds image archives and an owner-run scp + ctr import script for RKE2/containerd cache warm-up.
tests/nginx/proof.shValidates routing, HSTS, liveness, JVM thread metrics, trace headers, UI visibility, and rate limiting.

Client evidence pack

Artifacts to capture during rehearsals

Evidence Repository template Purpose
GitHub commit docs/evidence/01-github-commit.md Shows source control provenance.
Jenkins build and Nexus artifact 02-jenkins-build.md, 03-nexus-artifact.md Shows CI and artifact lifecycle.
GitOps sync 04-gitops-sync.md Shows deployment through OpenShift GitOps.
Canary, health, metrics, and rate limit 05 through 08 Shows the client web-tier requirements directly.
Kiali and Perses 09-kiali-topology.md, 10-perses-dashboard.md Shows ambient mesh and observability proof after platform readiness.
Tempo trace search 11-tempo-trace-search.md Shows the fixed OpenTelemetry trace path from Liberty to Tempo.

Platform handoffs

Maintainer guidance for dependencies

docs/handoff/rollout-sequence.mdPhased order, hard gates, and parallel paths for the seven open draft PRs (#45–#51) that close all wiki Manual Actions. Owner reference for taking the drafts live.
docs/handoff/jenkins-release-worker-handoff.mdCurrent Jenkins release blocker: source Vault access works, ci/nexus returns 403, and a container-capable build worker is still required.
docs/handoff/nexus-maintainer-handoff.mdNexus readiness status covering Maven repositories, Docker Registry API readiness, Jenkins credentials, and OpenShift pull-secret actions.
docs/handoff/image-cache-speedup-runbook.mdOwner-run cache warm-up, crictl pre-pull, scp + ctr archive import for containerd targets, IPv4 validation, and Jenkins cache guidance for slow networks.
docs/handoff/openshift-observability-maintainer-handoff.mdVerified OpenShift collector trace path and future forwarding notes for Splunk, SigNoZ, or Kafka.
docs/adr/0004-vault-first-secret-policy.mdVault-first policy for CI/CD credentials, runtime secrets, pull secrets, TLS material, and future integration tokens.
docs/adr/0005-build-speed-and-image-cache-policy.mdNexus-backed image cache, IPv4-preferred egress, owner-controlled crictl pre-pull, and scp + ctr policy for containerd targets.
docs/adr/0006-sso-and-api-gateway-integration-points.mdIdentity contract across Client → API gateway → NGINX → Liberty: gateway is the trust boundary, NGINX scrubs client-supplied identity headers, Liberty re-verifies JWTs via MicroProfile JWT. Service identity (mesh mTLS) and user identity (JWT) are orthogonal.

Maven dependency resolution is verified through Nexus, and the Docker Registry API now returns the expected unauthenticated 401. Maven and image publishing should wait until Nexus credentials are readable through Vault by the Jenkins release role, a container-capable Jenkins worker is available, and the OpenShift pull secret is created or synchronized by the owner.

Demo flow

Recommended client walkthrough

  1. Open the app normally and show the request is served by Instance A.
  2. Open /app and show request_id, trace_id, span_id, latency, and route decision on the page.
  3. Send version: beta and show the request is served by Instance B.
  4. Open /health/live through NGINX and show liveness status.
  5. Open /metrics and highlight the JVM thread count metric.
  6. Trigger repeated requests against /app/login to show rate limiting.
  7. Open an unknown path to show the custom NGINX 404 page.
  8. Show Jenkins build output and explain the pending Nexus/GitOps promotion path.
  9. Show Tempo trace search for a stable and beta request; Kiali topology follows in the ambient phase.

Delivery plan

Repository structure for the POC

GitHub app repo
brac-poc-openliberty-v5/
|-- app/                  Open Liberty MicroProfile application
|-- liberty/              server.xml and runtime config
|-- nginx/
|   |-- nginx.conf
|   |-- conf.d/
|   |   `-- brac-poc.conf
|   `-- error-pages/
|       |-- 404.html
|       |-- 429.html
|       `-- 50x.html
|-- jenkins/              Jenkinsfile and pipeline helper scripts
|-- deploy/openshift/     current OpenShift manifest used for the direct exception
|-- gitops-drafts/        GitOps-ready draft resources for lab-workloads.git
|-- docs/                 Cloudflare Pages wiki
|   |-- adr/              Architecture decisions and change-control rules
|   |-- evidence/         client demo evidence templates
|   `-- handoff/          platform maintainer handoff notes
|-- observability/
|   |-- trace-context.md
|   |-- splunk-export.md
|   |-- perses-dashboard.md
|   |-- otel-collector-pipeline.md
|   `-- signoz-export.md
|-- security/
|   |-- sso-integration.md
|   |-- api-gateway.md
|   `-- vault-integration.md
|-- Dockerfile.liberty
|-- Dockerfile.liberty.runtime
|-- Dockerfile.nginx
|-- docker-compose.yml    two Liberty instances plus NGINX
`-- README.md             local runbook
GitLab GitOps repo
lab-workloads.git/
|-- components/apps/
|   `-- brac-poc-openliberty-v5/
|       |-- kustomization.yaml
|       |-- namespace.yaml
|       |-- liberty-sa.yaml             # per-component SAs
|       |-- nginx-sa.yaml
|       |-- eso-vault-auth-sa.yaml
|       |-- secretstore.yaml            # ESO -> Vault
|       |-- eso-tokenrequest-rbac.yaml
|       |-- externalsecret-nexus-docker-pull.yaml
|       |-- liberty-a.yaml              # OpenLibertyApplication CR
|       |-- liberty-b.yaml              # OpenLibertyApplication CR
|       |-- nginx-deployment.yaml       # plain Deployment
|       |-- services.yaml               # nginx Service only
|       |-- route.yaml
|       |-- servicemonitor.yaml
|       |-- prometheusrule.yaml
|       |-- networkpolicy.yaml
|       |-- waypoint.yaml               # OSSM3 ambient
|       |-- authorizationpolicy.yaml
|       |-- requestauthentication.yaml  # mpJwt
|       |-- externalsecret-runtime-sso.yaml
|       |-- externalsecret-observability-export.yaml
|       |-- otel-fanout-collector.yaml
|       `-- otel-fanout-queue-pvc.yaml
`-- clusters/
    `-- spoke-dc/kustomization.yaml     # `- ../../components/apps/brac-poc-openliberty-v5`

Decision: C (verified 2026-05-07). Liberty instances are OpenLibertyApplication CRs (apps.openliberty.io/v1), reconciled by the Open Liberty Operator v1.6.1 installed on spoke-dc via lab-gitops. The two CRs share applicationName: brac-poc-openliberty-v5 and split on applicationVersion: stable / beta, which the operator surfaces as app.kubernetes.io/version labels for canary telemetry. NGINX is still a plain Deployment (the operator only manages OpenLibertyApplication). Trade-off discussion is preserved in docs/handoff/gitops-promotion-runbook.md.