All documentation Download (Markdown) Technical

Gateway · gateway.pdhc.se

Technical manual

Gateway.pdhc — Technical Documentation

Architecture

Flask application with PostgreSQL (pgvector image) running in Docker as part of the PDHC microservice platform. The whole service is containerised: gateway_pdhc_app (Flask/gunicorn) and gateway_pdhc_db (Postgres) are two containers in the same compose project.

The gateway is a validating relay, not a datastore. Inbound provider reports are validated, queued for delivery, and forwarded to the clinical data repository (cdr1), which is the single source of truth for observation data. The gateway keeps only delivery/dedup bookkeeping, per-service-request status, an upstream-lookup cache, and the audit trail. The former local observation tables (inbound_observations, observation_vectors) have been dropped.

Ports: 9050 (Flask/gunicorn, bound to 127.0.0.1), 9051 (PostgreSQL, bound to 127.0.0.1)

Database models

Table	Model	Purpose
`guid_resolution_cache`	`GuidResolutionCache`	Cache for upstream GUID/context lookups, keyed by `(source_guid, source_type)` with a per-entry TTL
`service_request_status`	`ServiceRequestStatus`	Per-SR completion/expiry tracking (active / completed / expired / partial), delivery counters, grant expiry
`cdr_delivery_log`	`CdrDeliveryLog`	The forwarder queue. One row per accepted observation; holds the FHIR resource, denormalised dedup keys, retry/status bookkeeping, and the captured operator session id
`audit_log`	`AuditLog`	GDPR-compliant audit trail

Dropped tables (do not reference — they no longer exist):

inbound_observations — dropped in migration d7e8f9a0b1c2. The gateway no longer stores observations locally; cdr_delivery_log is the sole queue + dedup index + provenance source, and cdr1 is the permanent store.
validation_log — dropped in migration d7e8f9a0b1c2.
observation_vectors — dropped in migration f3a4b5c6d7e8.

`cdr_delivery_log` (forwarder queue)

Written directly by ReportIngestionService (there is no intermediate InboundObservation row). Each row carries patient_guid, service_request_guid, concept_guid, transaction_guid, contract_guid, provider_org_guid, fhir_observation_json, dedup columns (payload_hash, dedup_key), operator_session_id (X2 #408), and delivery state (status, attempt_count, last_attempt_at, last_error, delivered_at, cdr_resource_id).

Row status:
- pending — a complete production observation, ready to forward. Requires service_request_guid, concept_guid, contract_guid, provider_org_guid (enforced at both the ingest boundary and in the forwarder; a pending row with any of these NULL is marked terminally failed rather than shipped dirty).
- skipped — QR parent rows and freeform/unresolved-concept rows that are not forwardable. Never sent to cdr1.
- delivered / failed — terminal states set by the forwarder.

Services

Service	Purpose
`PATValidationService`	Validates provider tokens via request.pdhc upstream
`GrantValidationService`	Validates the composite key by delegating to request.pdhc `/internal/grant/validate` — the HMAC secret never leaves request.pdhc (see below)
`ObservationValidator`	FHIR R5 observation schema validation
`ReportIngestionService`	Full validation chain + write to `cdr_delivery_log` + receipt push
`GuidResolutionService`	Resolves transaction → careplan → plandefinition chain (cached)
`SRContextService` / `ContractScopeService`	Fetch SR context and contract return-scope from upstream (cached)
`RequestCompletionService`	Maintains `service_request_status` (delivery progress, expiry)
`FeedService`	Proxies provider feed/download from request.pdhc
`PushService` / `ReceiptService`	Per-PAT receipt push to providers + acknowledgement
`fhir_observation_builder`	Builds the fully referenced FHIR R5 Observation + the canonical typed observation core (#500)
`CdrClient` + `cdr_forwarder` worker	Forwards `cdr_delivery_log` rows to cdr1 (`/api/v1/ingest`)
`AnalyseClient`	Analyse-pull proxy client → analyse.pdhc `/api/v1/observations`

GrantValidationService no longer computes HMAC locally. It POSTs {sr_guid, patient_guid, org_guid, grant_token} with X-Service-Key: REQUEST_INTERNAL_SERVICE_KEY to REQUEST_SERVICE_URL/internal/grant/validate and returns request.pdhc's verdict (with contract_guid, grant_type, uses_remaining on success). The old local VectorService has been removed.

The gateway → cdr1 forwarder

Accepted observations are inserted into cdr_delivery_log with status='pending' (insert-then-send). A background APScheduler job, run_forwarding_cycle, drains the queue:

Started in create_app (_start_cdr_forwarder_scheduler), gated on CDR_FORWARDING_ENABLED so a first deploy can be dark. Interval is CDR_FORWARDING_INTERVAL_SECONDS (default 60s).
Both gunicorn workers start a scheduler; the worker claims rows with SELECT ... FOR UPDATE SKIP LOCKED (Postgres) so the two schedulers never double-process. The SQLite test path falls back to a plain SELECT (single-process).
Per-row exponential backoff (BASE_BACKOFF=10s, MAX_ATTEMPTS=5, BATCH_LIMIT=50). A 5xx / network error is retryable; a terminal 4xx from cdr1 marks the row failed without burning the retry budget.
Delivery goes through CdrClient.deliver_one → POST {CDR_BASE_URL}/api/v1/ingest with headers X-Source-Service: gateway.pdhc, X-Service-Key: GATEWAY_PDHC_SERVICE_KEY, X-Request-Id, and X-Operator-Session-Id (replayed from the row so the chain-of-custody survives the async gap).
The payload includes the FHIR R5 resource, provenance re-hydrated from the SR context (best-effort — a context-fetch failure does not block delivery), and the canonical typed observation (build_canonical_observation, #500). The canonical core is also carried inside the FHIR resource as extension urn:pdhc:fhir:extension:canonical-observation, so the openEHR projection (rosetta, #501) reads real typed values rather than the lossy FHIR value[x].

CDR_FORWARDING_DELETE_AFTER_DELIVERY is now dead config (there is no local observation row left to delete) and is ignored.

The analyse-pull proxy (`GET /api/v1/observations`)

GET /api/v1/observations?organization=<org_guid> is an SSO-gated JSON proxy for analysis-phase consumers — it is not a local observation-browsing page (the gateway stores no observations). Gateway responsibilities on this route:

validate the SSO bearer token and require analysis-phase access (403 otherwise),
org-membership check; admin cross-org read requires an X-Admin-Justification header (#220),
resolve SR → contract → requesting_org locally (gateway owns this mapping; contract.pdhc is its upstream) and compute the matching SR list,
proxy the pre-computed SR filter to the analyse layer, which federates over CDR1–6 and returns one merged FHIR R5 searchset Bundle,
apply the IPS spärr filter to drop blocked provider rows (#206 / PDL Ch 4 §4),
audit the read (observations.read / observations.admin_read).

The proxy target is ANALYSE_BASE_URL, which now points at analyse.pdhc (http://host.docker.internal:9110). This replaced the earlier dashboard.pdhc target — the code default was repointed to analyse (#540) and the server .env follows. AnalyseClient authenticates with X-Source-Service: gateway.pdhc + X-Service-Key: GATEWAY_PDHC_SERVICE_KEY (the same key value gateway uses for cdr1) plus the outbound operator-session headers.

Upstream / downstream dependencies

Service	Purpose	Config
request.pdhc	PAT validation, grant validation, GUID chain, SR context, feed/download proxy	`REQUEST_SERVICE_URL`, `REQUEST_INTERNAL_SERVICE_KEY`
contract.pdhc	Contract scope / parties validation	`CONTRACT_SERVICE_URL`, `CONTRACT_INTERNAL_SERVICE_KEY`
sso.pdhc	Bearer-token validation for analysis endpoints + operator login	`SSO_BASE_URL`, `SSO_CLIENT_ID`, `SSO_CLIENT_SECRET`, `SSO_CALLBACK_URL`
cdr.pdhc (cdr1)	Forwarding target — permanent observation store	`CDR_BASE_URL`, `GATEWAY_PDHC_SERVICE_KEY`
analyse.pdhc	Analyse-pull federation target	`ANALYSE_BASE_URL`, `GATEWAY_PDHC_SERVICE_KEY`
provider (per-PAT)	Receipt delivery — routed per-PAT via `push_endpoint_url` / `push_secret` from request.pdhc, not a fixed URL

Configuration (.env)

# Database (Postgres container gateway_pdhc_db)
DATABASE_URL=postgresql://gateway_pdhc:password@db:5432/gateway_pdhc_db
FLASK_SECRET_KEY=<random-64-char>

# request.pdhc — PAT + grant validation, GUID chain, SR context
REQUEST_SERVICE_URL=https://request.pdhc.se/api/v1
REQUEST_INTERNAL_SERVICE_KEY=<shared key with request.pdhc /internal/*>
GUID_CACHE_TTL_SECONDS=3600
GRANT_CACHE_TTL_SECONDS=60

# contract.pdhc — contract scope validation
CONTRACT_SERVICE_URL=https://contract.pdhc.se
CONTRACT_INTERNAL_SERVICE_KEY=<shared key with contract.pdhc /internal/*>

# SSO (analysis endpoints + operator login)
SSO_BASE_URL=https://sso.pdhc.se
SSO_CLIENT_ID=<gateway sso client id>
SSO_CLIENT_SECRET=<gateway sso client secret>
SSO_CALLBACK_URL=https://gateway.pdhc.se/auth/callback
# AUTH_DISABLED=true   # local dev only — never set on the server

# cdr1 forwarding
CDR_BASE_URL=http://cdr_pdhc_app:9046
CDR_FORWARDING_ENABLED=true
CDR_FORWARDING_INTERVAL_SECONDS=60
CDR_TIMEOUT_SECONDS=30

# Shared service key: cdr1 ingest AND analyse-pull both authenticate with it.
# Must match GATEWAY_PDHC_SERVICE_KEY in cdr.pdhc and analyse.pdhc .envs.
GATEWAY_PDHC_SERVICE_KEY=<shared service key>

# Analyse-pull federation target (analyse.pdhc, #540)
ANALYSE_BASE_URL=http://host.docker.internal:9110

# Bootstrap + receipt push
BOOTSTRAP_SU_API_KEY=<initial-superuser-key>
PUSH_TIMEOUT_SECONDS=30
PUSH_RETRY_COUNT=3

Notes:
- There is no HMAC_SECRET — grant validation is delegated to request.pdhc, so gateway holds no HMAC secret.
- The DB user/name are gateway_pdhc / gateway_pdhc_db (compose defaults).
- Receipt delivery has no fixed PROVIDER_SERVICE_URL; each PAT carries its own push_endpoint_url / push_secret.

Admin pages (SSO, admin-only)

Route	Purpose
`/admin/cache`	Cache stats by source type (total/fresh/stale), per-type and global flush buttons, recent entries
`/admin/cache/flush`	POST — flush `mode=stale` or `mode=all`, optional `source_type` filter
`/admin/health-report`	Live probes of the gateway DB, request.pdhc, contract.pdhc, sso.pdhc + error events from last 24h
`/pats`	PAT validation/rejection activity per provider org
`/audit`	Paginated GDPR audit trail
`/grants`	Grant/`service_request_status` validity tracking (active / expiring / expired)

Cache flush details

The guid_resolution_cache table stores upstream responses keyed by (source_guid, source_type) with a per-entry TTL. Flush options:

Flush stale: iterates entries and deletes those where is_expired() is true (age > ttl_seconds). No disruption — these would be re-fetched on next access anyway.
Flush all: deletes all entries (optionally filtered by source_type). Causes a burst of upstream calls as the cache repopulates.

Neither flush requires a service restart. The cache is in Postgres, not in-process memory.

Health endpoint

GET /api/v1/health (and /metadata for the FHIR CapabilityStatement) is public/anonymous — it pings the DB and returns the canonical shape:

{"status": "ok", "service": "gateway.pdhc", "database": "connected"}

HTTP 200 when the DB is reachable, 503 when degraded. It sets Access-Control-Allow-Origin: https://www.pdhc.se so services.html can read the real status/DB fields cross-origin (CLAUDE.md §10).

Running

The service is containerised — there is no start.sh / safe_restart.sh. Deploy and restart via docker compose from the service directory on the macmini:

# Build + (re)start both containers, picking up code and .env changes
docker compose up -d --build

# Apply migrations against the running app container
docker exec gateway_pdhc_app flask db upgrade

# Tail logs
docker compose logs -f app

docker compose up -d --build is required after code changes: the image bakes source via COPY . ., so scp'ing files + a bare restart runs stale code. docker restart also keeps create-time env — use docker compose up -d to pick up host .env changes. Never docker compose down -v (deletes the data volume) and never touch Colima or sibling services.

Testing

cd gateway_app
source venv/bin/activate

# Unit tests (excludes integration)
pytest tests/ -m "not integration"

# All tests including live service probes
pytest tests/ -m integration

# Full suite
pytest tests/ -v

Read-side audit granularity (ticket #221, PDL Ch 4 § 3)

A read returning rows about N patients can audit either per-query
(one row per call, cheap) or per-patient (one row per patient
touched, richer for kontroller — "did anyone read patient P's data").
PDL Ch 4 § 3 obliges every vårdgivare to be able to answer that
kontroller question. The right granularity per route is a deliberate
trade between answerability and audit-table volume.

Decision matrix

Route	Method	Patient identifiers in response?	Granularity	`event_type`	Rationale
`/observations` (normal scope)	GET	Yes — FHIR Bundle with patient refs	per-query with `patient_guids[]` in payload	`observations.read`	High-volume analyse-phase polling. The per-query row carries the full sorted patient_guids list, so kontroller can answer "was P in any read by X?" with `WHERE payload_snapshot->'patient_guids' ? '<guid>'`. Storing the list once per query is ~36 bytes × n_patients vs. one full row per patient.
`/observations` (admin off-org bypass)	GET	Yes	per-patient	`observations.admin_read`	Rare and high-stakes (PDL Ch 4 § 1 — admins are not exempt from need-to-know without an audit trail). One row per patient touched, each carrying the same justification text + correlation_id, so kontroller filters cheaply by patient. Volume cost is acceptable because this path is rare; the X-Admin-Justification gate (#220) caps frequency.
`/provider/feed`	GET	Metadata only (SR refs, no PHI)	per-query	`provider.feed.polled`	Polled on a 30s cadence. PDL § 3 cares about reads of patient data; metadata polling that yields no PHI is the wrong granularity for per-patient rows. Snapshot carries `since` cursor + `n_items` + `limit`, enough to reconstruct what window the provider polled.
`/provider/download/<sr_guid>`	GET	Yes — full Bundle for a single SR	per-query (which IS per-patient: one SR == one patient)	`bundle.downloaded`	The route is single-SR. Per-query and per-patient collapse to the same row. No change needed.
`/provider/report/<sr_guid>`	POST	Inbound (not a read)	per-row inside ReportIngestionService	`report.received` / `report.rejected`	Write path. Not in this ticket's scope but noted for completeness.

When to use which

Per-query is the right shape when (a) the result set is
predictable from query args alone, and (b) the audit consumer can
answer "did P appear?" from a list in the snapshot. Cheaper writes.
Per-patient is the right shape when (a) the act of reading is
high-stakes per patient (admin bypass, indispensable-care lifts,
cross-caregiver reads), or (b) the result set is sparse and adding
the patient list to the snapshot would be more data than one row
per patient.
Neither is right for routes that don't touch patient-identified
data (capability statements, metadata, healthchecks) — those skip
audit rows entirely.

Per-patient kontroller query (PG)

For the observations.read normal path (per-query with patient_guids
list), an operator can answer "show every read of patient P by any
user" with:

SELECT *
FROM audit_log
WHERE event_type = 'observations.read'
  AND payload_snapshot->'patient_guids' ? '<patient-guid>'
ORDER BY created_at DESC;

For the admin-read path (per-patient rows), the equivalent is the
direct filter:

SELECT *
FROM audit_log
WHERE event_type = 'observations.admin_read'
  AND payload_snapshot->>'patient_guid' = '<patient-guid>'
ORDER BY created_at DESC;

Both surface the same answer; the granularity choice changes only
which side pays the storage cost (writer-side row explosion vs.
reader-side JSONB index lookup).

Port Allocation

All ports bind to 127.0.0.1 (loopback only); external traffic arrives
via the reverse proxy.

Port	Service
9050	Flask application (Gunicorn)
9051	PostgreSQL database