Provision Feed Ingestion¶

What this is: standing up the NiFi → track_hits path so external AIS / ADS-B / GeoJSON feeds land in the deployment's database and render on the map. When you'd do it: a deployment that needs to ingest external position feeds (vessel AIS, aircraft ADS-B, a GeoJSON source). How long it takes: an hour or so — most of it is the NiFi-side processor setup.

Who does this: a deployment engineer with database access and access to the NiFi host. The DB role is created by a migration; granting it a login and credential is an out-of-band step you do by hand.

This is one of two ingest paths and the only one that writes track_hits directly via a Postgres role. The other — the HTTP /api/feed/* API used by force-tracking producers — uses a bearer JWT instead; see Issue an API credential and Machine Ingest API. Use this runbook for NiFi feeds.

The shape of it¶

flowchart LR
    A["Grant the<br/>nifi_ingest role<br/>a login (out-of-band)"] --> B["Create / identify<br/>the track_feed layer"]
    B --> C["Deploy NiFi NAR +<br/>configure feed<br/>processors"]
    C --> D["Feed rows land<br/>in track_hits<br/>→ render on map"]

Before you start¶

The web tier and its database are live (see Stand up the web client) and its migrations have run — the nifi_ingest role and the track_hits validation trigger come from a migration.
Database admin access (to grant the role a login and credential).
Access to the NiFi host, and the bedrock-nifi-processors build (the NAR) — the source processors live in that repo.

Step 1 — Give the `nifi_ingest` role a login¶

The migration creates a least-privilege Postgres role, nifi_ingest:

It is NOLOGIN and the migration deliberately stores no secret — there is nothing to leak in the schema.
Its grants are column-scoped: INSERT on track_hits (only the columns it needs) and SELECT on layers(id, slug, type) so a feed can resolve a layer slug → id.

Grant it a login and a credential out of band — this is the one secret-handling step, and it is intentionally not in the migration:

ALTER ROLE nifi_ingest LOGIN PASSWORD '<generated-secret>';

Hand that credential to the NiFi side securely; it's what PutDatabaseRecord authenticates with.

A validation trigger guards the table. A BEFORE INSERT trigger on track_hits rejects bad rows — null or out-of-range lat/lon, the (0, 0) AIS no-fix sentinel, an empty principal, or the wrong layer type. Rejected rows route to NiFi's failure relationship rather than landing silently, so wire that relationship up so you can see them.

Step 2 — Create or identify the `track_feed` layer¶

Feed rows are attached to a layer. Create (or identify) a layer of type track_feed for the feed, then resolve its slug → layer_id (the role has SELECT on layers(id, slug, type) for exactly this). Use that layer_id for the rows the feed inserts.

Set the layer's source_type (in /admin/maps) to the feed kind — one of ais | adsb | radar | geojson (CHECK-enforced). The kind is a property of the feed, so it lives on the layer, not on each row; the server carries it onto every position at fan-out, and it drives the client filter chips. Give each kind its own track_feed layer — a feed must not mix kinds within one layer.

Step 3 — Deploy and configure the NiFi side¶

Deploy the processor NAR from the bedrock-nifi-processors build onto the NiFi host.
Configure the source processors for the feeds you're ingesting — Telesto AIS, Nightingale, and/or GeoJSON — each pointed at its matching-kind track_feed layer from Step 2.
Point PutDatabaseRecord at the database using the nifi_ingest role's credential from Step 1, writing into track_hits against the layer_id from Step 2.
Wire the failure relationship so rows the validation trigger rejects are visible.

Feed kind comes from the layer — NiFi does not write it per row

The feed kind (ais | adsb | radar | geojson) is a property of the layer (layers.source_type, Step 2), not a column NiFi writes — there is no per-row source_type. NiFi just targets the right track_feed layer; the server derives the kind from that layer at fan-out. Put the provider/feed name (e.g. telesto, nightingale) in the free-text source column for provenance.

ADS-B is in the kind enum but has no NiFi producer yet, so there is currently no NiFi path to ingest ADS-B.

How to know it worked¶

Insert a test feed row through the NiFi flow and confirm it lands in track_hits and renders on the map for the track_feed layer.
Confirm the guard rejects a bad row: push a row with (0, 0) coordinates (the AIS no-fix sentinel) or an out-of-range lat/lon and confirm it is rejected and routed to the failure relationship rather than inserted.
Confirm the rendered feature carries the layer's source_type kind (and so gets its filter chip) — it is derived from the track_feed layer, not the row.

If something goes wrong¶

Every insert is rejected (permission denied). The nifi_ingest role hasn't been granted LOGIN / a credential, or PutDatabaseRecord is using the wrong credential. Redo Step 1.
Rows land but show no filter chip. The row's track_feed layer has no source_type kind set. Set it in /admin/maps (Step 2).
Good-looking rows are still rejected. Check the validation trigger's conditions: lat/lon range, the (0, 0) sentinel, an empty principal, and that the row's layer is a track_feed-type layer.
Trying to ingest ADS-B and there's no processor. ADS-B has no NiFi producer yet — there is no NiFi path for it at present.