Direct tactical payload contract

Status: Common #257 is the authoritative metadata contract. The complete chat migration is Common #256. Common #258 has landed the serialization-only migration for Heartbeat, Voice, Drawing, DrawingDelete, Target and TargetDelete in the affected publish/consume paths listed below. Position and Android publication remain blocked as recorded in the implementation evidence; #258 is not complete. This page is authoritative for the metadata formerly carried by TacNetMessage. The wire protocol, security gates, and capability and opaque-address contract continue to govern authentication, authorization, encryption and transport.

Common merge 7785c520a5fff35dd1e2dbc0e6ecd8e9b0f4b743 removes the chat arm, reserves wrapper tag/name 11, and adds canonical ChatMessage.origin_uid = 5. Common merge 587115d09ae6640a3b7ea7f09bd7d78ca3a7c907 removes the remaining non-position wrapper constructors and replay-metadata adapters while retaining the generated type for #259. Chat and the migrated non-position families have no wrapped compatibility decoder in the merged Common, Web, Gateway and Node paths. Published Android still uses the wrapper for its non-chat families while its behavior-equivalence review remains unresolved. TacNetMessage also remains temporarily for Position. There are no live deployments or legacy clients. Common #259 chooses and records the final breaking package version and removes every wrapper fallback. Docs #210 owns final consumer revision alignment and the one live integration smoke. This contract does not claim that those migrations or that smoke have run.

Authority and field disposition

Every current wrapper field has exactly one disposition below. A payload fact is integrity-protected by the endpoint signature over the sealed payload; it is not independent Directory authority. The oneof discriminator itself is removed: the verified concrete key selects exactly one direct decoder.

Wrapper field (tag)	Disposition	Authoritative successor / rule
sender_public_key (1)	Verified-envelope fact	Verified IdentityToken principal_sign_key; stable attribution uses principal_id, never an inner key or routing segment
timestamp_ms (3)	Removal	Redundant wrapper timestamp disappears. Each plane's event time is specified below; envelope issued_at_ms remains publication time
classification (5)	Verified-envelope fact	Required signed AuthEnvelope.classification, validated by Common CMBAC; no absent-label default
position (10)	Canonical payload fact	Direct PositionUpdate
chat (11)	Canonical payload fact	Direct ChatMessage
drawing (12)	Canonical payload fact	Direct DrawingShape
drawing_delete (13)	Canonical payload fact	Direct DrawingDelete
heartbeat (14)	Canonical payload fact	Direct Heartbeat
voice (15)	Canonical payload fact	Direct VoiceFrame
identity_token (16)	Verified-envelope fact	Exact outer Directory-signed token. No cached inner-token fallback or “unknown operator” bypass
target (17)	Canonical payload fact	Direct TargetCop
target_delete (18)	Canonical payload fact	Direct TargetDelete
origin_uid (19)	Canonical payload fact	Position/chat bridge correlation inside sealed canonical payload; drawing provenance uses external_source; absent on native traffic

Already retired sequence (2) and device_callsign (4) remain removed and their reserved tags/names must never be reused. No payload-level identity token, sender key, callsign, receipt timestamp, or generic metadata carrier replaces the wrapper. Callsigns retain the permanent verified human-ORBAT override → signed token fallback rule; devices and services use the signed token callsign only.

Direct shapes and key witnesses

This table defines the target plaintext contract. Chat publishers and consumers now use direct canonical ChatMessage, including shipped string origin_uid = 5. The contract allocates PositionUpdate string origin_uid = 11 to #258; that field is not yet in the protobuf schema. The strict serialization-only #258 implementation could not add that field without changing Position's payload contract, semantic digest and Gateway track attribution. Position therefore remains wrapped pending an explicit contract decision. No other new payload metadata fields are needed; voice uses envelope time.

All content except heartbeat is sealed with the existing group-key format. Heartbeat stays plaintext inside the signed AuthEnvelope; it still needs the same identity, capability, proof, classification and replay checks. Semantic identifiers used by these bindings contain 1–256 UTF-8 bytes and no Unicode control characters. operation_id additionally uses Common valid_operation_identifier (including whitespace/BOM rejection).

H(domain, id) below means Common's existing framed opaque derivation with the 32-byte deployment salt. It never means a hash of a callsign, an unsalted id, or a semantic id copied into a key. Every key must first pass the existing canonical key grammar and envelope digest binding.

Plane / canonical plaintext	Purpose and authority	Semantic key witnesses	Event time / semantic order
Chat / ChatMessage	LIVE, retained REUSABLE_CONTENT	chat-channel(chat_id) and chat-message(message_id)	timestamp_ms; immutable (chat_id, message_id)
Position / PositionUpdate	LIVE_PUBLICATION, LIVE	position-principal(verified principal_id); geohash-5 of signed coordinates	timestamp_ms; latest observation per native device or Gateway/origin tuple
Heartbeat / Heartbeat	LIVE_PUBLICATION, LIVE	heartbeat-principal(verified principal_id); signed key cell is a presence-routing hint, not a coordinate claim	timestamp_ms; latest observation per verified device
Voice / VoiceFrame	LIVE_PUBLICATION, LIVE	voice-channel(voice_id), voice-principal(verified principal_id), voice-session(voice_stream_id)	Envelope issued_at_ms; frame_sequence only orders frames within that publisher/session
Drawing / DrawingShape	DURABLE_CONTENT, existing drawing publication capability	drawing-id(shape_id); geohash-5 of first wire geometry vertex	timestamp_ms; positive revision orders the object
Drawing delete / DrawingDelete	Same as drawing	drawing-id(shape_id) on draw/del; exact signed deletion cell	timestamp_ms; positive revision advances the same object stream
Target / TargetCop	DURABLE_CONTENT, existing target publication capability	target-id(target_id); geohash-5 of signed lat/lon; nonempty canonical operation_id is sealed replacement-scope witness	Positive timestamp_ms is the existing target semantic version
Target delete / TargetDelete	Same as target	target-id(target_id) on target/del; exact signed deletion cell and sealed operation_id	Positive timestamp_ms advances the same target stream

A drawing's first vertex is line/polygon point 0, circle center, rectangle corner1, arrow start, marker/sensor position, or TCM control point 0. Missing geometry is malformed. Derive the cell from quantized wire coordinates, not an unquantized UI point or the operator's position. Coordinate limits are inclusive ±90 latitude and ±180 longitude. At subdivision boundaries geohash chooses the upper half (>= midpoint), interleaving longitude first for exactly 25 bits.

Deletion payloads have no coordinates. Their authenticated concrete key is the cell witness: a delete-first tombstone must be accepted without needing a prior local row, subject to publication and object mutation authority. A tombstone must never be rejected merely because its object is absent. A move that affects multiple cells needs a separately authorized/signed publication for each key; one signed envelope cannot be aliased onto a second cell. Operation membership and target authority remain application authorization checks, never inferred from a syntactically valid operation_id.

Identity, event time and provenance

Attribution records the verified publisher principal, signing key and device, the exact signed envelope/key, signed classification, publication time and payload event time. Local receipt time is a separate diagnostic only. Recording an archive or projection must preserve these facts without reconstructing an endpoint publication. A snapshot uses its own Web authority and existing source provenance rules; it is never an original endpoint envelope.

Event time must be positive and no later than signed publication time plus the fixed DIRECT_PAYLOAD_MAX_EVENT_LEAD_MS = 60_000 semantic allowance. An older event is valid: disconnected chat/drawing content and Gateway observations need not have been created when they are finally published. Payload time cannot extend token/capability life, relax freshness, change replay mode or refresh an old observation's displayed age. Voice has no independent capture-time field; publication time is its event time. Frame sequence zero is valid; sequence wrap requires a new stream id. A speaking frame requires nonempty Opus data. A stop frame may include its final audio bytes or be empty; its speaking=false marker must always be applied.

Only a Directory-signed device identity with the gateway service-role claim may supply a nonempty position/chat origin_uid. This is a provenance-kind restriction, not a publication grant: the existing Common effective-capability/action and signed publication-authority gates remain mandatory; the raw role never grants an action. The value preserves code points and case, has at most 256 UTF-8 bytes, excludes Unicode control characters (U+0000–001F, U+007F–009F) and /, *, ?, #, $. It is a foreign correlation handle, not an identity or callsign. It never enters keys, operational logs or identity labels. Two Gateways asserting the same origin remain different tracks; multiple origins under one Gateway share its one authorized opaque source. Native tracks retain the verified device anchor; Gateway tracks use exactly (verified principal_id, origin_uid). Missing/invalid bridge provenance must not fall back to an external principal claim.

Drawing imports retain the existing ExternalDrawingSource: protocol, ingress, foreign object UID and original Gateway principal. Initial import binds the Gateway UUID to the verified publisher and requires Gateway authority. Later properly authorized edits and deletes preserve the original owner and provenance; the current verified publisher is recorded separately as the actor. A moderator is not forced to equal the original owner. Do not infer ownership from a payload sender key, from an external source alone, or from “first packet observed”. Unknown ownership must be resolved through the existing verified recovery path before an owner-only mutation can be accepted. With no verified owner-recovery fact, a delete-first drawing tombstone requires manage_annotations; ordinary draw_annotations alone cannot prove ownership. Targets require manage_targets and the existing verified operation/cell relationship. Absence is not authorization.

Fail-closed receive order

1. Parse the outer envelope and transport authorization only. Verify Directory identity, endpoint signature, nonce/freshness under the explicitly selected live or retained path, and current revocation through Common's existing verifier and authorize typestate.
2. Complete the existing publication-capability/use-proof composition for the exact envelope and delivered key. Require the family purpose, classification permit for publisher/grant/recipient, exact digest/key/subject bindings and active query authorization for retained replies. Retained pages stay provisional until their authenticated terminal receipt passes.
3. Only then open sealed content (heartbeat is the sole plaintext exception). Select one decoder from the verified key family, including /del. TypeScript uses Common's strictUtf8Reader at the outermost plaintext decode; Rust prost already rejects invalid UTF-8. Require Common's validate_direct_payload_encoding / validateDirectPayloadEncoding against the exact opened plaintext. This canonical-byte check rejects unknown fields, duplicate singular/oneof fields, noncanonical varints and unsorted maps; no two-variant drawing may reach semantics on either runtime. Run existing sensor/geometry validation as well as these metadata checks; a metadata helper is not a complete renderer validator. A failed direct decode is a rejection, never permission to try a wrapper, another plane, plaintext, a cached identity, or a relaxed verification policy.
4. Validate required payload fields, semantic key witnesses, event time and provenance. Target's retained inner classification must be present and exactly equal to the canonical outer label, including policy, categories, creation DTG and originator. Security-equivalent labels alone are insufficient. Every other plane gets its classification solely from the envelope.
5. Check application mutation authority and immutable owner/provenance, then logical duplicate/version/tombstone rules. Only after all gates pass may any UI, recorder, dispatch callback or projection observe the value. No partial writes on a mismatch; missing trusted context fails closed.

A payload with extra sender/key/token/classification claims is not a second source of authority. During source migration, an existing wrapper path must reject any supplied wrapper key/token/label that conflicts with its verified outer counterpart; it must not copy those fields into trusted metadata. The new direct path has no such fields or wrapper fallback. Malformed protobuf, wrong-family plaintext, missing identifiers, mismatched channel/message/object/ source/session/cell, future event time, unauthorized origin, wrong target label and wrong purpose all fail before application dispatch.

Replay, duplicates and state

Transport replay and semantic duplicates are distinct. Keep the existing nonce/proof replay caches; the semantic helpers do not replace them. A repeated fresh envelope/proof fails the cryptographic gate. Re-signing identical content with a fresh nonce does not make it a new logical event.

• Position, heartbeat and voice are non-retained: no transport outbox, mesh history/query/restart replay or fallback to allow-expired verification. A trusted application archive can render verified history with its original age and provenance; it cannot republish it as live endpoint traffic.
• Chat uses the explicit retained LIVE/REUSABLE_CONTENT history path. Retained query authority, receipt, historical publication authority and current revocation remain mandatory. Deduplicate (chat_id, message_id) across live, reconnect and multiple storage copies. Identical semantic content and author is idempotent; a different value or author for the same identity is a conflict.
• Drawings and deletes share one positive revision stream. Targets and deletes share the existing positive timestamp version stream. Higher authorized versions win; lower versions cannot resurrect state. Equal version plus equal canonical semantic content is idempotent; unequal content is a fork and fails closed, including a value/delete collision. A later permitted resurrection requires a strictly higher version. Neither envelope reissue time, receipt order, HLC nor a local database counter breaks a tie.
• Voice deduplicates (verified principal_id, voice_stream_id, frame_sequence); a fresh envelope cannot cause the same frame to play twice. End-of-transmission (speaking=false) is meaningful and must not be discarded as an empty frame. Bounded jitter reordering may play unseen frames; sequence alone confers no authenticity and cannot cross sessions or publishers.

Persist semantic dedup/version/tombstone state with the existing session cache and offline horizon. Refresh revocation and the full capability set before shared operation/outbox drainage on reconnect. Beyond-horizon recovery remains purpose-separated authoritative snapshot recovery; it is not wrapper replay.

For chat, Web archives the exact canonical key, signed envelope bytes, publication time, signing key/device, origin, signed classification and semantic digest. Replay requires both that signed publication label and the current or retired bound-net label to be readable; neither substitutes for the other. Android stores the same verified publication evidence and scopes durable logical identity by (chat_id, message_id); an identical locally provisional row may be upgraded with verified evidence, but a conflicting row is not overwritten. Gateway keeps a bounded process-owned semantic cache across subscriber reconnects for the same offline horizon, suppressing identical reissues and rejecting conflicts before GeoChat adaptation.

Shared conformance API and evidence

Common owns src/direct_payload.rs, its browser-safe TypeScript mirror, and the single direct_payload_v1.json corpus. validate_direct_payload / validateDirectPayload validate decoded semantic facts against an exact authenticated envelope/AuthorizedIdentity, key, salt and explicit origin witness. They return no dispatch authority and do not prove that a caller-supplied decoded value/origin came from that envelope's ciphertext. Callers must verify the exact live/retained publication composition, open it, select its one direct decoder, and pass only that decoded value. The explicit position-origin argument remains a conformance seam until #258; chat provenance is read from ChatMessage.origin_uid. The helpers do not implement consumer migration or an alternate verifier. validate_direct_drawing_provenance / validateDirectDrawingProvenance check initial Gateway/source equality or immutable source across an already-authorized edit; they do not grant ownership/moderation. validate_migration_wrapper_metadata / validateMigrationWrapperMetadata reject supplied redundant sender-key/token/label conflicts. They are marked TACNET_REMOVAL_259 and are deleted with the wrapper; they never decode or provide a direct-path fallback.

The corpus explicitly labels its signed, plaintext semantic-unit envelopes as not publishable transport samples: its capability is a verifier fixture, not authority for each semantic key. Both languages authenticate before the key-selected test decoder, test malformed protobuf, and reject an identical envelope on the same replay cache. End-to-end capability/proof, ciphertext and retained-page composition remain covered by Common's live_publication_v1, storage_authorization_v1, sealed-content and retained_replay_v2 corpora.

compare_direct_versions / compareDirectVersions compare only already authorized values within the same logical-id/owner/session scope. Their fixed 32-byte digest is generated by direct_semantic_digest / directSemanticDigest:

SHA-256("petra-direct-semantic-v1\0"
  || str32(plane) || str32(verified publisher principal_id)
  || bytes32(canonical ConfidentialityLabel protobuf)
  || str32(origin_uid) || bytes32(canonical direct payload protobuf))

Each str32/bytes32 is a u32 big-endian byte length followed by UTF-8/raw bytes. The plane is derived from the typed payload and is exactly chat, position, heartbeat, voice, drawing, drawing_delete, target, or target_delete. Common's canonical_direct_payload / canonicalDirectPayload owns canonical generated encoding, including maps sorted by UTF-8 key; repeated-field order is preserved. Publishers must use this encoder. The typed digest API calls it itself, so arbitrary caller-supplied bytes cannot become semantic state. This includes actor, label, payload event time and value/delete kind, but excludes envelope nonce, key-cell fan-out and reissue time. Voice has no payload event-time field: its envelope-derived event time is intentionally excluded from the digest so reissuing the same session/frame does not play it twice. The allocated origin field, once shipped, is included in the canonical payload as well as the explicit framed witness. Never hash unvalidated unknown fields.

Chat and voice use version 1 for each immutable logical id; drawing uses revision, target uses timestamp, and position/heartbeat use observation time. Voice jitter policy still decides playback order for unseen frames. These functions return Apply/Duplicate/Stale/Conflict; they are neither replay caches nor state storage.

Ordered consumer inventory and removal proof

Owner	Migration starting points	Required end state
Common	proto/messages.proto, src/proto/impls.rs, generated Rust/TS, committed dist, wrapper tests, migration-only helpers	#262, merge 587115d09ae6640a3b7ea7f09bd7d78ca3a7c907: non-position wrapper helpers removed; generated wrapper type and Position helper retained
Android	TacNetUtils.kt, TacNetEngine.kt, MessageProcessor.kt, pipeline DecodeStage.kt/DispatchStage.kt, replay/catch-up adapters	Local direct-family implementation exists but is not published: independent review found the full Common validator added acceptance-policy changes, and automatic approval rejected replacing it with the old key/content checks. Android remains on merge d627faad4a6eead76e34f90e0ef5240cc020fbec
Web	drawing publication/subscription/history, plan-bound drawings, target distribution, heartbeat and existing voice paths	#749, merge 3a3923c79f574bdb1b7513b6776e069a46711d1c: affected non-position families direct-only; Position remains wrapped
Gateway	outbound.rs, transport.rs, gateway.rs, inbound.rs, replay metadata and tests	#128, merge 39000e6127e5f7bc025f1a5dee001d9eaa509d8e: drawings direct-only; Position remains wrapped to preserve Gateway origin attribution
Node	src/sender.rs, position/heartbeat constructors and fixtures	#101, merge 0e229a1714ab378211264c03b772170ce86cde27: Heartbeat direct-only; Position remains wrapped
Server	Generic envelope/storage paths	Does not decode chat and is unchanged by #256; remains content-blind with no group key
Directory	app/domains/admin/devices/device_platforms.ts	Does not touch chat and is unchanged by #256; its stale wrapper comment remains #259 cleanup

Run from the sibling-repository workspace after #259 (include source, generated bindings, committed builds, fixtures and comments; exclude dependency/build caches):

rg -n --hidden -g '!.git/**' -g '!node_modules/**' -g '!target/**' \
  'TacNetMessage|tac_net_message|tacNetMessage|buildTacNetMessage|build_tac_net_message|shapeJsonToTacNet|TACNET_REMOVAL_259' \
  common android web gateway node directory

Every hit must be removed or be an explicitly identified historical release note; there must be zero live schema, generated type, constructor, encoder, decoder, metadata adapter, fixture or fallback references. Also inspect decoder decision points (position_payload_decoder, DecodeStage, decode, fromBinary, parseFrom, replay_metadata) for unnamed “direct failed → wrapped” or “wrapped failed → direct” branches; symbol renaming is not removal evidence.

259 records the exact search command/output and final breaking version, and

Docs #210 records each consumer's merged revision before its single final smoke.

258 has not run the live smoke reserved by Docs #210 and has not begun #259,

resumed Docs #205, modified the rack, or regenerated the immutable candidate. Artifact, matrix and release-lock regeneration remain separate work.