SENTO-ZMQ-000 ZeroMQ Transport Runtime

This document extends SENTO-000 Managed Component Runtime.

ZeroMQ¹ is the transport layer² used when a Sento³ message crosses a declared component boundary⁴ that needs routing, isolation, language independence, or process mobility⁵.

ZeroMQ does not replace the ordinary Sento mailbox⁶ used by actors inside one tightly coupled actor tree⁷. Local actor-to-actor messages remain direct Sento messages unless the manifest⁸ explicitly declares a ZeroMQ boundary.

The result is:

Sento actor -> local mailbox -> local actor

Sento actor -> ZMQ gateway actor -> ZeroMQ socket -> ZMQ gateway actor -> Sento actor

The first path is the normal local path. The second path is used at an explicit transport boundary.

Use one ZeroMQ context⁹ per operating-system process¹⁰.

Every ZeroMQ socket¹¹ is owned by exactly one socket-owner actor¹² or one dedicated input/output loop¹³.

Do not allow arbitrary Sento dispatcher threads¹⁴ to call the same ZeroMQ socket. Traditional ZeroMQ socket types are not safe to share across unrelated threads.

Every domain¹⁵ may define:

• one control gateway
• one or more data gateways
• one or more event publishers
• one or more event subscribers
• one or more pipeline producers
• one or more pipeline consumers
• one or more subdomain gateways

Each gateway is supervised by the domain supervisor¹⁶ defined by SENTO-000.

@startuml
skinparam componentStyle rectangle

component "Sento Actor System" as Sento
component "Domain Supervisor" as Supervisor
component "Domain Actors" as Actors
component "ZMQ Gateway Supervisor" as GatewaySupervisor
component "ROUTER Gateway" as Router
component "DEALER Gateway" as Dealer
component "PUB Gateway" as Pub
component "SUB Gateway" as Sub
component "PUSH Gateway" as Push
component "PULL Gateway" as Pull
component "ZeroMQ Context" as Context
component "Remote Process or Subdomain" as Remote

Sento --> Supervisor
Supervisor --> Actors
Supervisor --> GatewaySupervisor
GatewaySupervisor --> Router
GatewaySupervisor --> Dealer
GatewaySupervisor --> Pub
GatewaySupervisor --> Sub
GatewaySupervisor --> Push
GatewaySupervisor --> Pull
Router --> Context
Dealer --> Context
Pub --> Context
Sub --> Context
Push --> Context
Pull --> Context
Context --> Remote
@enduml

The gateway supervisor belongs on the domain control dispatcher¹⁷. Socket input/output work belongs on a dedicated gateway thread or a domain transport dispatcher¹⁸. Application payload processing belongs on the domain or subdomain dispatcher.

Create one process-owned context before creating sockets.

The context configuration includes:

• input/output thread count
• maximum sockets
• maximum message size
• thread name prefix
• blocking termination behavior
• optional Internet Protocol version 6 support

When all endpoints use only inproc://²², the ZeroMQ input/output thread count may be zero. Any ipc://²³ or tcp://²⁴ endpoint requires at least one ZeroMQ input/output thread.

The context is owned by a platform component named zmq-runtime.

The context supervisor is responsible for:

1. creating the context
2. applying context options before socket creation
3. starting platform sockets
4. registering endpoint ownership
5. refusing duplicate bind ownership
6. stopping domain gateways
7. closing every socket
8. terminating the context
9. reporting leaked sockets or blocked termination

The context must be terminated only after all socket owners have acknowledged shutdown.

The context registry stores:

• endpoint identifier
• transport address
• socket type
• bind or connect role
• owning component
• owning actor
• domain
• subdomain
• security policy
• high-water marks
• current state
• monitor actor

A socket-owner actor is the only component allowed to call one socket.

It owns:

• socket creation
• socket options
• bind or connect
• send queue
• receive loop
• multipart frame parsing²⁵
• socket monitor
• reconnect state
• shutdown handshake
• socket close

The actor's Sento mailbox carries control commands and outbound message requests. The socket owner converts those requests into ZeroMQ frames.

Incoming ZeroMQ frames are converted into typed Sento transport envelopes and sent to a local ingress actor²⁶.

Blocking ZeroMQ receive calls must not run on a normal domain dispatcher worker.

Use one of these implementations:

• a pinned Sento actor with one dedicated thread
• a dedicated Bordeaux Threads²⁷ loop controlled by a Sento actor
• a poller actor²⁸ that owns multiple sockets on one dedicated thread

Initial implementation: one dedicated poller thread per domain gateway group, not one thread per socket.

The poller owns all sockets registered to it. Other actors communicate with the poller through command queues or inproc:// control sockets.

A ZeroMQ socket object must never be placed in shared actor state accessible from multiple dispatcher threads.

A socket reference may exist only inside its owning loop.

Use ROUTER²⁹ and DEALER³⁰ for addressed asynchronous commands, requests, replies, and component ingress.

Recommended topology:

component callers -> DEALER -> ROUTER -> component ingress

Use ROUTER at a service or domain boundary that must address multiple connected peers.

Use DEALER for asynchronous clients and workers that may have multiple requests in flight.

Do not use REQ³¹ and REP³² for the primary runtime protocol because their strict send-receive lockstep makes cancellation, multiplexing, and concurrent requests harder.

The ROUTER identity frame maps to a transport peer identity, not directly to a StarIntel user identity.

Use PUB³³ and SUB³⁴ for telemetry, status broadcasts, cache invalidations, manifest announcements, and other events where subscribers may legitimately miss earlier messages.

Do not use PUB/SUB for:

• commands that must be processed
• evidence-bearing state changes
• durable queues
• requests requiring a reply
• events that must survive subscriber restart

PUB/SUB has no built-in replay. A subscriber joining late misses earlier messages. A slow subscriber may also lose messages when queues reach their configured limits.

Reliable event distribution must add a replay source or use ROUTER/DEALER with acknowledgements.

Use PUSH³⁵ and PULL³⁶ for one-way work pipelines where any one equivalent worker may process the message and no direct reply is required.

Examples:

• independent parsing jobs
• attachment conversion
• stateless enrichment
• bulk normalization

Do not use PUSH/PULL for keyed state or target-affine work because ZeroMQ load balancing does not preserve application keys.

Use PAIR³⁷ only for exclusive internal control links such as:

• stopping one poller thread
• receiving socket monitor events
• waking one gateway loop

Do not use PAIR as a general service transport.

Use XPUB³⁸ and XSUB³⁹ only when implementing a managed publish-subscribe proxy⁴⁰ that must observe subscription changes.

This belongs in a later phase after ordinary PUB/SUB behavior is tested.

Use inproc:// when both socket owners are inside the same operating-system process and ZeroMQ semantics are required.

Examples:

• gateway-to-gateway control
• poller wakeup
• explicit subdomain boundary
• high-volume fan-out inside one process

All inproc:// sockets must share the same ZeroMQ context.

Use ipc:// for processes on the same Unix-like host⁴¹.

Requirements:

• endpoint path under a managed runtime directory
• restrictive file permissions
• stale socket cleanup
• ownership validation
• no world-writable endpoint directory

Use tcp:// for containers, virtual machines, or physical hosts.

Requirements:

• explicit bind address
• no default public bind
• configured security mechanism
• peer authentication
• connection monitoring
• message-size limits
• heartbeat policy

• protocol version
• message identifier
• message type
• source component
• destination component
• domain
• subdomain when present
• trace identifier
• creation time
• deadline when present
• content type
• payload length

• correlation identifier
• reply endpoint
• idempotency key⁴³
• attempt number
• tenant identifier
• routing key
• capability requirement

When a non-blocking ZeroMQ send reports unavailable capacity, the socket owner converts it into the structured overload result from SENTO-000.

The socket owner must not block a domain control dispatcher waiting indefinitely for socket capacity.

Allowed policies:

• reject immediately
• retry until message deadline
• place in a bounded local outbound queue
• shed low-priority messages
• fail the request with overload status

PUB and ROUTER sockets may drop messages under some high-water-mark conditions. Any route that cannot tolerate loss must use acknowledgements or another pattern.

Use CURVE⁵⁰ for authenticated encryption on ZeroMQ TCP endpoints unless the endpoint is protected by another explicitly documented transport-security layer.

Use ZAP⁵¹ or an equivalent application authentication handler to map remote keys to principals⁵² and capabilities.

The gateway must authorize:

• source principal
• destination component
• message type
• domain
• subdomain
• payload size
• current quota

For ipc:// endpoints:

• use a private runtime directory
• set restrictive permissions
• reject symlinked endpoint paths
• remove stale socket files safely
• validate owner and group

inproc:// is not an authorization boundary. It is only an addressing and queueing mechanism inside one process.

Before delivering a network message to a Sento actor:

1. validate frame count
2. validate each frame size
3. validate protocol version
4. validate content type
5. validate schema
6. validate deadline
7. validate source identity
8. authorize destination
9. apply admission control
10. deserialize payload

Malformed messages never enter a domain actor mailbox.

• select and pin Common Lisp binding
• record libzmq version
• build a feature matrix
• run minimal socket-pattern tests
• test context shutdown

• implement process-wide context component
• implement endpoint registry
• implement socket-owner actor
• implement dedicated poller loop
• implement controlled shutdown

• implement framed command protocol
• implement request correlation
• implement deadlines
• implement structured transport errors
• implement component-handle resolution

• configure high-water marks
• map non-blocking send failures
• add bounded outbound queues
• add admission control
• test saturated peers

• add socket monitors
• add typed transport events
• add reconnect health state
• add restart generation
• add failure injection

• implement lossy event bus
• implement work pipelines
• document reliability limits
• add replay integration where required

• implement CURVE
• implement ZAP policy
• add IPC filesystem controls
• add principal and capability checks

• implement Star Router protocol over ROUTER/DEALER
• expose logical component routes
• connect heterogeneous services
• test process and host mobility