SENTO-000 ZeroMQ Transport Analysis

Determine how ZeroMQ¹ should connect Sento² components across processes while preserving Sento actor semantics, per-domain dispatchers³, supervision⁴, bounded overload handling⁵, and explicit security policy.

Affected design:

• SENTO-000 Managed Component Runtime

Reviewed on 2026-07-19:

• libzmq⁶ official repository and API documentation
• ZeroMQ Message Transport Protocol⁷ specifications
• ZeroMQ Authentication Protocol⁸
• CurveZMQ⁹
• ROUTER and DEALER socket behavior
• PUB, SUB, XPUB, and XSUB behavior
• PUSH and PULL behavior
• socket high-water marks¹⁰
• socket monitor API
• steerable proxy API
• Common Lisp PZMQ binding repository
• current Sento actor, dispatcher, router, event-stream, and remoting implementation

ZeroMQ is an optional transport adapter and process bridge.

ZeroMQ does not replace local Sento mailboxes¹¹, domain dispatchers, actor supervision, managed local routers, the component registry, or Star Router¹².

The component-handle layer selects ZeroMQ only when a message must cross a process boundary, reach a non-Sento service, use a ZeroMQ message pattern, or join a ZeroMQ-compatible topology.

A ZeroMQ context¹³ is thread-safe and may be shared by application threads.

Most classic ZeroMQ sockets are not thread-safe. A socket must be created, used, and closed by one owning thread.

A Sento actor on a shared dispatcher may run mailbox handlers on different dispatcher worker threads over time. Such an actor must not directly hold or call a classic ZeroMQ socket.

A pinned Sento actor does have a dedicated mailbox thread, but blocking that receive handler forever in zmq_poll would prevent the actor mailbox from processing lifecycle and control messages.

Use one process-wide libzmq context by default.

The context owns libzmq internal input and output threads and every ZeroMQ socket created by the process.

Each domain may own one reactor, and may split into subdomain reactors when measured load, endpoint ownership, security policy, or independent restart justifies it.

Multiple contexts are possible but duplicate internal resources and complicate shutdown. They require an explicit design reason.

Sento actor system
└── /internal/transports/zeromq
    ├── context-supervisor
    ├── zap-handler
    ├── monitor-collector
    └── reactors
        ├── humint
        ├── socmint
        ├── sigint
        ├── cyber
        ├── bbp
        └── threat-intel

Each reactor contains:

• supervisor child specification
• domain or subdomain identifier
• dedicated reactor thread
• bounded outbound command queue
• socket specifications
• endpoint registry
• pending request table
• serializer
• connection monitor
• transport generation
• metrics

1. A Sento actor sends a transport command to a bridge actor.
2. The bridge resolves the component handle.
3. The bridge checks deadline, payload size, schema, and policy.
4. The bridge enqueues a bounded command for the reactor thread.
5. The reactor thread owns the socket call.
6. The reactor reports success, overload, route failure, or transport failure through Sento messaging.

A successful local enqueue is not proof that the remote component received or processed the message.

1. The reactor poll loop receives multipart frames.
2. It checks frame count and hard byte limits.
3. It decodes only the fixed transport header.
4. It sends a typed transport envelope to the domain ingress actor.
5. The ingress actor checks authorization, deadline, schema, component generation, and destination export.
6. The ingress actor sends the application message to the destination component.
7. Replies use the stored routing identity and correlation identifier.

• inproc://: ZAP, proxy control, socket monitor channels, and internal ZeroMQ-only control inside one process
• ipc://: same-host process communication when supported
• tcp://: communication across hosts and containers

Normal actors inside one Sento process continue to use Sento messages. inproc:// is not a replacement local actor bus.

The ZeroMQ routing identity remains outside the application envelope.

Application frames contain:

1. protocol magic and version
2. message kind
3. message identifier
4. correlation identifier
5. trace identifier
6. source node and component
7. target component export
8. operation or event topic
9. deadline
10. content type and schema identifier
11. flags
12. payload

Supported kinds:

• tell
• ask
• reply
• error
• event
• work
• cancel
• heartbeat

The reactor rejects unsupported versions, wrong frame counts, oversized frames, and malformed fixed fields before payload decoding.

The reviewed Common Lisp PZMQ repository is orivej/pzmq.

Its README describes ZeroMQ 4.0 bindings, warns that functions outside examples and tests may be untested, and states that raw performance was not fully optimized.

The latest reviewed commit is:

6f7b2ca02c23ea53510a9b0e0f181d5364ce9d32
Grovel zmq_pollitem_t (#35)
2021-05-16

ZeroMQ high-water marks bound queued messages per peer.

Depending on socket type and options, a full queue may block, return EAGAIN²⁵, or drop a message.

Recommended settings:

• finite send and receive high-water marks
• finite send timeout or non-blocking send
• finite receive polling interval
• ZMQ_IMMEDIATE on outbound sockets where pre-connection queuing would hide unavailability
• ZMQ_ROUTER_MANDATORY on ROUTER sockets
• finite ZMQ_LINGER so shutdown cannot wait forever
• explicit heartbeat and reconnect policy

Map EAGAIN to a structured overload result.

Map EHOSTUNREACH²⁶ to a structured route-unavailable result.

The full path must bound:

• Sento sender mailbox
• component ingress mailbox
• reactor command queue
• ZeroMQ peer queue
• remote receive queue
• remote ingress mailbox

Use CurveZMQ for encrypted and authenticated tcp:// endpoints.

Run one ZAP handler per process at:

inproc://zeromq.zap.01

ZAP maps a remote public key to a principal²⁷, node, allowed component exports, allowed operations, quota class, and audit metadata.

The ZAP handler starts before server sockets begin accepting peers.

Use filesystem ownership and permissions for ipc:// endpoints, while retaining application-level target and operation policy.

Do not use ZeroMQ PLAIN authentication over an untrusted network because it does not provide confidentiality.

ZeroMQ queues are memory-based. They are not durable storage²⁸.

Default semantics:

• ROUTER/DEALER tell: at-most-once transport attempt
• ROUTER/DEALER ask: at-most-once transport attempt with application correlation
• PUSH/PULL work: at-most-once transport attempt
• PUB/SUB event: best-effort delivery

Automatic retries are disabled by default.

A retry requires an idempotency key²⁹ and duplicate-handling policy.

Durable replay belongs in a database, event log, RabbitMQ³⁰, or another declared durable broker.

zmq_socket_monitor emits socket lifecycle events over an internal PAIR channel.

Relevant events include:

• connected
• connect delayed
• connect retried
• listening
• bind failed
• accepted
• accept failed
• closed
• close failed
• disconnected
• handshake succeeded
• handshake protocol failure
• handshake authentication failure

A monitor collector converts these events into typed Sento runtime events.

Native file descriptors reported by monitor events are transient observations and must not become stable actor identifiers.

The reactor supervisor treats these as failures:

• reactor thread exit
• unexpected context termination
• socket bind failure
• repeated poll failure
• unrecoverable serializer failure
• command queue corruption
• monitor channel failure

Restart sequence:

1. close admission
2. mark transport not ready
3. fail or cancel pending requests
4. close sockets on the owning reactor thread
5. join the reactor thread
6. increase transport generation
7. recreate sockets
8. wait for endpoint and authentication readiness
9. reopen admission

Old-generation replies are rejected as stale.

1. Reject new transport work.
2. Stop subscriptions and external admission.
3. Drain the command queue until the deadline.
4. Resolve or cancel pending requests.
5. Pause and terminate proxies.
6. Close sockets on owner threads with finite linger.
7. Join reactor and monitor threads.
8. Stop ZAP after accepting sockets are closed.
9. Shut down and terminate the ZeroMQ context.
10. Continue normal Sento shutdown.

The context must not terminate while live sockets remain.

• classic sockets are never used from multiple threads
• socket creation and closure occur on owner threads
• malformed multipart input is rejected before decoding
• high-water saturation returns overload results
• mandatory routing returns route errors
• finite linger prevents permanent shutdown blocking
• unauthorized CURVE keys are denied
• authorized keys map to declared principals
• old-generation replies are rejected
• pending asks expire and are removed
• proxy control works under load
• socket monitor events become Sento events
• domain reactor failure does not stop sibling domains
• subdomain reactor restart does not restart the whole process
• no socket, context, monitor, proxy, or reactor thread leaks after shutdown

Implement the ZeroMQ path in this order:

1. narrow CFFI binding
2. context supervisor
3. single domain reactor with command queue
4. ROUTER/DEALER component messaging
5. fixed schema envelope
6. high-water and route errors
7. socket monitoring
8. CURVE and ZAP
9. XPUB/XSUB event proxy
10. optional PUSH/PULL work pipeline
11. component-handle adapter
12. benchmarks against Sento remoting and Star Router

• Retrieved 2026-07-19: https://github.com/zeromq/libzmq
• Retrieved 2026-07-19: https://api.zeromq.org/4-2:zmq-socket
• Retrieved 2026-07-19: https://libzmq.readthedocs.io/en/latest/zmq_socket.html
• Retrieved 2026-07-19: https://libzmq.readthedocs.io/en/latest/zmq_setsockopt.html
• Retrieved 2026-07-19: https://libzmq.readthedocs.io/en/latest/zmq_getsockopt.html
• Retrieved 2026-07-19: https://libzmq.readthedocs.io/en/latest/zmq_socket_monitor.html
• Retrieved 2026-07-19: https://libzmq.readthedocs.io/en/latest/zmq_proxy_steerable.html
• Retrieved 2026-07-19: https://rfc.zeromq.org/spec/23/
• Retrieved 2026-07-19: https://rfc.zeromq.org/spec/25/
• Retrieved 2026-07-19: https://rfc.zeromq.org/spec/26/
• Retrieved 2026-07-19: https://rfc.zeromq.org/spec/27/
• Retrieved 2026-07-19: https://github.com/orivej/pzmq
• Retrieved 2026-07-19: https://github.com/orivej/pzmq/commit/6f7b2ca02c23ea53510a9b0e0f181d5364ce9d32
• Retrieved 2026-07-19: https://github.com/mdbergmann/cl-gserver/blob/a889b8375709ca9396bba5d06b6e460689a23716/src/actor-system.lisp
• Retrieved 2026-07-19: https://github.com/mdbergmann/cl-gserver/blob/a889b8375709ca9396bba5d06b6e460689a23716/src/dispatcher.lisp
• Retrieved 2026-07-19: https://github.com/mdbergmann/cl-gserver/blob/a889b8375709ca9396bba5d06b6e460689a23716/src/router.lisp
• Retrieved 2026-07-19: https://github.com/mdbergmann/cl-gserver/blob/a889b8375709ca9396bba5d06b6e460689a23716/src/remoting/remoting.lisp