fpc-msgbase/docs/architecture.md

fpc-msgbase — architecture

Layers

        ┌──────────────────────────────────────────────────┐
        │  Caller (BBS, tosser, editor, importer, …)       │
        └──────────────────────────────────────────────────┘
                              │
                              ▼
        ┌──────────────────────────────────────────────────┐
        │  ma.api (TMessageBase, factory, TUniMessage)     │
        ├──────────────────────────────────────────────────┤
        │  ma.events   ma.lock   ma.paths                  │
        │  ma.batch (concurrent tosser helper)             │
        ├──────────────────────────────────────────────────┤
        │  Format backends — one .pas per format           │
        │  ma.fmt.hudson   ma.fmt.jam      ma.fmt.squish   │
        │  ma.fmt.msg      ma.fmt.pkt      ma.fmt.pcboard  │
        │  ma.fmt.ezycom   ma.fmt.goldbase ma.fmt.wildcat  │
        ├──────────────────────────────────────────────────┤
        │  RTL: TFileStream, BaseUnix/Windows for locking  │
        └──────────────────────────────────────────────────┘

Polymorphism

Every backend descends from TMessageBase and implements the abstract DoOpen, DoClose, DoMessageCount, DoReadMessage, DoWriteMessage contract. Callers can either:

1. Use the unified API — MessageBaseOpen(format, path, mode) returns a TMessageBase. Read/write through TUniMessage. Format-agnostic.
2. Drop down to format-specific class methods (e.g. TJamBase.IncModCounter, TSquishBase.SqHashName) when they need behaviour the unified API cannot express. Each backend keeps its rich API public.

TUniMessage

A single message record covering every backend. Format-specific bit fields (Hudson byte attr, JAM 32-bit attr, Squish attr, MSG word attr, PCB status, EzyCom dual byte) are mapped into a canonical Attr: cardinal bit set defined in ma.types (MSG_ATTR_PRIVATE, MSG_ATTR_LOCAL, MSG_ATTR_RECEIVED, …). Conversion helpers expose the original encoding when needed.

Dates land in TDateTime regardless of how the backend stored them (Hudson MM-DD-YY strings with 1950 pivot, Squish FTS-0001 strings, JAM Unix timestamps, PCBoard / EzyCom DOS PackTime).

The body is an AnsiString with kludge lines intact and CR-separated.

Locking

Three layers, applied in order on every Open:

1. In-process — TRTLCriticalSection per TMessageBase instance.
2. Cross-process — advisory lock on a sentinel file (<base>.lck or, for Squish, <base>.SQL so we coexist with other Squish-aware tools). fpflock(LOCK_EX|LOCK_SH) on Unix, LockFileEx on Windows. Retry with backoff up to a configurable timeout (default 30s). Lock acquire/release fires events.
3. OS share modes — fmShareDenyWrite for writers, fmShareDenyNone for readers, matching DOS-era multi-process sharing conventions every classic format expects.

Events

TMessageEvents lets callers subscribe one or more handlers to receive metBaseOpened, metMessageRead, metMessageWritten, metLockAcquired, metPackProgress, etc. Internally the dispatcher serialises calls so handlers do not need to be reentrant.

Concurrent tossers

TPacketBatch owns a queue of .pkt paths and a worker thread pool. Each worker opens its packet, reads messages, hands each to the caller-provided processor. The batch caches one TMessageBase per destination area so writes serialise through layer-1 locking; layer-2 keeps separate processes (e.g. an editor) safe at the same time.

Behavioural fidelity

Every format backend is implemented from the published format specification (FTSC documents and the original format authors' own spec papers — see docs/ftsc-compliance.md). Tests read and write real sample bases captured from working BBS installations; round-trip tests verify byte-for-byte preservation across read → write → read cycles.