fpc-msgbase/docs/API.md

fpc-msgbase — API reference

Every callable the library exposes, with parameters, return values, and runnable examples. See docs/architecture.md for the big picture and docs/ftsc-compliance.md for spec notes.

Contents

• Quick start
• Opening a base
• TUniMessage
• Reading
• Writing
• Updating & deleting
• Packing & reindexing
• Events & logging
• Locking
• Path helpers
• Concurrent tossers — TPacketBatch
• Dropping to a native backend
• Format cheat-sheet

---

Quick start

program hello;
{$mode objfpc}{$H+}
uses
  SysUtils,
  ma.types, ma.events, ma.api,
  ma.fmt.jam, ma.fmt.jam.uni;

var
  base: TMessageBase;
  msg:  TUniMessage;
  i:    longint;
begin
  base := MessageBaseOpen(mbfJam,
                          '/home/ken/fidonet/msg/jam/10thamd',
                          momReadOnly);
  try
    if not base.Open then Halt(1);
    for i := 0 to base.MessageCount - 1 do
      if base.ReadMessage(i, msg) then
        WriteLn(msg.WhoFrom, ' -> ', msg.WhoTo, ': ', msg.Subject);
  finally
    base.Close;
    base.Free;
  end;
end.

Add format support by including the matching pair of units:

Format	Units to add to uses
Hudson	ma.fmt.hudson, ma.fmt.hudson.uni
JAM	ma.fmt.jam, ma.fmt.jam.uni
Squish	ma.fmt.squish, ma.fmt.squish.uni
FTS-1 MSG	ma.fmt.msg, ma.fmt.msg.uni
FTN PKT	ma.fmt.pkt, ma.fmt.pkt.uni
PCBoard	ma.fmt.pcboard, ma.fmt.pcboard.uni
EzyCom	ma.fmt.ezycom, ma.fmt.ezycom.uni
GoldBase	ma.fmt.goldbase, ma.fmt.goldbase.uni
Wildcat	ma.fmt.wildcat, ma.fmt.wildcat.uni

The .uni unit's initialization section registers the factory; without it, MessageBaseOpen(<format>, ...) raises EMessageBase.

Threads on Unix need cthreads first in the program's uses:

uses
  {$IFDEF UNIX}cthreads,{$ENDIF}
  ma.api, ...;

---

Opening a base

MessageBaseOpen

function MessageBaseOpen(AFormat: TMsgBaseFormat;
                         const AAreaPath: AnsiString;
                         AMode: TMsgOpenMode;
                         const AAreaTag: AnsiString = ''): TMessageBase;

Returns a not-yet-opened TMessageBase for the requested format. Call .Open before reading/writing. Raises EMessageBase if the format has no registered backend.

Parameter	Meaning
AFormat	mbfHudson / mbfJam / mbfSquish / etc.
AAreaPath	Directory for dir-based formats, basename for JAM/Squish/PCB, full filename for PKT.
AMode	momReadOnly, momReadWrite, or momCreate.
AAreaTag	Optional area tag; passed through to the adapter.

MessageBaseOpenAuto

function MessageBaseOpenAuto(const AAreaPath: AnsiString;
                             AMode: TMsgOpenMode): TMessageBase;

Sniffs the directory for signature files (MSGINFO.BBS, *.JHR, *.SQD, numbered *.MSG, …) and picks the right backend. Returns nil when no format matches.

DetectFormat

function DetectFormat(const AAreaPath: AnsiString;
                      out AFormat: TMsgBaseFormat): boolean;

Same sniffer as MessageBaseOpenAuto, but just returns the format without opening.

TMessageBase.Open / Close

function  Open: boolean;
procedure Close;

Open acquires the cross-process sentinel lock (exclusive for writers, shared for readers), then delegates to the backend's native open. Fires metBaseOpened. Returns False if the base files are missing or another writer is holding the lock.

Close delegates to the backend, releases the sentinel lock, unlinks the .lck file, and fires metBaseClosed.

Mode semantics

Mode	Behaviour
momReadOnly	Read-only. No write/pack allowed. Shared lock, failures tolerated (e.g. RO FS).
momReadWrite	Files must exist; exclusive lock.
momCreate	Creates empty format files if missing, then opens exclusive.

---

TUniMessage

Single format-agnostic message record. Backends convert between their native record and this on each read/write.

TUniMessage = record
  MsgNum:       longint;        { backend-assigned number }
  WhoFrom:      AnsiString;
  WhoTo:        AnsiString;
  Subject:      AnsiString;
  DateWritten:  TDateTime;
  DateReceived: TDateTime;      { 0 if not received }
  Attr:         cardinal;       { MSG_ATTR_* bitset, see below }
  OrigAddr:     TFTNAddress;
  DestAddr:     TFTNAddress;
  Cost:         word;
  Body:         AnsiString;     { CR-separated, kludges intact }
  AreaTag:      AnsiString;     { optional echo-area tag }
  Board:        word;           { conference/board; 0 = default }
end;

Canonical attribute bits (ma.types)

MSG_ATTR_PRIVATE        $00000001    MSG_ATTR_DELETED       $00010000
MSG_ATTR_CRASH          $00000002    MSG_ATTR_READ          $00020000
MSG_ATTR_RECEIVED       $00000004    MSG_ATTR_ECHO          $00040000
MSG_ATTR_SENT           $00000008    MSG_ATTR_DIRECT        $00080000
MSG_ATTR_FILE_ATTACHED  $00000010    MSG_ATTR_IMMEDIATE     $00100000
MSG_ATTR_IN_TRANSIT     $00000020    MSG_ATTR_FREQ_PENDING  $00200000
MSG_ATTR_ORPHAN         $00000040    MSG_ATTR_SEEN          $00400000
MSG_ATTR_KILL_SENT      $00000080    MSG_ATTR_LOCKED        $00800000
MSG_ATTR_LOCAL          $00000100    MSG_ATTR_NO_KILL       $01000000
MSG_ATTR_HOLD           $00000200    MSG_ATTR_NET_PENDING   $02000000
MSG_ATTR_FILE_REQUEST   $00000400    MSG_ATTR_ECHO_PENDING  $04000000
MSG_ATTR_RETURN_RCPT    $00000800
MSG_ATTR_IS_RECEIPT     $00001000
MSG_ATTR_AUDIT_REQ      $00002000
MSG_ATTR_FILE_UPD_REQ   $00004000

Bits 0..15 match FTS-0001 exactly; 16+ are for storage-layer flags (deleted/read/echo/etc.) that aren't part of the FTN wire format.

FTN addressing

TFTNAddress = record Zone, Net, Node, Point: word; end;

function MakeFTNAddress(AZone, ANet, ANode, APoint: word): TFTNAddress;
function FTNAddressToString(const A: TFTNAddress): string;       { "1:100/200.3" }
function ParseFTNAddress(const S: string; out A: TFTNAddress): boolean;
function FTNAddressEqual(const A, B: TFTNAddress): boolean;

---

Reading

function ReadMessage(Index: longint; var Msg: TUniMessage): boolean;
property MessageCount: longint;

Zero-based index. Returns False on EOF or backend failure. Message numbers (Msg.MsgNum) are backend-assigned and typically don't match the index (most formats keep a gap-tolerant index). Fires metMessageRead on success.

for i := 0 to base.MessageCount - 1 do
  if base.ReadMessage(i, msg) then
    Handle(msg);

---

Writing

function WriteMessage(var Msg: TUniMessage): boolean;

Appends a new message. Backend assigns Msg.MsgNum on success. Fires metMessageWritten. Raises EMessageBase in read-only mode.

msg.WhoFrom     := 'Sysop';
msg.WhoTo       := 'All';
msg.Subject     := 'Hello';
msg.DateWritten := Now;
msg.Attr        := MSG_ATTR_LOCAL or MSG_ATTR_ECHO;
msg.OrigAddr    := MakeFTNAddress(1, 100, 1, 0);
msg.DestAddr    := MakeFTNAddress(1, 100, 2, 0);
msg.Body        := 'Hello, world' + #13;
base.WriteMessage(msg);

---

Updating & deleting

function UpdateMessage(Index: longint; var Msg: TUniMessage): boolean;
function DeleteMessage(Index: longint): boolean;

Optional operations. Not every backend implements them — the default returns False. Attribute-only changes can also be made through the native API (.Native) by flipping the MSG_ATTR_DELETED bit and calling Pack; that pattern works on every backend.

---

Packing & reindexing

function Pack(PurgeAgeDays, PurgeMaxCount: longint;
              Backup: boolean = True): boolean;
function ReIndex: boolean;

Pack purges deleted/old messages and rebuilds the physical files. PurgeAgeDays = 0 and PurgeMaxCount = 0 means "don't purge by age or count; just compact deleted messages." Fires metPackStarted / metPackComplete.

ReIndex rebuilds the index file from the header file (JAM/Hudson).

{ no-op pack: only drop anything already marked deleted }
base.Pack(0, 0, False);

{ purge anything older than 90 days, keep at most 500 messages }
base.Pack(90, 500, True);

---

Events & logging

Every TMessageBase owns a TMessageEvents reachable as .Events. Subscribe with either a log-shaped handler or a full-info handler, or both. Multiple hooks can be registered with AddHook.

TMessageLogHandler =
  procedure(Level: TMsgEventType;
            const Source, Msg: AnsiString) of object;

TMessageEventHandler =
  procedure(Sender: TObject;
            const Info: TMessageEventInfo) of object;

Example — console logger

type
  TLogger = class
    procedure OnLog(Level: TMsgEventType;
                    const Source, Msg: AnsiString);
  end;

procedure TLogger.OnLog(Level: TMsgEventType;
                        const Source, Msg: AnsiString);
begin
  WriteLn('[', EventTypeToStr(Level), '] ', Source, ': ', Msg);
end;

var
  log: TLogger;
begin
  log := TLogger.Create;
  base.Events.OnLog := @log.OnLog;
  ...

Event types (TMsgEventType)

Group	Values
Log levels	metInfo, metWarning, metError, metDebug
Lifecycle	metBaseOpened, metBaseClosed, metBaseCreated
Messages	metMessageRead, metMessageWritten, metMessageDeleted, metMessageUpdated
Locking	metLockAcquired, metLockReleased, metLockTimeout, metLockFailed
Pack	metPackStarted, metPackProgress, metPackComplete, metReindexStarted, metReindexComplete
Packets	metPacketStart, metPacketEnd, metPacketMessage, metPacketError

Filter by severity:

base.Events.MinLevel := metWarning;  { drop info + debug }

---

Locking

Three layers, all in ma.lock:

1. TRTLCriticalSection per TMessageBase instance — serialises concurrent Read/Write/Update/Delete calls on the same instance.
2. Advisory sentinel — fpflock (Unix), LockFileEx (Windows), DosSetFileLocks (OS/2), exclusive-open (DOS) on a per-base .lck file. Released and unlinked on close.
3. Native share modes — each backend opens its data streams with fmShareDenyWrite / fmShareDenyNone as the backstop.

The sentinel timeout is configurable per instance:

base.LockTimeoutMs := 5000;   { retry for 5 s, fail otherwise }
base.LockTimeoutMs := 0;      { fail fast — don't wait at all }
base.LockTimeoutMs := -1;     { wait forever }

Direct access: base.Lock returns the TMessageLock for the rare case where you want to hold the lock across multiple operations.

---

Path helpers

function MessageBasePathFor(AFormat: TMsgBaseFormat;
                            const AAreaPath, AAreaTag: AnsiString): AnsiString;
function FindExistingFile(const APath: AnsiString): AnsiString;
function PathJoin(const ADir, ATail: AnsiString): AnsiString;
function LockFilePath(AFormat: TMsgBaseFormat;
                      const ABasePath, AAreaTag: AnsiString): AnsiString;

MessageBasePathFor produces the canonical constructor argument each backend expects from an area directory + optional tag. FindExistingFile does case-insensitive resolution (path → UPPER → lower) for Linux hosts where on-disk names may be mixed case.

---

Concurrent tossers — TPacketBatch

TPacketBatch = class
  constructor Create(const APacketDir: AnsiString; AThreadCount: integer = 4);
  function  Run: longint;                 { returns packet count }
  procedure RequestStop;
  function  GetOrCreateBase(AFormat: TMsgBaseFormat;
                            const APath: AnsiString): TMessageBase;
  property  PacketDir:   AnsiString;
  property  PacketMask:  AnsiString;      { default '*.pkt' }
  property  ThreadCount: integer;
  property  Processor:   TPacketProcessor;
  property  Events:      TMessageEvents;
end;

TPacketProcessor =
  procedure(const APacketPath: AnsiString;
            var Msg: TUniMessage;
            var Stop: boolean) of object;

Run scans PacketDir for PacketMask, spawns ThreadCount workers, and blocks until every packet has been processed. Each worker opens one packet at a time (TPktFile), hands each message to Processor. Writes to destination bases should go through GetOrCreateBase so the batch caches and serialises them.

Minimal tosser

uses {$IFDEF UNIX}cthreads,{$ENDIF} ma.api, ma.batch, ...;

type
  TTosser = class
    Batch: TPacketBatch;
    procedure OnMsg(const Path: AnsiString;
                    var Msg: TUniMessage;
                    var Stop: boolean);
  end;

procedure TTosser.OnMsg(const Path: AnsiString;
                        var Msg: TUniMessage;
                        var Stop: boolean);
var
  base: TMessageBase;
begin
  base := Batch.GetOrCreateBase(mbfJam, '/msg/echo/' + Msg.AreaTag);
  if base <> nil then
    base.WriteMessage(Msg);
end;

var
  t: TTosser;
begin
  t := TTosser.Create;
  t.Batch := TPacketBatch.Create('/inbound', 4);
  t.Batch.Processor := @t.OnMsg;
  t.Batch.Run;
  ...
end.

---

Dropping to a native backend

Every .uni adapter exposes Native for callers who need format- specific features the unified API doesn't abstract (JAM subfields, Squish UMsgId, EzyCom dual-byte attributes, Wildcat NextMsg walking, etc.):

var
  adapter: TJamMessageBase;
  hdr:     JamHdr;
  nat:     TJamMessage;
begin
  adapter := base as TJamMessageBase;
  adapter.Native.ReadMessage(Index, nat);
  adapter.Native.ReadHeader(nat.HdrOffset, hdr);
  hdr.Attribute := hdr.Attribute or longint($80000000);   { mark deleted }
  adapter.Native.UpdateHeader(nat.HdrOffset, hdr);
  adapter.Native.IncModCounter;
  adapter.Native.UpdateHdrInfo;
end;

Native class names:

Format	Native class	Adapter
Hudson	THudsonBase	THudsonMessageBase
JAM	TJamBase	TJamMessageBase
Squish	TSquishBase	TSquishMessageBase
FTS-1 MSG	TMsgDir+TMsgFile	TMsgMessageBase
FTN PKT	TPktFile	TPktMessageBase
PCBoard	TPCBoardBase	TPCBoardMessageBase
EzyCom	TEzyComBase	TEzyComMessageBase
GoldBase	TGoldBase	TGoldBaseMessageBase
Wildcat	TWildcatBase	TWildcatMessageBase

---

Format cheat-sheet

Format	Base path shape	Caveats
Hudson	directory	32767-message ceiling (smallint MsgNum)
GoldBase	directory	Widened Hudson (word MsgAttr, 500 boards)
JAM	dir + basename, no ext	Adapter appends .JHR/.JDT/.JDX/.JLR
Squish	dir + basename, no ext	Adapter appends .SQD/.SQI/.SQL
FTS-1 MSG	directory	Numbered *.msg / *.MSG (case mixed)
FTN PKT	full packet filename	Stream-only; WriteMessage not supported through adapter — use Native
PCBoard	dir + basename, no ext	Adapter appends .MSG/.IDX
EzyCom	directory + .Board	Set adapter.Board/.BBSType before Open
Wildcat	WC data dir + .Conference	Set adapter.Conference before Open

---

See also

• docs/architecture.md — layered design
• docs/ftsc-compliance.md — spec refs
• docs/format-notes/ — per-format quirks and dependencies
• examples/ — runnable example_read, example_write, example_tosser
• tests/ — test_read, test_roundtrip, test_lock, test_batch, test_wildcat, test_write_existing, test_pack