fpc-msgbase/README.md

# fpc-msgbase

A unified Free Pascal library for reading and writing classic BBS message bases.

Implements every supported format from the FTSC specifications and the original
format authors' published documentation, behind one polymorphic API
(`TMessageBase`). BBS software, mail tossers, message editors, and utilities
can target a single interface regardless of the underlying format on disk.

## Supported formats

| Format    | Files                                | Backend unit              |
|-----------|--------------------------------------|---------------------------|
| Hudson    | MSGINFO/IDX/HDR/TXT/TOIDX.BBS        | `ma.fmt.hudson.pas`       |
| JAM       | `*.JHR` `*.JDT` `*.JDX` `*.JLR`      | `ma.fmt.jam.pas`          |
| Squish    | `*.SQD` `*.SQI` `*.SQL`              | `ma.fmt.squish.pas`       |
| FTS-1 MSG | numbered `*.MSG` per directory       | `ma.fmt.msg.pas`          |
| PCBoard   | `*.MSG` + `*.IDX`                    | `ma.fmt.pcboard.pas`      |
| EzyCom    | `MH#####.BBS` / `MT#####.BBS`        | `ma.fmt.ezycom.pas`       |
| GoldBase  | MSGINFO/IDX/HDR/TXT/TOIDX.DAT        | `ma.fmt.goldbase.pas`     |
| Wildcat 4 | WC SDK databases                     | `ma.fmt.wildcat.pas`      |

**FTN PKT** is a transport-tier wire format and lives in the
sibling [`fpc-ftn-transport`](../fpc-ftn-transport/) library
(units `tt.pkt.reader`, `tt.pkt.writer`, `tt.pkt.format`,
`tt.pkt.batch`). The `mbfPkt` factory enum stays in fpc-msgbase
so consumers wanting to iterate `.pkt` files via the unified API
(`MessageBaseOpen(mbfPkt, ...)`) just add `uses tt.pkt.reader`
alongside fpc-msgbase. PKT was here through 0.3.x; moved out
in 0.4.0 because PKT is a wire format, not a storage format.

## Features

- One `TMessageBase` abstract class — read, write, pack, reindex through the
  same methods regardless of format.
- **Lossless two-area message model.** `TUniMessage` = `Body` (just the
  message text) + `Attributes` (key/value bag holding from/to/subject/dates/
  addresses/MSGID/SEEN-BY/PATH and per-format extras). Same shape as RFC 822
  email. Round-trip preservation enforced by the test suite.
- **Capabilities API** — `base.SupportsAttribute('attr.returnreceipt')` lets
  UIs hide controls the underlying backend has no slot for. Each backend
  publishes its key list via `ClassSupportedAttributes`. Full per-format
  matrix in [`docs/attributes-registry.md`](docs/attributes-registry.md).
- **Per-user High-Water Mark** — `base.GetHWM('NetReader')` /
  `base.SetHWM(...)` plus auto-bump via `base.ActiveUser`. Native
  for JAM (`.JLR`), Squish (`.SQL`), Hudson + GoldBase
  (`LASTREAD.BBS/DAT`, with `MapUser` + `Board` context). Tossers
  and scanners register as named users in the format's native
  lastread file, so multiple consumers coexist without colliding.
  Unsupported formats return -1 honestly.
- Layered locking: in-process `TRTLCriticalSection` + cross-process advisory
  lock (`fpflock` on Unix, `LockFileEx` on Windows, `.LCK` sentinel fallback)
  + the existing `fmShareDenyWrite` / `fmShareDenyNone` share modes.
- Event hooks for logging, progress, and status reporting.
- Path / filename auto-derivation per format from a base directory plus
  optional area tag (`area` attribute auto-populated on Read).
- **Shared FTSC kludge plumbing** in `ma.kludge` — single source of truth
  for kludge-line parse/emit (`ParseKludgeLine`, `SplitKludgeBlob`,
  `BuildKludgePrefix/Suffix`). Unknown FTSC kludges round-trip uniformly
  as `kludge.<lowername>` regardless of which backend stored them, so
  consumers don't switch on format to find passthrough kludges.

## Building

**Use both the native and `.uni` adapter units in your `uses` clause** —
the `.uni` adapter's `initialization` block is what registers the backend
with the unified-API factory. Forgetting it produces
`EMessageBase: No backend registered for <format>`.

```pascal
uses
  ma.types, ma.events, ma.api,
  ma.fmt.jam, ma.fmt.jam.uni;     { both — .uni registers }
```

Native Linux:

```sh
fpc -Fusrc -Fusrc/formats examples/example_read.pas
```

Lazarus package:

```sh
lazbuild fpc-msgbase.lpk
```

The repo includes a `fpc.cfg` template covering the multi-target build
(`i386-go32v2`, `i386-win32`, `i386-linux`, `i386-os2`).

## Layout

```
src/                ma.api, ma.types, ma.events, ma.lock, ma.paths, ma.kludge
src/formats/        ma.fmt.<format>.pas — one per supported format
docs/               architecture, locking semantics, format notes
tests/              FPCUnit tests, sample data
examples/           small CLI programs that double as smoke tests
```

Unit-name convention follows the Fimail style: `ma.<category>.<name>.pas`
(`ma.` is the library's namespace prefix; the project was originally named
`message_api`, hence `ma`). All units use `{$mode objfpc}{$H+}`.

## Documentation

- [`docs/API.md`](docs/API.md) — full API reference with examples
- [`docs/architecture.md`](docs/architecture.md) — layered design + Body/Attributes contract
- [`docs/attributes-registry.md`](docs/attributes-registry.md) — canonical attribute keys + per-format support matrix
- [`docs/ftsc-compliance.md`](docs/ftsc-compliance.md) — spec notes
- [`docs/format-notes/`](docs/format-notes/) — per-format quirks

## Status

Early development. APIs may move until 1.0.

**Current: 0.5.2** — see [`CHANGELOG.md`](CHANGELOG.md) for the full
release history.  Downstream consumers should pin by tag
(`v0.5.2`), not commit hash.

**0.2 is a breaking change vs 0.1.** `TUniMessage` lost its 13 named
fields (WhoFrom/WhoTo/Subject/MsgNum/Attr/etc.) in favour of a strict
`Body` + `Attributes` two-area model. Migration:

```pascal
{ before }                       { after }
msg.WhoFrom                       msg.Attributes.Get('from')
msg.Subject := 'foo';             msg.Attributes.SetValue('subject', 'foo');
msg.Attr := MSG_ATTR_LOCAL;       msg.Attributes.SetBool('attr.local', true);
msg.OrigAddr                      msg.Attributes.GetAddr('addr.orig')
msg.DateWritten                   msg.Attributes.GetDate('date.written')
```

This change makes the unified API lossless — kludges (MSGID, SEEN-BY,
PATH, etc.) round-trip cleanly, where 0.1 silently dropped them. See
[`docs/attributes-registry.md`](docs/attributes-registry.md) for the
full key catalog.

Format backends are spec-driven implementations validated against
real-world sample bases.