diff --git a/README.md b/README.md index a2d3f59..b151578 100644 --- a/README.md +++ b/README.md @@ -25,6 +25,14 @@ can target a single interface regardless of the underlying format on disk. - One `TMessageBase` abstract class — read, write, pack, reindex through the same methods regardless of format. +- **Lossless two-area message model.** `TUniMessage` = `Body` (just the + message text) + `Attributes` (key/value bag holding from/to/subject/dates/ + addresses/MSGID/SEEN-BY/PATH and per-format extras). Same shape as RFC 822 + email. Round-trip preservation enforced by the test suite. +- **Capabilities API** — `base.SupportsAttribute('attr.returnreceipt')` lets + UIs hide controls the underlying backend has no slot for. Each backend + publishes its key list via `ClassSupportedAttributes`. Full per-format + matrix in [`docs/attributes-registry.md`](docs/attributes-registry.md). - Layered locking: in-process `TRTLCriticalSection` + cross-process advisory lock (`fpflock` on Unix, `LockFileEx` on Windows, `.LCK` sentinel fallback) + the existing `fmShareDenyWrite` / `fmShareDenyNone` share modes. @@ -68,11 +76,32 @@ Unit-name convention follows the Fimail style: `ma...pas` ## Documentation - [`docs/API.md`](docs/API.md) — full API reference with examples -- [`docs/architecture.md`](docs/architecture.md) — layered design +- [`docs/architecture.md`](docs/architecture.md) — layered design + Body/Attributes contract +- [`docs/attributes-registry.md`](docs/attributes-registry.md) — canonical attribute keys + per-format support matrix - [`docs/ftsc-compliance.md`](docs/ftsc-compliance.md) — spec notes - [`docs/format-notes/`](docs/format-notes/) — per-format quirks ## Status -Early development. APIs may move until 1.0. Format backends are -spec-driven implementations validated against real-world sample bases. +Early development. APIs may move until 1.0. + +**0.2 is a breaking change vs 0.1.** `TUniMessage` lost its 13 named +fields (WhoFrom/WhoTo/Subject/MsgNum/Attr/etc.) in favour of a strict +`Body` + `Attributes` two-area model. Migration: + +```pascal +{ before } { after } +msg.WhoFrom msg.Attributes.Get('from') +msg.Subject := 'foo'; msg.Attributes.SetValue('subject', 'foo'); +msg.Attr := MSG_ATTR_LOCAL; msg.Attributes.SetBool('attr.local', true); +msg.OrigAddr msg.Attributes.GetAddr('addr.orig') +msg.DateWritten msg.Attributes.GetDate('date.written') +``` + +This change makes the unified API lossless — kludges (MSGID, SEEN-BY, +PATH, etc.) round-trip cleanly, where 0.1 silently dropped them. See +[`docs/attributes-registry.md`](docs/attributes-registry.md) for the +full key catalog. + +Format backends are spec-driven implementations validated against +real-world sample bases. diff --git a/docs/PROPOSAL.md b/docs/PROPOSAL.md index 8738a64..051a3c0 100644 --- a/docs/PROPOSAL.md +++ b/docs/PROPOSAL.md @@ -134,6 +134,20 @@ Caller (BBS, tosser, editor, importer) ### `TMsgRecord` with lossless `Extras` +> **SUPERSEDED 2026-04-17.** The "named fields + Extras bag" hybrid +> below was rejected during implementation in favour of a stricter +> two-area model: `Body` holds only the message text; **everything +> else** (from/to/subject/dates/addresses + every kludge + every +> format-specific field) is an attribute. See +> [`docs/attributes-registry.md`](attributes-registry.md) for the +> key catalog and [`docs/architecture.md`](architecture.md) for the +> updated TUniMessage contract. The original Extras-bag design is +> retained below as historical context. +> +> The capabilities API discussed in this section landed essentially +> as proposed (`base.SupportsAttribute(K)` + class-level +> `ClassSupportedAttributes`). + ```pascal TMsgRecord = record { Universal fields every format has: } diff --git a/docs/architecture.md b/docs/architecture.md index 800496f..e2a2c56 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -35,20 +35,72 @@ contract. Callers can either: `TSquishBase.SqHashName`) when they need behaviour the unified API cannot express. Each backend keeps its rich API public. -## TUniMessage +## TUniMessage — two-area model -A single message record covering every backend. Format-specific bit fields -(Hudson byte attr, JAM 32-bit attr, Squish attr, MSG word attr, PCB status, -EzyCom dual byte) are mapped into a canonical `Attr: cardinal` bit set -defined in `ma.types` (`MSG_ATTR_PRIVATE`, `MSG_ATTR_LOCAL`, -`MSG_ATTR_RECEIVED`, …). Conversion helpers expose the original encoding -when needed. +```pascal +TUniMessage = record + Body: AnsiString; { only the message text } + Attributes: TMsgAttributes; { everything else, key/value } +end; +``` -Dates land in `TDateTime` regardless of how the backend stored them -(Hudson `MM-DD-YY` strings with 1950 pivot, Squish FTS-0001 strings, JAM -Unix timestamps, PCBoard / EzyCom DOS PackTime). +Two areas, no surprises: -The body is an `AnsiString` with kludge lines intact and CR-separated. +- **Body** carries the user-visible message text and nothing else. + Never kludge lines, never headers, never SEEN-BY/PATH. Always a + ready-to-display blob. +- **Attributes** carries every other piece of data: From, To, + Subject, dates, addresses, attribute bits, FTSC kludges (MSGID, + ReplyID, PID, SEEN-BY, PATH, …), and per-format extras + (`jam.msgidcrc`, `squish.umsgid`, `pcb.confnum`, …). + +Same model as RFC 822 email (headers + body). Lossless round-trip +across Read → Write → Read is enforced by the regression suite in +`tests/test_roundtrip_attrs.pas`. + +**The library never composes presentation.** A BBS that wants to +display kludges inline walks `Attributes` and prepends `^aMSGID:` +etc. to its own display. A BBS that hides kludges just shows +`Body`. A tosser that needs MSGID for dupe detection reads +`Attributes.Get('msgid')` directly — no body parsing required. + +Dates land in `TDateTime` regardless of how the backend stored +them (Hudson `MM-DD-YY` strings with 1950 pivot, Squish FTS-0001 +strings, JAM Unix timestamps, PCBoard / EzyCom DOS PackTime). +Stored in attributes as `date.written` / `date.received` via +`SetDate` / `GetDate`. + +Format-specific bit fields (Hudson byte attr, JAM 32-bit attr, +Squish attr, MSG word attr, PCB status, EzyCom dual byte) are +unrolled into individual `attr.*` boolean attributes on Read via +`UniAttrBitsToAttributes` and recomposed on Write via +`UniAttrBitsFromAttributes` and the per-format `XxxAttrFromUni` +helpers. The canonical `MSG_ATTR_*` cardinal bitset stays as the +internal pivot. + +### Capabilities API — backend self-description + +Each backend declares the canonical list of attribute keys it +understands via a class function: + +```pascal +class function TMessageBase.ClassSupportedAttributes: TStringDynArray; +``` + +Callers query before setting: + +```pascal +if base.SupportsAttribute('attr.returnreceipt') then + RenderReceiptCheckbox +else + HideReceiptCheckbox; +``` + +Backends silently ignore unknown attributes on Write (RFC 822 +X-header semantics — fine for forward compatibility); the +capabilities API exists so callers know in advance which keys won't +survive on a given format. The full per-format support matrix lives +in `docs/attributes-registry.md`. ## Locking diff --git a/docs/attributes-registry.md b/docs/attributes-registry.md new file mode 100644 index 0000000..5e80baf --- /dev/null +++ b/docs/attributes-registry.md @@ -0,0 +1,225 @@ +# Attribute registry + +This document is the source of truth for attribute key names used +across all `fpc-msgbase` backends. + +`TUniMessage` has only two areas: + +```pascal +TUniMessage = record + Body: AnsiString; { only the message text } + Attributes: TMsgAttributes; { everything else, key/value } +end; +``` + +Every header field, kludge, control line, and per-format flag a +backend understands lives in `Attributes` under one of the keys +below. Backends drop keys they don't recognise on Write (RFC 822 +X-header semantics). Callers can query `base.SupportsAttribute(K)` +before setting a key to know up-front which backends carry it. + +Naming convention: lowercase, dot-namespaced. Universal / +FTSC-defined keys are unqualified (`from`, `msgid`). Format- +specific keys are prefixed with the format name (`jam.msgidcrc`, +`squish.umsgid`, `pcb.confnum`). + +## Tier 1 — Universal headers + +Every Fido format sets these on Read; every backend reads them on +Write. Equivalent to the always-present headers in any classic +BBS message. + +| Key | Type | Meaning | +|---|---|---| +| `msg.num` | int | Backend-assigned message number / index | +| `from` | string | Author name | +| `to` | string | Recipient name | +| `subject` | string | Message subject | +| `date.written` | date | When author wrote the message | +| `date.received` | date | When local system received | +| `addr.orig` | ftn | Originating FTN address (`zone:net/node[.point]`) | +| `addr.dest` | ftn | Destination FTN address | +| `area` | string | Echo area tag (echomail) | +| `board` | int | Conference / board number for multi-board formats | +| `cost` | int | Cost in cents (FTS-1 backends) | + +## Tier 2 — Canonical attribute bits + +Boolean flags derived from the FTS-1 attribute word, plus extensions +some backends carry as separate bits. Always written via +`SetBool(K, true)`; absent (or false) when not set. + +| Key | FTS-1 | Meaning | +|---|---|---| +| `attr.private` | 0x0001 | Private message | +| `attr.crash` | 0x0002 | Crash priority | +| `attr.received` | 0x0004 | Recipient has read | +| `attr.sent` | 0x0008 | Sent to destination | +| `attr.fileattach` | 0x0010 | File attached | +| `attr.intransit` | 0x0020 | In transit | +| `attr.orphan` | 0x0040 | No matching destination | +| `attr.killsent` | 0x0080 | Kill after sending | +| `attr.local` | 0x0100 | Originated locally | +| `attr.hold` | 0x0200 | Hold for pickup | +| `attr.filereq` | 0x0400 | File request | +| `attr.returnreceipt` | 0x0800 | Return receipt requested | +| `attr.isreceipt` | 0x1000 | This message is a receipt | +| `attr.auditreq` | 0x2000 | Audit trail requested | +| `attr.fileupdreq` | 0x4000 | File update request | +| `attr.deleted` | — | Tombstoned (per-base) | +| `attr.read` | — | Marked-read (per-user) | +| `attr.echo` | — | Echomail (vs netmail) | +| `attr.direct` | — | Direct flavour | +| `attr.immediate` | — | Immediate flavour | +| `attr.locked` | — | Locked (no edit) | +| `attr.netpending` | — | Pending netmail toss | +| `attr.echopending` | — | Pending echomail toss | +| `attr.nokill` | — | Protected from purge | + +## Tier 3 — FTSC kludges + +Standard FTSC kludge lines, named after the kludge name without +the leading `^A`. Multi-line attributes (SEEN-BY, PATH, Via) join +their lines with `#13`. + +| Key | Type | Spec | Meaning | +|---|---|---|---| +| `msgid` | string | FTS-9 | Globally unique message ID | +| `replyid` | string | FTS-9 | Reply linkage to a previous MSGID | +| `pid` | string | FRL-1004 | Producer ID (creating tosser/editor) | +| `tid` | string | FSC-46 | Tosser ID | +| `flags` | string | FRL-1005 | Routing/handling flags | +| `chrs` | string | FTS-5003 | Character set declaration | +| `tzutc` | string | FTS-4001 | Time-zone offset from UTC | +| `seen-by` | multi-string | FTS-4 | SEEN-BY lines (one node-list per line) | +| `path` | multi-string | FTS-4 | PATH lines (one node-list per line) | +| `via` | multi-string | FTS-4009 | Via lines (one per relay) | + +## Tier 4 — Format-specific keys + +These are namespaced and only meaningful to the format that +produces them. Other backends ignore them on Write (silently +dropped — fine). + +### JAM + +| Key | Type | Meaning | +|---|---|---| +| `jam.msgidcrc` | int | Index fast-path CRC of MSGID | +| `jam.replycrc` | int | Index fast-path CRC of ReplyID | +| `jam.dateprocessed` | unix-int | Tosser timestamp | +| `jam.passwordcrc` | int | Per-message password CRC | +| `jam.cost` | int | JAM-level cost (separate from `cost`) | +| `jam.timesread` | int | Times-read counter | +| `jam.replyto` | int | Parent in reply chain | +| `jam.reply1st` | int | First child in reply chain | +| `jam.replynext` | int | Next sibling in reply chain | +| `jam.attribute2` | int | Reserved JAM attribute2 word | +| `jam.ftskludge` | multi | Passthrough for JAM_FTSKLUDGE subfields | +| `jam.subfield.` | multi | Passthrough for unknown subfield IDs | + +### Squish + +| Key | Type | Meaning | +|---|---|---| +| `squish.umsgid` | int | UMsgID (per-area unique number) | +| `squish.utcofs` | int | UTC offset in minutes | +| `squish.replyto` | int | Reply chain parent | +| `squish.kludge.` | multi | Passthrough for unknown CtrlInfo kludges | + +### Hudson / GoldBase + +| Key | Type | Meaning | +|---|---|---| +| `hudson.prevreply` | int | Previous message in reply chain | +| `hudson.nextreply` | int | Next message in reply chain | +| `hudson.timesread` | int | Times-read counter | +| `goldbase.prevreply` | int | (GoldBase variant) | +| `goldbase.nextreply` | int | (GoldBase variant) | +| `goldbase.timesread` | int | (GoldBase variant) | + +### EzyCom + +| Key | Type | Meaning | +|---|---|---| +| `ezy.extattr` | int | EzyCom ExtAttr byte | +| `ezy.prevreply` | int | Reply chain prev | +| `ezy.nextreply` | int | Reply chain next | + +### PCBoard + +| Key | Type | Meaning | +|---|---|---| +| `pcb.refnum` | int | RefNum field | +| `pcb.status` | int | Raw PCB status byte | +| `pcb.active` | int | Active flag | +| `pcb.echo` | int | Echo flag | +| `pcb.export` | int | Export flag | +| `pcb.extra2` | int | Extra2 byte (incl. Allfix-sent bit 6) | +| `pcb.extra3` | int | Extra3 byte | +| `pcb.hastags` | int | HasTags flag | +| `pcb.origin` | int | Origin flag | +| `pcb.readnum` | int | ReadNum word | +| `pcb.extendedstatus` | int | Extended status byte | +| `pcb.password` | string | Per-message password (max 12 chars) | + +### Wildcat + +| Key | Type | Meaning | +|---|---|---| +| `wildcat.confnum` | int | Conference number this message lives in | +| `wildcat.mflags` | int | Raw mFlags word from TMsgHeader | + +### PKT + +| Key | Type | Meaning | +|---|---|---| +| `pkt.cost` | int | Packet-level cost (separate from `cost`) | +| `pkt.kludge.` | multi | Passthrough for unknown body kludges | + +### MSG (FTS-1) + +| Key | Type | Meaning | +|---|---|---| +| `msg.replyto` | int | Reply chain parent | +| `msg.nextreply` | int | Reply chain next | +| `msg.timesread` | int | Times-read counter | +| `msg.kludge.` | multi | Passthrough for unknown body kludges | + +## Per-format support matrix + +`X` = the backend's `ClassSupportedAttributes` lists the key. +Blank = backend has no slot for it; setting the key has no effect +on Write, and the key won't appear in `Attributes` after a Read. + +| Key | JAM | Squish | MSG | PKT | Hudson | GoldBase | EzyCom | PCB | WC | +|---|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:| +| `msg.num` | X | X | X | X | X | X | X | X | X | +| `from` / `to` / `subject` | X | X | X | X | X | X | X | X | X | +| `addr.orig` / `addr.dest` | X | X | X | X | X | X | X | X | X | +| `date.written` | X | X | X | X | X | X | X | X | X | +| `date.received` | X | X | | | | | X | | X | +| `area` | X | X | X | X | X | X | X | X | X | +| `board` | | | | | X | X | X | | | +| `cost` | | | X | | X | X | X | | X | +| `attr.private` | X | X | X | X | X | X | X | X | X | +| `attr.crash` | X | X | X | X | X | X | X | | | +| `attr.received` | X | X | X | X | X | X | X | | X | +| `attr.sent` | X | X | X | X | X | X | X | | X | +| `attr.killsent` | X | X | X | X | X | X | X | | | +| `attr.local` | X | X | X | X | X | X | X | X | | +| `attr.hold` | X | X | X | X | X | X | | | | +| `attr.fileattach` | X | X | X | X | X | X | X | | | +| `attr.returnreceipt` | X | X | X | X | | | X | | X | +| `attr.deleted` | X | X | | | X | X | X | X | X | +| `msgid` | X | X | X | X | | | | | | +| `replyid` | X | X | X | X | | | | | | +| `pid` | X | X | X | X | | | | | | +| `flags` | X | X | X | X | | | | | | +| `seen-by` | X | X | X | X | | | | | | +| `path` | X | X | X | X | | | | | | +| Format-specific (`.*`) | X | X | X | X | X | X | X | X | X | + +(Always check at runtime via `SupportsAttribute(K)` rather than +relying on this table — it can drift. The capability list each +backend ships in `ClassSupportedAttributes` is authoritative.)