Initial scaffold: layout, fpc.cfg, README, architecture doc
This commit is contained in:
25
.gitignore
vendored
Normal file
25
.gitignore
vendored
Normal file
@@ -0,0 +1,25 @@
|
||||
# FPC build artifacts
|
||||
*.o
|
||||
*.ppu
|
||||
*.compiled
|
||||
*.or
|
||||
*.a
|
||||
*.lib
|
||||
*.dcu
|
||||
|
||||
# Lazarus
|
||||
*.lps
|
||||
backup/
|
||||
lib/
|
||||
units/
|
||||
|
||||
# Build output trees
|
||||
exe/
|
||||
units/
|
||||
|
||||
# Editor / OS junk
|
||||
.vscode/
|
||||
.idea/
|
||||
*.swp
|
||||
*~
|
||||
.DS_Store
|
||||
69
README.md
Normal file
69
README.md
Normal file
@@ -0,0 +1,69 @@
|
||||
# message_api
|
||||
|
||||
A unified Free Pascal library for reading and writing classic BBS message bases.
|
||||
|
||||
Wraps proven message-format handlers from Allfix behind one polymorphic API
|
||||
(`TMessageBase`) so BBS software, mail tossers, message editors, and utilities
|
||||
can target a single interface regardless of the underlying format on disk.
|
||||
|
||||
## Supported formats
|
||||
|
||||
| Format | Files | Backend unit |
|
||||
|-----------|--------------------------------------|---------------------------|
|
||||
| Hudson | MSGINFO/IDX/HDR/TXT/TOIDX.BBS | `ma.fmt.hudson.pas` |
|
||||
| JAM | `*.JHR` `*.JDT` `*.JDX` `*.JLR` | `ma.fmt.jam.pas` |
|
||||
| Squish | `*.SQD` `*.SQI` `*.SQL` | `ma.fmt.squish.pas` |
|
||||
| FTS-1 MSG | numbered `*.MSG` per directory | `ma.fmt.msg.pas` |
|
||||
| FTN PKT | `*.pkt` (Type-2 / 2+ / 2.2) | `ma.fmt.pkt.pas` |
|
||||
| PCBoard | `*.MSG` + `*.IDX` | `ma.fmt.pcboard.pas` |
|
||||
| EzyCom | `MH#####.BBS` / `MT#####.BBS` | `ma.fmt.ezycom.pas` |
|
||||
| GoldBase | MSGINFO/IDX/HDR/TXT/TOIDX.DAT | `ma.fmt.goldbase.pas` |
|
||||
| Wildcat 4 | WC SDK databases | `ma.fmt.wildcat.pas` |
|
||||
|
||||
## Features
|
||||
|
||||
- One `TMessageBase` abstract class — read, write, pack, reindex through the
|
||||
same methods regardless of format.
|
||||
- Layered locking: in-process `TRTLCriticalSection` + cross-process advisory
|
||||
lock (`fpflock` on Unix, `LockFileEx` on Windows, `.LCK` sentinel fallback)
|
||||
+ the existing `fmShareDenyWrite` / `fmShareDenyNone` share modes.
|
||||
- Event hooks for logging, progress, and status reporting.
|
||||
- `TPacketBatch` worker pool for tossers that need to process many `.pkt`
|
||||
files concurrently while serialising writes per destination base.
|
||||
- Path / filename auto-derivation per format from a base directory plus
|
||||
optional area tag.
|
||||
|
||||
## Building
|
||||
|
||||
Native Linux:
|
||||
|
||||
```sh
|
||||
fpc -Fusrc -Fusrc/formats examples/example_read.pas
|
||||
```
|
||||
|
||||
Lazarus package:
|
||||
|
||||
```sh
|
||||
lazbuild message_api.lpk
|
||||
```
|
||||
|
||||
The repo includes a `fpc.cfg` template mirroring the Allfix multi-target
|
||||
build (`i386-go32v2`, `i386-win32`, `i386-linux`, `i386-os2`).
|
||||
|
||||
## Layout
|
||||
|
||||
```
|
||||
src/ ma.api, ma.types, ma.events, ma.lock, ma.paths, ma.batch
|
||||
src/formats/ ma.fmt.<format>.pas — one per supported format
|
||||
docs/ architecture, locking semantics, format notes
|
||||
tests/ FPCUnit tests, sample data
|
||||
examples/ small CLI programs that double as smoke tests
|
||||
```
|
||||
|
||||
Unit-name convention follows the Fimail style: `ma.<category>.<name>.pas`
|
||||
(`ma` = message_api). All units use `{$mode objfpc}{$H+}`.
|
||||
|
||||
## Status
|
||||
|
||||
Early development. APIs may move until 1.0. Format backends are ports of the
|
||||
working Allfix code so behaviour matches what Allfix produces today.
|
||||
88
docs/architecture.md
Normal file
88
docs/architecture.md
Normal file
@@ -0,0 +1,88 @@
|
||||
# message_api — architecture
|
||||
|
||||
## Layers
|
||||
|
||||
```
|
||||
┌──────────────────────────────────────────────────┐
|
||||
│ Caller (BBS, tosser, editor, importer, …) │
|
||||
└──────────────────────────────────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌──────────────────────────────────────────────────┐
|
||||
│ ma.api (TMessageBase, factory, TUniMessage) │
|
||||
├──────────────────────────────────────────────────┤
|
||||
│ ma.events ma.lock ma.paths │
|
||||
│ ma.batch (concurrent tosser helper) │
|
||||
├──────────────────────────────────────────────────┤
|
||||
│ Format backends — one .pas per format │
|
||||
│ ma.fmt.hudson ma.fmt.jam ma.fmt.squish │
|
||||
│ ma.fmt.msg ma.fmt.pkt ma.fmt.pcboard │
|
||||
│ ma.fmt.ezycom ma.fmt.goldbase ma.fmt.wildcat │
|
||||
├──────────────────────────────────────────────────┤
|
||||
│ RTL: TFileStream, BaseUnix/Windows for locking │
|
||||
└──────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
## Polymorphism
|
||||
|
||||
Every backend descends from `TMessageBase` and implements the abstract
|
||||
`DoOpen`, `DoClose`, `DoMessageCount`, `DoReadMessage`, `DoWriteMessage`
|
||||
contract. Callers can either:
|
||||
|
||||
1. Use the unified API — `MessageBaseOpen(format, path, mode)` returns a
|
||||
`TMessageBase`. Read/write through `TUniMessage`. Format-agnostic.
|
||||
2. Drop down to format-specific class methods (e.g. `TJamBase.IncModCounter`,
|
||||
`TSquishBase.SqHashName`) when they need behaviour the unified API cannot
|
||||
express. Each backend keeps its rich API public.
|
||||
|
||||
## TUniMessage
|
||||
|
||||
A single message record covering every backend. Format-specific bit fields
|
||||
(Hudson byte attr, JAM 32-bit attr, Squish attr, MSG word attr, PCB status,
|
||||
EzyCom dual byte) are mapped into a canonical `Attr: cardinal` bit set
|
||||
defined in `ma.types` (`MSG_ATTR_PRIVATE`, `MSG_ATTR_LOCAL`,
|
||||
`MSG_ATTR_RECEIVED`, …). Conversion helpers expose the original encoding
|
||||
when needed.
|
||||
|
||||
Dates land in `TDateTime` regardless of how the backend stored them
|
||||
(Hudson `MM-DD-YY` strings with 1950 pivot, Squish FTS-0001 strings, JAM
|
||||
Unix timestamps, PCBoard / EzyCom DOS PackTime).
|
||||
|
||||
The body is an `AnsiString` with kludge lines intact and CR-separated
|
||||
(matches what Allfix already produces).
|
||||
|
||||
## Locking
|
||||
|
||||
Three layers, applied in order on every `Open`:
|
||||
|
||||
1. **In-process** — `TRTLCriticalSection` per `TMessageBase` instance.
|
||||
2. **Cross-process** — advisory lock on a sentinel file
|
||||
(`<base>.lck` or, for Squish, `<base>.SQL` so we coexist with other
|
||||
Squish-aware tools). `fpflock(LOCK_EX|LOCK_SH)` on Unix,
|
||||
`LockFileEx` on Windows. Retry with backoff up to a configurable
|
||||
timeout (default 30s). Lock acquire/release fires events.
|
||||
3. **OS share modes** — `fmShareDenyWrite` for writers,
|
||||
`fmShareDenyNone` for readers. Same as the existing Allfix code.
|
||||
|
||||
## Events
|
||||
|
||||
`TMessageEvents` lets callers subscribe one or more handlers to receive
|
||||
`metBaseOpened`, `metMessageRead`, `metMessageWritten`, `metLockAcquired`,
|
||||
`metPackProgress`, etc. Internally the dispatcher serialises calls so
|
||||
handlers do not need to be reentrant.
|
||||
|
||||
## Concurrent tossers
|
||||
|
||||
`TPacketBatch` owns a queue of `.pkt` paths and a worker thread pool.
|
||||
Each worker opens its packet, reads messages, hands each to the
|
||||
caller-provided processor. The batch caches one `TMessageBase` per
|
||||
destination area so writes serialise through layer-1 locking; layer-2
|
||||
keeps separate processes (e.g. an editor) safe at the same time.
|
||||
|
||||
## Behavioural fidelity
|
||||
|
||||
Every format backend is a port of the proven Allfix implementation. Tests
|
||||
read/write the same sample bases that Allfix tests use; outputs are
|
||||
diffed against captures produced by the existing `msgutil` / `domsg`
|
||||
code paths. Anything that works in Allfix today must keep working through
|
||||
this library.
|
||||
49
fpc.cfg
Normal file
49
fpc.cfg
Normal file
@@ -0,0 +1,49 @@
|
||||
# message_api fpc.cfg — mirrors the Allfix template.
|
||||
#
|
||||
# Pulls compiler-bundled RTL paths from /opt/fpcup, then adds the library
|
||||
# source roots so callers can build with a single -Fucfg=fpc.cfg.
|
||||
|
||||
#IFNDEF FPC_CROSSCOMPILING
|
||||
#IFDEF LINUX
|
||||
#INCLUDE /opt/fpcup/fpc/bin/x86_64-linux/fpc.cfg
|
||||
#ENDIF LINUX
|
||||
#IFDEF OS2
|
||||
#INCLUDE P:\fpc\bin\os2\fpc.cfg
|
||||
#ENDIF OS2
|
||||
#IFDEF DOS
|
||||
-Fuc:\pp/units/i386-$FPCTARGET/
|
||||
-Fuc:\pp/units/i386-$FPCTARGET/*
|
||||
-Fuc:\pp/units/i386-$FPCTARGET/rtl
|
||||
#ENDIF DOS
|
||||
#ELSE
|
||||
-Fu/opt/fpcup/fpc/units/$FPCTARGET/rtl
|
||||
-Fu/opt/fpcup/fpc/units/$FPCTARGET/*
|
||||
#ENDIF
|
||||
|
||||
# Allow GOTO/LABEL, inline, C-operators (matches Allfix dialect)
|
||||
-Sgic
|
||||
|
||||
# Library source roots
|
||||
-Fusrc
|
||||
-Fusrc/formats
|
||||
|
||||
# Output trees per target
|
||||
-FE./exe/$FPCTARGET
|
||||
-FU./units/$FPCTARGET
|
||||
|
||||
# Cross-compile binutils prefixes (mirrors Allfix)
|
||||
#IFDEF FPC_CROSSCOMPILING
|
||||
#IFDEF CPUI386
|
||||
#IFDEF OS2
|
||||
-FD/opt/fpcup/cross/i386-pc-os2-emx
|
||||
-XPi386-emx-
|
||||
-k-e
|
||||
-k0x10000
|
||||
#ELSE
|
||||
-FD/opt/fpcup/cross/bin
|
||||
-XP$FPCTARGET-
|
||||
#ENDIF OS2
|
||||
#ENDIF CPUI386
|
||||
#ENDIF FPC_CROSSCOMPILING
|
||||
|
||||
-Xs
|
||||
Reference in New Issue
Block a user