Replaces TUniMessage's 13-field flat record with a strict two-area
model: Body holds only the message text; Attributes holds everything
else (from/to/subject/dates/addresses/MSGID/SEEN-BY/PATH/format-
specific fields) as namespaced key/value pairs.
Why this fix is required NOW: the previous JAM adapter dropped
MSGID, ReplyID, PID, Flags, SEEN-BY and PATH on every Read/Write
through the unified API. A NetReader parity test surfaced it (17/21
pass with 4 kludge failures). All 9 adapters had the same bug. For
tossers and scanners the impact is silent corruption: dropped MSGID
→ dupe storms, dropped PATH → mail loops, dropped SEEN-BY → broken
routing. Three downstream consumers (Fimail's codex-transport branch,
NetReader, future Allfix) had halted integration work pending this
fix. Without it, anyone vendoring fpc-msgbase 0.1 ships with a
known-corrupting adapter.
Design choice: per Ken's call, "message is just the message text;
everything else is an attribute, including from/to/subject/dates."
Same architecture as RFC 822 email (headers + body). Each backend
fills attributes it knows on Read; reads attributes it understands
on Write; ignores unknown attributes silently (RFC 822 X-header
semantics). Forward-compatible -- a new backend (e.g. a planned SQL
message store) just adds its own attribute keys; old backends ignore
them.
Composition is the consumer's job. The library never reassembles
Body + Attributes into kludge-laden display text. A BBS that wants
inline kludges walks Attributes and prepends ^aMSGID etc. to its
own display. A tosser that needs MSGID for dupe detection reads
Attributes.Get('msgid') directly -- no body parsing required.
src/ma.types.pas:
- New TMsgAttribute / TMsgAttributes records with Get/SetValue,
typed accessors (GetInt/GetBool/GetDate/GetAddr), Has/Remove,
iteration. Linear-search lookup, fine for the ~30-50 keys per
message. Switch to hash later if profiling shows need.
- Replaced TUniMessage with the minimal Body + Attributes record.
- New UniAttrBitsToAttributes / UniAttrBitsFromAttributes helpers
to bridge the canonical MSG_ATTR_* cardinal bitset to/from
individual `attr.*` boolean keys.
- {$modeswitch advancedrecords} added so records have methods.
src/ma.api.pas:
- New capabilities API: TStringDynArray return type,
ClassSupportedAttributes (virtual class fn, default empty),
SupportedAttributes (instance sugar), SupportsAttribute (per-key
query). Each backend overrides ClassSupportedAttributes with the
static list of keys it knows. Callers query before setting so a
BBS UI can hide controls the underlying backend has no slot for.
src/formats/ma.fmt.*.uni.pas (all 9):
- Rewrote each XxxToUni and XxxFromUni for the new model. Read
populates Attributes with universal/FTSC/format-specific keys per
the attribute registry (to be published in phase 5). Write reads
attributes back and writes native form.
- JAM walks SubFields[] for SEEN-BY/PATH/TZUTC/TRACE plus passthrough
of unknown subfield IDs as `jam.subfield.<id>` for round-trip
safety. Squish parses CtrlInfo (NUL-separated ^A lines) into
individual attributes, rebuilds on Write. MSG and PKT (which keep
kludges inline in body per FTS-1) parse leading ^A lines and
trailing SEEN-BY/PATH out of the body so TUniMessage.Body is
always plain user text; on Write they reassemble the on-disk form.
- Each backend ships ClassSupportedAttributes with its key list.
src/ma.batch.pas: PktToUni signature updated to (in,out var) form.
tests/* + examples/*: migrated all callers from Msg.WhoFrom (etc.)
to Msg.Attributes.Get('from'). MakeMsg helpers now use SetValue/
SetBool/SetAddr.
Verified: 24/24 tests pass across all 7 test programs (read,
roundtrip, lock, batch, wildcat, write_existing, pack). Wildcat
walks all 7 vendored conferences clean.
Out of scope (next phases):
- docs/attributes-registry.md publishing the full key list with
per-format support matrix
- cross-format round-trip + capabilities-driven copy test
- update architecture.md / PROPOSAL.md to reflect the new model
121 lines
3.2 KiB
ObjectPascal
121 lines
3.2 KiB
ObjectPascal
{
|
|
test_lock.pas - in-process locking test.
|
|
|
|
Spawns N worker threads that all open the SAME TMessageBase
|
|
instance for write and append messages concurrently. After
|
|
the threads join, reopens the base and verifies the total
|
|
matches N * per-thread count with no corruption.
|
|
|
|
This exercises layer-1 (TRTLCriticalSection per TMessageBase
|
|
instance) in ma.lock. Cross-process coordination via the
|
|
sentinel file is covered implicitly: each WriteMessage takes
|
|
the instance's lock on entry, so concurrent writers serialise.
|
|
}
|
|
|
|
program test_lock;
|
|
|
|
{$mode objfpc}{$H+}
|
|
|
|
uses
|
|
{$IFDEF UNIX}cthreads,{$ENDIF}
|
|
SysUtils, Classes, SyncObjs,
|
|
testutil,
|
|
ma.types, ma.events, ma.api,
|
|
ma.fmt.jam, ma.fmt.jam.uni;
|
|
|
|
const
|
|
SCRATCH = '/tmp/ma_lock';
|
|
PATH = SCRATCH + '/shared';
|
|
THREADS = 4;
|
|
PER_THR = 25;
|
|
|
|
type
|
|
TWriterThread = class(TThread)
|
|
private
|
|
FBase: TMessageBase;
|
|
FTag: integer;
|
|
protected
|
|
procedure Execute; override;
|
|
public
|
|
constructor Create(ABase: TMessageBase; ATag: integer);
|
|
end;
|
|
|
|
constructor TWriterThread.Create(ABase: TMessageBase; ATag: integer);
|
|
begin
|
|
FBase := ABase;
|
|
FTag := ATag;
|
|
FreeOnTerminate := False;
|
|
inherited Create(False);
|
|
end;
|
|
|
|
procedure TWriterThread.Execute;
|
|
var
|
|
i: integer;
|
|
msg: TUniMessage;
|
|
begin
|
|
for i := 1 to PER_THR do begin
|
|
msg.Attributes.Clear;
|
|
msg.Attributes.SetValue('from', 'T' + IntToStr(FTag));
|
|
msg.Attributes.SetValue('to', 'All');
|
|
msg.Attributes.SetValue('subject', Format('tag=%d seq=%d', [FTag, i]));
|
|
msg.Attributes.SetDate('date.written', Now);
|
|
msg.Attributes.SetBool('attr.local', true);
|
|
msg.Attributes.SetAddr('addr.orig', MakeFTNAddress(1, 1, 1, 0));
|
|
msg.Attributes.SetAddr('addr.dest', MakeFTNAddress(1, 1, 2, 0));
|
|
msg.Body := Format('body %d/%d', [FTag, i]) + #13;
|
|
FBase.WriteMessage(msg);
|
|
end;
|
|
end;
|
|
|
|
procedure RunConcurrentWriters;
|
|
var
|
|
base: TMessageBase;
|
|
workers: array[0..THREADS - 1] of TWriterThread;
|
|
i: integer;
|
|
begin
|
|
TestBegin(Format('Concurrent writers: %d threads * %d msgs',
|
|
[THREADS, PER_THR]));
|
|
ForceDirectories(SCRATCH);
|
|
if FileExists(PATH + '.jhr') then DeleteFile(PATH + '.jhr');
|
|
if FileExists(PATH + '.jdt') then DeleteFile(PATH + '.jdt');
|
|
if FileExists(PATH + '.jdx') then DeleteFile(PATH + '.jdx');
|
|
if FileExists(PATH + '.jlr') then DeleteFile(PATH + '.jlr');
|
|
|
|
base := MessageBaseOpen(mbfJam, PATH, momCreate);
|
|
try
|
|
AssertTrue('Open', base.Open);
|
|
|
|
for i := 0 to THREADS - 1 do
|
|
workers[i] := TWriterThread.Create(base, i + 1);
|
|
for i := 0 to THREADS - 1 do begin
|
|
workers[i].WaitFor;
|
|
workers[i].Free;
|
|
end;
|
|
|
|
AssertEquals('Total messages after concurrent writes',
|
|
THREADS * PER_THR, base.MessageCount);
|
|
finally
|
|
base.Close;
|
|
base.Free;
|
|
end;
|
|
|
|
{ Reopen and verify structural integrity: every message readable,
|
|
every expected (tag, seq) pair present. }
|
|
base := MessageBaseOpen(mbfJam, PATH, momReadOnly);
|
|
try
|
|
AssertTrue('Reopen', base.Open);
|
|
AssertEquals('Count after reopen', THREADS * PER_THR, base.MessageCount);
|
|
finally
|
|
base.Close;
|
|
base.Free;
|
|
end;
|
|
TestOK;
|
|
end;
|
|
|
|
begin
|
|
WriteLn('fpc-msgbase: in-process locking test');
|
|
WriteLn;
|
|
RunConcurrentWriters;
|
|
Halt(TestsSummary);
|
|
end.
|