====================================================================== COMET MAILER - OPERATOR MANUAL Version 1.2 Direct TCP File Transfer for FidoNet Copyright (C) 2026 Ken Johnson, 1:218/720 ====================================================================== TABLE OF CONTENTS 1. Introduction 2. System Requirements 3. Installation 4. Quick Start 5. Command Reference 6. Configuration Reference 7. Authentication 8. Nodelist Integration 9. File Requests (FREQ) 10. Post-Session Events 11. BinkP Compatibility 12. Cross-Platform Notes 13. Troubleshooting 14. Security Notes 15. Data Compression 16. Technical Reference 17. Embedding in Host Applications 18. Change Log ====================================================================== 1. INTRODUCTION ====================================================================== Comet is a modern FidoNet TCP mailer daemon designed for the 2020s. It provides: - Direct TCP file transfer with SHA-256 verification - Bidirectional transfer with sliding window flow control - ED25519 public-key authentication (no shared secrets needed) - CRAM-MD5 password authentication (passwords never sent in clear) - X25519 + ChaCha20 session encryption (with ED25519 auth) - Full BinkP/1.1 compatibility (FTS-1026) with auto-detection - Inline file requests (Hydra-style, within active sessions) - FTS-5 nodelist integration (automatic IP/port lookup) - BSO outbound scanning with point directory support - Multi-session daemon with per-node BSY locking - Config hot-reload (edit config while Comet is running) - Cross-platform: Linux, FreeBSD, DOS, Windows, OS/2 Comet uses TCP port 24554, the standard BinkP port. Both Comet native and BinkP connections are accepted on the same port. The protocol specification is in FSP-COMET.001. ====================================================================== 2. SYSTEM REQUIREMENTS ====================================================================== Linux / FreeBSD: - Any modern x86 or x86-64 system - TCP/IP networking - Recommended: /dev/urandom for ED25519 key generation DOS (FreeDOS recommended): - 386 or better with DPMI (CWSDPMI.EXE) - Packet driver for your network card - Watt-32 TCP/IP (WATTCP.CFG) - Approximately 1 MB RAM for the executable Windows: - Windows 7 or later (64-bit) - WinSock2 (included with Windows) OS/2: - OS/2 Warp or eComStation with TCP/IP stack ====================================================================== 3. INSTALLATION ====================================================================== 3.1. Linux / FreeBSD 1. Copy the 'comet' binary to /usr/local/bin/ or your preferred location. 2. Copy COMET.SAM to /etc/comet/comet.cfg (or wherever you prefer) and edit it for your system. 3. Create the required directories: mkdir -p /var/lib/fidonet/{inbound,secure,outbound,temp} 4. Generate an ED25519 keypair: comet keygen 5. Add the PrivateKey to your config file. 6. Start the daemon: comet -c /etc/comet/comet.cfg 7. To run as a system service, create a systemd unit file or add to /etc/rc.local. 3.2. DOS 1. Copy COMET.EXE and CWSDPMI.EXE to your Comet directory (e.g., F:\COMET\). 2. Copy COMET.SAM to COMET.CFG and edit it. 3. Create directories: INBOUND, SECURE, OUTBOUND, TEMP 4. Configure WATTCP.CFG for your network: my_ip = dhcp or: my_ip = 192.168.1.100 netmask = 255.255.255.0 gateway = 192.168.1.1 5. Set the environment variable: SET WATTCP.CFG=F:\COMET 6. Load your packet driver: NE2000 0x60 3 0x300 7. Run Comet: COMET CALL 1:213/723 3.3. Signal Handling (Unix) SIGTERM / SIGINT : Clean shutdown (waits for active sessions) SIGHUP : Reload configuration SIGPIPE : Ignored (handled internally) ====================================================================== 4. QUICK START ====================================================================== 4.1. Daemon Mode (listen + poll outbound) comet comet -c /path/to/comet.cfg 4.2. Single Call comet call 1:213/723 comet -c /path/to/comet.cfg call 1:218/720 4.3. Generate ED25519 Keypair comet keygen This generates a random keypair and displays: - PrivateKey: add to [System] section of your config - PublicKey: give to remote nodes (they add to [Node:] section) 4.4. Show Version comet -v 4.5. Show Help comet -h 4.6. Debug Mode comet -d comet -d call 1:213/723 Enables verbose protocol-level logging. ====================================================================== 5. COMMAND REFERENCE ====================================================================== comet [options] [command] Commands: (none) Run as daemon (listen + poll outbound) call
Single outbound call to a FidoNet node keygen Generate ED25519 keypair showkey Print public key from configured private key Options: -c Use specified config file (default: comet.cfg) -d Enable debug/trace logging -q Quiet mode (errors only on console) -v Show version and exit -h, --help Show help and exit Address format: zone:net/node e.g., 1:213/723 zone:net/node.point e.g., 1:213/723.1 ====================================================================== 6. CONFIGURATION REFERENCE ====================================================================== Configuration is read from comet.cfg (or the file specified with -c). The file uses INI-style sections with key = value pairs. Lines beginning with ; are comments. Comet monitors the config file for changes and automatically reloads when modifications are detected (checked every 5 seconds). On Unix, you can also send SIGHUP to force a reload. 6.1. [System] Section Address = 1:218/720 Your FidoNet address. Multiple addresses supported (one per line). First address is primary. SysOp = Ken Johnson Your name as system operator. SystemName = The Danger Zone Your system/BBS name. Location = Sparks, NV City, state/country. PrivateKey = <64 hex characters> Your ED25519 private key. Generate with: comet keygen Keep this SECRET. Never share it. 6.2. [Network] Section Port = 24554 TCP port to listen on. Default: 24554. Bind = Network interface to bind to. Empty = all interfaces. MaxSessions = 5 Maximum simultaneous sessions. Range: 1-32. MaxSessionsPerIP = 0 Optional cap on simultaneous inbound sessions from a single IP address. 0 = no per-IP cap (default). Applies only to inbound — outbound calls are unaffected. MaxSessionTime = 0 Optional wall-clock limit (in seconds) per session. Any session running longer than this is force-terminated. 0 = no limit (default). Protects against stuck transfers and slow-drip attackers. PollInterval = 60 Outbound poll interval in seconds. 0 = no polling. 6.3. [Paths] Section Inbound = /var/lib/fidonet/inbound/ Directory for files from unauthenticated nodes. SecInbound = /var/lib/fidonet/secure/ Directory for files from password-verified nodes. Outbound = /var/lib/fidonet/outbound/ BSO outbound base directory. Temp = /var/lib/fidonet/temp/ Temporary files during transfer. LogFile = /var/log/fidonet/comet.log Main log file path. DebugLog = /var/log/fidonet/comet-debug.log Debug/trace log (very verbose). Nodelist = /var/lib/fidonet/nodelist/ Directory containing nodelist files. Freq = /var/lib/fidonet/freq/ Directory for file request searches. Flags = /var/lib/fidonet/flags/ Semaphore/flag files directory. 6.4. [DriveMap] Section (Linux/FreeBSD only) Maps DOS drive letters to Unix paths for reading .FLO files written by DOS mailers. C = /home/user/dos/c D = /home/user/dos/d 6.5. [Protocol] Section MaxBlockSize = 65528 Maximum data block size in bytes (512-65528). WindowSize = 8 Sliding window depth (1-16). Timeout = 120 Braindead timeout in seconds. HandshakeTimeout = 30 Handshake timeout in seconds. NoSHA256 = no Disable SHA-256 file verification (NOT recommended). NoFREQ = no Disable file request support. 6.6. [BinkP] Section Enabled = yes Enable BinkP fallback for outbound connections. Port = 24554 Default BinkP port for fallback connections. 6.7. [Logging] Section FileLevel = info Minimum log level for file: debug, info, warning, error, fatal ConsoleLevel = info Minimum log level for console. Debug = no Enable debug/trace logging. 6.8. [Events] Section flag Create a semaphore file when files matching pattern are received. exec "" Run external command when files matching pattern are received. Variables for exec commands: *A = Remote FidoNet address *I = Inbound directory path *N = Number of files received Examples: flag /var/lib/fidonet/flags/toss.now *.pkt exec "hpt toss" *.pkt exec "allfix RP -SRIF *S" *.req 6.9. [Node:address] Sections Per-node configuration. Section name includes the address: [Node:1:213/723] [Node:21:1/100] Password = SECRET Session password (CRAM-MD5 authenticated, never sent in clear). PublicKey = <64 hex characters> Remote node's ED25519 public key (from their sysop). Host = bbs.example.com IP address or hostname. Port = 24554 Outbound port override. If not set, the port from the nodelist (IBN flag) is used. Default: 24554. NoBinkp = no Disable BinkP fallback for this node. NoComet = no Use only BinkP for this node (skip Comet protocol). CallWindows = mon-fri 09:00-17:00 Time restriction for outbound polls. Empty (default) = always allowed. Format: "dayspec HH:MM-HH:MM" with multiple windows separated by ';'. Days (case-insensitive): sun mon tue wed thu fri sat, or '*' for every day. Ranges "mon-fri" and lists "sat,sun" are supported. Times are 24-hour local time. Wrap-around "22:00-02:00" (overnight) is supported. Examples: "*" every day, all hours (default) "mon-fri 09:00-17:00" weekdays 9am-5pm "* 22:00-06:00" overnight only "mon-fri 09:00-17:00; sat 10:00-15:00" mixed schedule The poller and the [Scheduler] poll_all action honour this restriction. Manual "Call" from the WebUI bypasses it, so the operator can always force a call when needed. HookPreCall = command HookOnSuccess = command HookOnFail = command Per-node external program hooks. Each is a shell command template; variables are substituted at fire time and shell-quoted on Unix to prevent injection. Variables: *A = node address (e.g. 1:218/720) *N = remote system name (after handshake) *S = remote sysop name *I = inbound directory *F = files received this session *X = files sent this session *B = total bytes (sent + received) *R = remote mailer name HookPreCall fires before the call attempt — *F/*X/*B will be 0. Use for pre-flight notifications, dial-up activation, session logging. HookOnSuccess fires after a successful session, after files have been moved into the inbound directory. Use for tossing mail, syncing files, success notifications. HookOnFail fires after a connect failure, handshake reject, or session-thread exception. Use for alerts, retry counters, escalation. These hooks are independent of the global [Events] section, which fires for ALL sessions. Per-node hooks fire only for the matching address. Examples: HookPreCall = logger -t comet "Calling *A" HookOnSuccess = /usr/local/bin/toss-incoming.sh *I *F HookOnFail = /usr/local/bin/page-oncall.sh *A "Comet failed" ====================================================================== 7. AUTHENTICATION ====================================================================== Comet supports three authentication methods, tried in order of strength: 7.1. ED25519 Digital Signatures (Strongest) Public-key authentication. Each node generates a keypair: - Private key: stays on your system, never shared - Public key: given to remote nodes Setup: 1. Run: comet keygen 2. Add PrivateKey to your [System] section 3. Give your PublicKey to remote sysops 4. They add it to their [Node:your_address] section 5. They give you their PublicKey 6. You add it to your [Node:their_address] section No passwords needed. The private key never leaves your system and cannot be derived from the public key. 7.2. CRAM-MD5 (Shared Secret) Challenge-response authentication per RFC 2104. Both sides must know the same password. The password is NEVER sent in plain text. Setup: 1. Agree on a password with the remote sysop 2. Add Password = SECRETWORD to both [Node:] sections 7.3. No Authentication If neither ED25519 keys nor passwords are configured, the session proceeds without authentication. Files are placed in the unsecure Inbound directory. This is logged as a warning. 7.4. Authentication Fallback When connecting, Comet tries methods in order: ED25519 -> CRAM-MD5 -> Plain password -> No auth Only available methods are advertised to the remote. If ED25519 is not configured, it is not offered. 7.5. Session Encryption When both sides support encryption and ED25519 authentication succeeds, Comet automatically encrypts the session using X25519 ephemeral key exchange and ChaCha20 stream cipher. - Encryption is automatic: no additional configuration needed beyond setting up ED25519 keys (see 7.1 above). - Forward secrecy: each session generates fresh ephemeral keys, so compromising a long-term private key does not reveal the content of past sessions. - Encryption requires ED25519: CRAM-MD5 and passwordless sessions are not encrypted because there is no identity verification to prevent man-in-the-middle attacks. - Interoperable: if the remote does not support encryption, the session proceeds unencrypted with no configuration changes needed. ====================================================================== 8. NODELIST INTEGRATION ====================================================================== Comet reads standard FidoNet raw text nodelists (FTS-5 format). 8.1. Setup Set the Nodelist path in [Paths]: Nodelist = /var/lib/fidonet/nodelist/ Place nodelist files (e.g., NODELIST.081) in this directory. Comet automatically loads the most recent one. 8.2. How It Works When polling outbound or making a call, if no [Node:] section exists for the target address, Comet looks it up in the nodelist. It extracts: - INA: flag for IP address/hostname - IBN: flag for BinkP port (default 24554) [Node:] entries always take priority over nodelist data. 8.3. Nodelist Reload The nodelist is reloaded when the config file changes (hot-reload). ====================================================================== 9. FILE REQUESTS (FREQ) ====================================================================== 9.1. BinkP FREQ Standard .REQ file mechanism. The .REQ file is sent as a regular file. When the remote receives it, they process the patterns and send matching files back. To make a request: create a .REQ file in the outbound directory for the target node (e.g., 00D502D3.REQ for 1:213/723). Format: one filename pattern per line. 9.2. Comet FREQ (Inline) Comet protocol supports Hydra-style inline file requests via NPKT_FREQ packets. Requests are processed immediately during the session without requiring a separate .REQ file transfer. 9.3. FREQ Processing When a file request is received, Comet first checks the [FreqAliases] table for a case-insensitive name match. If found, the alias's pattern is served (which may live anywhere on disk — aliases are trusted config data). If not, Comet searches the Freq directory for matching files. Pattern sanitization rejects any remote-supplied pattern containing path separators (/, \), parent-dir refs (..), control characters, or absolute paths. After matching, every served file is verified to actually live inside FreqDir as a defense-in-depth check against symlinks or unusual filesystem layouts. Limits: 10 files, 10 MB per request. For advanced FREQ processing (file areas, per-link access control), use an external FREQ processor via [Events] exec rules with SRIF. 9.4. Setup Set the Freq path in [Paths]: Freq = /var/lib/fidonet/freq/ Place files you want to make available in this directory. 9.5. FREQ Aliases (Magic Names) Define short alias names that remote nodes can request via FREQ instead of knowing the real filename: [FreqAliases] NODELIST = >/var/lib/fidonet/nodelist/nodelist.* POINTLIST = >/var/lib/fidonet/nodelist/points24.* ARCHIVE = /var/lib/fidonet/archive/*.zip MAGIC = /var/lib/fidonet/freq/welcome.txt FILES = /var/lib/fidonet/freq/files.bbs PUBKEY = /etc/comet/pubkey.txt Aliases are matched case-insensitively. The pattern can be a literal path or a glob (* and ? wildcards). Anything not matching an alias falls back to a normal FreqDir search. Glob behaviour matches Argus and BTXE: - By default, all files matching the glob are sent, up to the 10-file / 10 MB per-request limits. - Prefix the pattern with '>' to serve only the NEWEST file(s) by mtime. Use this for NODELIST-style magic names where the remote wants today's file, not every archived copy. - A literal path serves that one file as-is. Extended form — per-alias password, limits, and external processor (xenia-mailer compatible feature set): ALIAS = | pw= | max_size= | max_count= | exec= pw=X Password-protect the alias. Remote must send "ALIAS!password" on the request line (FTN convention). Without the password, the alias returns nothing. max_size=X Per-alias total size cap. Accepts K/M/G suffix (e.g. 5M = 5,242,880 bytes). Overrides the global 10MB default when lower. max_count=N Per-alias file count cap. Overrides the global 10-file default when lower. exec=CMD External FREQ processor (SRIF-style). When set, the pattern field is ignored. CMD is spawned with three arguments: the alias name, the password the remote supplied (or empty), and the remote address. Whatever absolute file paths the program prints on stdout (one per line) are served back. Lines beginning with '#' are comments. Stdout is capped at 256KB and the per-alias / global size+count limits still apply. Use this for database-backed file areas, dynamic "latest build" generators, custom access control, or wrappers around legacy SRIF processors. On Unix the command goes through /bin/sh -c so pipes and redirection work in the config. On Windows it is passed directly to ExecuteProcess. Examples: [FreqAliases] PRIVATE = /data/private/*.zip | pw=sekret | max_size=50M LATEST = | exec=/usr/local/bin/comet-latest-build.sh FILEAREA = | exec=/usr/local/bin/comet-freq-db.py | max_count=20 Aliases are TRUSTED config data and may point to files outside the FreqDir — useful for serving system files (nodelist, pubkey) without exposing the whole directory. ====================================================================== 10. POST-SESSION EVENTS ====================================================================== After a session completes and files are received, Comet can: 10.1. Create Flag Files [Events] flag /var/lib/fidonet/flags/toss.now *.pkt Creates the specified file when files matching the pattern are received. Your tosser watches for this flag. 10.2. Run External Commands [Events] exec "hpt toss" *.pkt exec "allfix RP -SRIF *S" *.req Commands run AFTER the session is closed - they never block active transfers. ====================================================================== 11. BINKP COMPATIBILITY ====================================================================== Comet is fully compatible with BinkP/1.1 (FTS-1026). 11.1. Auto-Detection Both Comet and BinkP connections are accepted on the same TCP port. The protocol is auto-detected from the first bytes received. 11.2. Fallback When calling a node, Comet connects on the port from the nodelist (IBN flag) or the per-node Port setting. Both protocols are auto-detected by the remote. Set NoComet = yes for nodes that only speak BinkP. 11.3. Features in BinkP Mode - CRAM-MD5 authentication (FTS-1027) - ED25519 authentication (via OPT extension) - M_GET file resume - FREQ via .REQ file transfer - OPT negotiation (NR, EXTCMD) 11.4. Tested Against - binkd (reference BinkP implementation) - Radius/Argus 4.010 (Windows GUI mailer) ====================================================================== 12. CROSS-PLATFORM NOTES ====================================================================== 12.1. Linux (Primary Platform) Full support. Production ready. 12.2. FreeBSD Cross-compiled from Linux. Same features as Linux. Build: make freebsd 12.3. DOS (FreeDOS) Requires CWSDPMI.EXE and Watt-32 TCP/IP with packet driver. Single-session mode (no threading). Build: make dos 12.4. Windows WinSock2 code written, needs type compatibility fixes. Build: make win64 (work in progress) 12.5. OS/2 Should work via FPC's OS/2 Sockets unit. Untested. 12.6. Path Handling Comet handles DOS/Unix path conversion automatically: - Backslash <-> forward slash conversion - Drive letter mapping via [DriveMap] - Case-insensitive file lookup on Unix ====================================================================== 13. TROUBLESHOOTING ====================================================================== 13.1. "Cannot listen on port 24554" Another process is using the port, or a previous Comet instance hasn't fully shut down. Wait a few seconds and try again, or check with: netstat -tlnp | grep 24554 13.2. "No host configured for X:XXX/XXX" The node has no [Node:] section and isn't in the nodelist. Add a [Node:] section with Host, or set the Nodelist path. 13.3. "Password mismatch" CRAM-MD5 passwords don't match. Verify both sides have the same password configured. 13.4. "ED25519 verification failed" The public key configured for the remote node doesn't match the private key they're using. Verify the public key was copied correctly. 13.5. "NO PACKET DRIVER FOUND" (DOS) Watt-32 can't find the packet driver. Make sure: - Packet driver is loaded (NE2000 0x60 3 0x300) - WATTCP.CFG exists with IP configuration - SET WATTCP.CFG= is set 13.6. CPU usage is high Check for stale .BSY files in the outbound directory. Comet auto-cleans BSY files older than 10 minutes. 13.7. Config changes not taking effect Comet checks the config file every 5 seconds. On Unix, you can force a reload with: kill -HUP $(pidof comet) ====================================================================== 14. SECURITY NOTES ====================================================================== 14.1. Configuration File Permissions The config file (comet.cfg) may contain passwords and private keys. On Unix, set permissions to 0600: chmod 600 comet.cfg Comet will warn at startup if the config file is world-readable. 14.2. Input Validation Comet validates all data received from remote peers: - Filenames are stripped of path components (/ and \) to prevent directory traversal attacks. A malicious remote cannot write files outside the inbound directory. - File sizes, resume offsets, and reposition offsets are all validated against actual file boundaries before any seek operation. - Received data is checked against the declared file size. A remote cannot send more data than advertised in FINFO. - FREQ (file request) patterns are sanitized to prevent requests for files outside the designated FREQ directory. - Protocol strings from INIT packets are capped to prevent memory exhaustion from oversized payloads. 14.3. Authentication Rate Limiting Comet tracks failed authentication attempts per IP address. After 5 failures within 5 minutes, further connections from that IP are temporarily rejected. This prevents brute-force password guessing. 14.4. Password Handling Passwords and key material are wiped from memory (zeroed) after authentication completes. This limits the window for memory-based credential theft. 14.5. Post-Session Command Safety Variables substituted into Exec commands (*A, *I, *N) are shell-quoted on Unix to prevent command injection. 14.6. WebUI The WebUI has its own separate security and configuration model, including a SQLite-backed user database, API keys, webhooks, and WebAuthn passwordless login. See COMETWEB.DOC for the complete WebUI operator manual. ====================================================================== 15. DATA COMPRESSION ====================================================================== Comet supports optional per-block zlib compression. When both sides advertise the COPT_ZLIB capability during INIT negotiation, each DATA block is individually compressed before transmission. 15.1. How It Works - Each block is compressed with zlib/deflate (level 6) - If the compressed block is smaller than the original, it is sent with compression type 1 (zlib) - If compression doesn't help (already-compressed data), the block is sent uncompressed with compression type 0 - The decision is made per-block, not per-file 15.2. What Compresses Well - .PKT files (netmail/echomail packets): 50-70% reduction - Text files, .TIC files: good compression - .ZIP, .RAR, .7Z, .GZ attachments: sent uncompressed (no overhead beyond the 1-byte type field) 15.3. Configuration Compression is enabled by default. To disable: [Protocol] Compression = none When the remote node does not support compression, transfers proceed uncompressed with the original wire format. No configuration is needed for interoperability. 15.4. Wire Format Without compression: offset(4) + data(N) With compression: offset(4) + comp_type(1) + data(N) comp_type: 0 = uncompressed, 1 = zlib ====================================================================== 16. TECHNICAL REFERENCE ====================================================================== 16.1. BSO Outbound Structure outbound/ Default zone outbound.NNN/ Zone NNN (3-digit hex) outbound/XXXXXXXX.PNT/ Point directory File naming: NNNNNNNN.ext where NNNNNNNN = (net<<16)|node in hex 16.2. Flow File (.FLO) Format /path/to/file Send normally ^/path/to/file Delete after send #/path/to/file Truncate after send ~/path/to/file Already sent (skip) ;comment Ignored 16.3. Comet Frame Format +--------+------+-----+------------------+-------+ | LEN | TYPE | SEQ | PAYLOAD | CRC32 | | 4B LE | 1B | 1B | 0..65528 bytes | 4B LE | +--------+------+-----+------------------+-------+ See FSP-COMET.001 for complete wire protocol specification. 16.4. Default Ports Comet: 24554 (standard BinkP port) BinkP: 24554 (standard) 16.5. Source Code 29 Pascal source files, approximately 16,000 lines. Built with Free Pascal Compiler (FPC) 3.2.x. Core units: cometdef.pas Protocol definitions and types comettcp.pas Cross-platform TCP socket abstraction cometfrm.pas Length-prefixed frame layer cometses.pas Session handshake (banner, INIT, INITACK) cometxfer.pas Bidirectional file transfer engine cometbinkp.pas BinkP/1.1 protocol implementation cometdaemon.pas Multi-session daemon core cometbso.pas BSO outbound directory handler cometnodelist.pas FTS-5 nodelist parser cometcfg.pas Configuration file parser cometio.pas Stream-based file I/O provider interface cometfile.pas File transfer helpers cometpath.pas Cross-platform path operations cometlog.pas Logging subsystem Crypto units: cometcrypt.pas Session encryption (X25519 + ChaCha20) cometmd5.pas MD5 message digest (RFC 1321) cometcram.pas CRAM-MD5 authentication (RFC 2104) cometsha.pas SHA-256/384/512 (FIPS 180-4) cometcrc.pas CRC-32 checksum cometed25519.pas ED25519 digital signatures cometed25519sc.pas Scalar arithmetic mod l cometed25519ge.pas Group element operations cometed25519bp.pas Precomputed base point table DOS-specific: cometwatt.pas Watt-32 TCP/IP bindings cometlibc.pas DJGPP libc emulation ====================================================================== For bug reports and updates: FidoNet: 1:218/720 Email: ken@rail-city.net Comet is free software released under the GPL-2.0 license. ====================================================================== ====================================================================== 17. EMBEDDING IN HOST APPLICATIONS (BBS SOFTWARE) ====================================================================== Comet is designed to be embedded in BBS and mailer software as a library. Two callback mechanisms allow the host application to intercept all output and receive structured session events. 17.1. Log Callback By default, Comet writes to log files and stdout. To route log output to your application instead: CometLogSetCallback(@MyLogHandler); CometLogSetBuiltin(False); { Suppress file/console output } Callback signature: procedure MyLogHandler(Level: TCometLogLevel; const Msg: string); Levels: cllDebug, cllInfo, cllWarning, cllError, cllFatal. All LogInfo/LogError/LogWarning calls route through this callback. 17.2. Event Callback For structured session events (file progress, session status): CometLogSetEventCallback(@MyEventHandler); Callback signature: procedure MyEventHandler(const Event: TCometEventData); Event types (TCometEventType): cetSessionStart Remote identified. Fields: RemoteName, RemoteAddr, RemoteMailer, Protocol. cetSessionAuth Authentication result. Fields: AuthMethod (AUTH_ED25519, AUTH_CRAM, AUTH_NOPWD), Encrypted (Boolean). cetFileStart File transfer starting. Fields: FileName, FileSize, Sending (True=TX, False=RX), Protocol. cetFileProgress Transfer progress update (max 4/sec). Fields: FileName, FileSize, Position, CPS, Sending. cetFileEnd File transfer complete. Fields: FileName, FileSize, Position, CPS, Sending. cetFileSkip File skipped by remote. Fields: FileName. cetSessionEnd Session complete. Fields: FilesSent, FilesRecvd, BytesSent, BytesRecvd, Success. The Protocol field is 'Comet' or 'BinkP' — same events for both. 17.3. Minimal Integration Example uses cometdef, cometlog, cometses, cometxfer, cometbinkp, cometio; procedure BBSLog(Level: TCometLogLevel; const Msg: string); begin BBSActivityLog(Msg); { Your BBS logging } end; procedure BBSEvent(const Event: TCometEventData); begin case Event.EventType of cetFileProgress: BBSStatusBar(Event.FileName, Event.Position, Event.FileSize, Event.CPS); cetSessionEnd: BBSSessionComplete(Event.FilesSent, Event.FilesRecvd); end; end; { During initialization: } CometLogSetCallback(@BBSLog); CometLogSetBuiltin(False); CometLogSetEventCallback(@BBSEvent); { Create a file provider for the protocol engines: } FileIO := TCometLocalFileProvider.Create; { Then call CometHandshake/CometTransfer or BinkpRunOutbound, passing FileIO — all output routes through your callbacks. } { Comet: } CometXferInit(XS, State, InboundDir, TempDir, AbortLogPath, FileIO); CometTransfer(XS, FileName, ''); { BinkP: } BResult := BinkpRunOutbound(Sock, Cfg, Addr, InboundDir, TempDir, SendEntries, FileIO); { Clean up when done: } FileIO.Free; 17.4. Authentication Constants AUTH_NONE = 0 Not yet determined AUTH_ED25519 = 1 ED25519 signature verified (encryption eligible) AUTH_CRAM = 2 CRAM-MD5 password verified (no encryption) AUTH_NOPWD = 3 Passwordless via COPT_NOPWD cap (no encryption) AUTH_PLAIN = 4 Plain text password (BinkP only) Encryption (X25519 + ChaCha20) is only enabled when AuthMethod is AUTH_ED25519. All other methods run unencrypted. 17.5. File I/O Provider (cometio.pas) As of version 1.2, both Comet and BinkP protocol engines access files through a stream provider interface (TCometFileProvider). The protocol code never touches the filesystem directly — it reads and writes TStream objects returned by the provider. The default provider (TCometLocalFileProvider) uses TFileStream for local filesystem access. BBS software can subclass TCometFileProvider to route file data anywhere — a database, a WebSocket connection, in-memory buffers, etc. Abstract class: TCometFileProvider function OpenForSend(const Path: string): TStream; Open a file for sending (TX). Returns a readable TStream positioned at byte 0. Returns nil on failure. Caller owns and frees the returned stream. function CreateForReceive(const TempDir: string; out TempPath: string): TStream; Create a new file for receiving (RX). Returns a writable TStream. TempPath receives the temporary file path for later finalization. Returns nil on failure. Caller owns and frees the returned stream. function OpenForResume(const Path: string; Offset: Int64): TStream; Open an existing partial file for resume. Returns a read/write TStream positioned at Offset. Returns nil if the file doesn't exist or Offset exceeds file size. Caller owns and frees the returned stream. function FinalizeReceive(const TempPath, InboundDir, OrigName: string; FileTime: LongInt): string; Move a completed temp file to its final location. Sets the file timestamp. Returns the final path on success, '' on failure. procedure CleanupReceive(const TempPath: string); Delete a failed/incomplete temp file. function GetFileSize(const Path: string): Int64; Returns file size in bytes, or -1 on failure. function GetFileTime(const Path: string): LongInt; Returns file modification time (Unix timestamp). function HashFile(const Path: string; out Digest: TSHA256Digest): Boolean; Compute SHA-256 of a file. Used before sending to populate the FINFO packet hash field. function HashStream(Stream: TStream; Len: Int64; out Digest: TSHA256Digest): Boolean; Compute SHA-256 of the first Len bytes of a stream. Used at EOF to verify received file integrity. Reads from position 0 and restores original position. function StreamSize(Stream: TStream): Int64; Returns the size of a stream. Default implementation returns Stream.Size. 17.6. Custom File Provider Example To implement a custom provider (e.g., for a BBS that stores files in a database), subclass TCometFileProvider: type TMyBBSFileProvider = class(TCometFileProvider) public function OpenForSend(const Path: string): TStream; override; function CreateForReceive(const TempDir: string; out TempPath: string): TStream; override; function OpenForResume(const Path: string; Offset: Int64): TStream; override; function FinalizeReceive(const TempPath, InboundDir, OrigName: string; FileTime: LongInt): string; override; procedure CleanupReceive(const TempPath: string); override; function GetFileSize(const Path: string): Int64; override; function GetFileTime(const Path: string): LongInt; override; function HashFile(const Path: string; out Digest: TSHA256Digest): Boolean; override; function HashStream(Stream: TStream; Len: Int64; out Digest: TSHA256Digest): Boolean; override; end; function TMyBBSFileProvider.OpenForSend( const Path: string): TStream; begin { Return any TStream descendant — TFileStream, TMemoryStream, a custom TDatabaseBlobStream, a TWebSocketStream, etc. The protocol engine just calls Read() on it. } Result := TFileStream.Create(Path, fmOpenRead or fmShareDenyNone); end; { ... implement remaining methods ... } Then pass your provider to the protocol engines: FileIO := TMyBBSFileProvider.Create; try CometXferInit(XS, State, InDir, TmpDir, AbortLog, FileIO); { ... transfer files ... } finally FileIO.Free; end; The same provider instance works for both Comet and BinkP. Both protocols use the identical interface. ====================================================================== 17.7. Triggering Outbound Calls (CallNodeExternal) Host applications can trigger outbound calls programmatically using the CallNodeExternal method on TCometDaemon. Simple form (uses BSO outbound scan): Daemon.CallNodeExternal(Addr); Daemon.CallNodeExternal(Addr, 'bbs.example.com', 24554); Full form (host-provided file list + stream provider): var Entries: array of TBinkpSendEntry; MyIO: TMyFileProvider; begin SetLength(Entries, 3); Entries[0].FilePath := 'netmail_001.pkt'; Entries[0].Action := csaDelete; Entries[1].FilePath := 'echomail_042.pkt'; Entries[1].Action := csaDelete; Entries[2].FilePath := 'nodediff.z42'; Entries[2].Action := csaNone; MyIO := TMyFileProvider.Create; Daemon.CallNodeExternal(Addr, Host, Port, Entries, MyIO); { Session runs in a thread. MyIO must stay valid until the session completes. Do NOT free MyIO here — wait for cetSessionEnd event. } end; Parameters: Addr FidoNet address to call (TCometAddress) Host Hostname/IP (empty = look up from config/nodelist) Port Port (0 = use config default) SendEntries Pre-built file list (bypasses BSO scan) FileIO Custom stream provider (host retains ownership) The SendEntries FilePath values are passed to FileIO.OpenForSend(). They can be real paths, database keys, URLs, or any identifier your FileIO implementation understands. Protocol selection (Comet vs BinkP) is automatic based on the node's config (NoComet setting) and remote detection. Thread-safe. Each call spawns a new session thread. TBinkpSendEntry record: FilePath string Passed to FileIO.OpenForSend() FloPath string .FLO file for cleanup ('' if N/A) FloLine string Original .FLO line for marking Action TCometSendAction csaNone, csaDelete, csaTruncate 18. CHANGE LOG ====================================================================== Version 1.2.1 (2026-04-21) --------------------------- BinkP embedder symmetry with Comet native: - TBinkpPostAuthCallback fires at the bsTransfer entry point — the equivalent of binkd's complete_login / select_inbound (protocol.c:1544-1573). Host applications can route secure vs unsecure inbound BEFORE any M_FILE arrives, matching the pre-transfer decision point Comet native already offers via its phased CometHandshake / CometXferInit API. - TCometBinkpResult gains Authenticated and AuthMethod fields for consumers that prefer to route post-session. - Daemon itself uses the new callback: InDir starts as Cfg.Inbound (unsecure) and promotes to Cfg.SecInbound only after the peer authenticates — fixes a pre-1.2.1 bug where SecInbound was selected unconditionally when configured, causing unauthenticated sessions to land in the secure dir. Version 1.2-2 (2026-04-21) --------------------------- BinkP full Argus/binkd parity. The BinkP stack now implements every major extension either reference mailer supports: - PLZ compression (Argus zlib, bit-14 frame flag) with adaptive block sizing that grows from 4KB toward the 14-bit PLZ max (16383 B) as the observed compression ratio warrants. - NR mode (FSP-1029) resume is fully honored: crashed transfers leave a `.bkp-part` partial in the temp directory, and the next session's M_FILE "-1" probe sends M_GET with the partial size so the peer resumes from there. - ND / NDA (FSP-1038) deferred cleanup: when the peer advertises ND we stash M_GOT receipts in a pending list and only apply the destructive cleanup after the session closes cleanly. A mid-session abort preserves the outbound for the next attempt. - MBT multi-batch (second-EOB handshake): a FREQ response served after the first EOB now rides the same session instead of being queued for the next poll. - M_NUL TRF advisory (FRL-1026) and M_NUL FREQ advisory when a .req file is queued. - M_NUL NDL / PHN info strings driven by new [System] config fields (NodelistFlags, Phone). - RFC 2822 date format for M_NUL TIME (replaces the previous locale-dependent format). - Strict M_GET validation: name/size/time must match the active TX file before we honor the resume offset. - Duplicate-file pre-check: incoming M_FILE that names a file already in the inbound with matching size + modtime is acknowledged immediately with M_GOT, skipping the receive. - EXTCMD kept in the OPT advertisement (binkd's prerequisite for GZ/BZ2 compression; Argus ignores it). Comet native protocol augmentations. The goal: keep Comet "truly augmented" relative to BinkP — if a feature exists in BinkP, Comet should match or beat it. - INIT payload extended (backward-compatible trailing strings) with Location, Time, Phone, and NodelistFlags. Older peers stop reading at Mailer and leave the new fields empty. - CometRFCDateStr shared between BinkP TIME and Comet INIT.Time so the two protocols' time strings cannot drift. - LST: new packet types NPKT_LSTREQ / NPKT_LSTITEM / NPKT_LSTEND provide a structured file-listing query. Server enumerates the FREQ directory and aliases for each request. New capability flag COPT_LST. - Transactional file cleanup (ND-equivalent). Destructive cleanup for successfully transferred files is deferred to session close; XFER_ABORT during the transfer loop preserves the outbound for the next attempt. Shared infrastructure: - TCometFileProvider gains GetPartialSize and OpenForReceiveNamed for NR-mode partial tracking. Providers that don't implement them fall back to the random-name scheme automatically — no plugin breakage. - Config: new Phone and NodelistFlags fields in [System]. - WebUI lands: /src/web/ HTML+JS+OpenAPI + /src/webui/ backend. Modeled after the Argus GUI: live session activity, outbound polls, FREQ requests, nodelist browser, config editor, scheduler. SSE event stream keeps the dashboard updated in real time; OpenAPI spec exposes the same endpoints for host applications. Tested against: - Argus (PLZ, MBT, ND, CRYPT) - binkd (NR, CRAM-MD5) Version 1.2 (2026-04-09) ------------------------- BinkP state machine aligned with Argus/Radius (p_binkp.pas): - CRAM-MD5 challenge sent in separate M_NUL (FTS-1027 compliance) - Answerer defers M_ADR until originator's M_ADR received - CRAM challenge parsed once as raw bytes (not reparsed from string) - Strip trailing NUL bytes from BinkP command frames (Argus compat) - State guard on OPT parsing failure BinkP multi-file transfer fix: - Added SentFiles tracking list (matches binkd's sent_fls). After sending all data for a file, close immediately and start next file — don't wait for M_GOT. Fixes frame ordering bug where M_FILE for the next file was sent before the last data frame of the previous file. - M_GOT/M_SKIP now match by filename against SentFiles list - EOB only sent when send queue empty AND all SentFiles confirmed - M_FILE receive handler: finalize complete files, skip only if actually incomplete (was always skipping) Daemon improvements: - Unified cetSessionEnd events: identical for BinkP and Comet, fired from daemon. Catch-all for failed handshakes prevents WFC node leaks in host applications. - Replace Sleep(100) with select() on listener socket — inbound connections accepted immediately instead of up to 100ms delay. Host application API: - New CallNodeExternal method on TCometDaemon for triggering outbound calls from host applications. Accepts optional pre-built send queue and custom TCometFileProvider. - FFileIO changed to TCometFileProvider base class (was hardcoded to TCometLocalFileProvider). Host-provided FileIO not freed by session thread (host retains ownership). New commands: - showkey: print public key derived from configured private key - keygen: updated output format (PublicKey shown as config comment) Tested against: - Argus/Radius 4.010 (CRAM-MD5 + CRYPT, 48 files inbound) - BinkIT/sbbs 2.42 (CRAM-MD5) - 5-node concurrent bidirectional: 45 files SHA256 verified Version 1.01 (2026-04-02) ------------------------- - Fixed bidirectional file transfers hanging or timing out, particularly over internet links and on FreeBSD systems. - Fixed file transfer resume reliability. - Improved BinkP inbound bidirectional support — the daemon now sends queued files back to the caller during inbound sessions. - Various stability improvements for long-running daemon sessions. Version 1.00 (2026-03-30) ------------------------- - Initial release.