Files
comet/COMET.DOC
Ken Johnson da732a10bd
Some checks failed
Build and Release / build-and-release (push) Failing after 13m54s
Version 1.2.1: full BinkP/Argus parity, Comet augmentation, WebUI
Version scheme: Major.Minor.Build-Revision.

BinkP gains every major Argus/binkd extension:

- PLZ (zlib) compression with adaptive block sizing (4KB→16KB)
- NR mode inbound resume via .bkp-part partials (FSP-1029)
- ND/NDA deferred cleanup: mid-session abort preserves outbound (FSP-1038)
- MBT multi-batch: FREQ response rides same session via second EOB
- M_NUL TRF traffic advisory and M_NUL FREQ (FRL-1026)
- M_NUL NDL/PHN info strings (new Phone, NodelistFlags config)
- RFC 2822 date format for M_NUL TIME
- Strict M_GET validation and duplicate-file pre-check
- TBinkpPostAuthCallback: host can route InboundDir before transfer
  (models binkd select_inbound / complete_login)
- TCometBinkpResult: Authenticated / AuthMethod fields

Comet native extensions keep the protocol ahead of BinkP:

- INIT payload adds Location/Time/Phone/NodelistFlags (trailing
  strings, backward-compatible)
- LST file listing: NPKT_LSTREQ/LSTITEM/LSTEND + COPT_LST
- Transactional file cleanup: destructive actions deferred until
  successful session close (matches ND semantics)
- Shared CometRFCDateStr across protocols — no drift between
  BinkP TIME and Comet INIT.Time

Daemon:
- BinkP inbound now starts unsecure and promotes to secure only
  after auth (fixes pre-1.2.1 bug where SecInbound was selected
  unconditionally).

TCometFileProvider: GetPartialSize and OpenForReceiveNamed for
NR partials; defaults preserve the random-temp scheme for
providers that don't track partials (Fastway plugin safe).

WebUI: /src/web/ + /src/webui/ backend, modeled after the Argus
GUI. Live session activity, outbound polls, FREQ requests,
nodelist, config editor, scheduler, SSE event stream.
2026-04-21 09:37:03 -07:00

1412 lines
48 KiB
Plaintext

======================================================================
COMET MAILER - OPERATOR MANUAL
Version 1.2
Direct TCP File Transfer for FidoNet
Copyright (C) 2026 Ken Johnson, 1:218/720
======================================================================
TABLE OF CONTENTS
1. Introduction
2. System Requirements
3. Installation
4. Quick Start
5. Command Reference
6. Configuration Reference
7. Authentication
8. Nodelist Integration
9. File Requests (FREQ)
10. Post-Session Events
11. BinkP Compatibility
12. Cross-Platform Notes
13. Troubleshooting
14. Security Notes
15. Data Compression
16. Technical Reference
17. Embedding in Host Applications
18. Change Log
======================================================================
1. INTRODUCTION
======================================================================
Comet is a modern FidoNet TCP mailer daemon designed for the 2020s.
It provides:
- Direct TCP file transfer with SHA-256 verification
- Bidirectional transfer with sliding window flow control
- ED25519 public-key authentication (no shared secrets needed)
- CRAM-MD5 password authentication (passwords never sent in clear)
- X25519 + ChaCha20 session encryption (with ED25519 auth)
- Full BinkP/1.1 compatibility (FTS-1026) with auto-detection
- Inline file requests (Hydra-style, within active sessions)
- FTS-5 nodelist integration (automatic IP/port lookup)
- BSO outbound scanning with point directory support
- Multi-session daemon with per-node BSY locking
- Config hot-reload (edit config while Comet is running)
- Cross-platform: Linux, FreeBSD, DOS, Windows, OS/2
Comet uses TCP port 24554, the standard BinkP port. Both
Comet native and BinkP connections are accepted on the same port.
The protocol specification is in FSP-COMET.001.
======================================================================
2. SYSTEM REQUIREMENTS
======================================================================
Linux / FreeBSD:
- Any modern x86 or x86-64 system
- TCP/IP networking
- Recommended: /dev/urandom for ED25519 key generation
DOS (FreeDOS recommended):
- 386 or better with DPMI (CWSDPMI.EXE)
- Packet driver for your network card
- Watt-32 TCP/IP (WATTCP.CFG)
- Approximately 1 MB RAM for the executable
Windows:
- Windows 7 or later (64-bit)
- WinSock2 (included with Windows)
OS/2:
- OS/2 Warp or eComStation with TCP/IP stack
======================================================================
3. INSTALLATION
======================================================================
3.1. Linux / FreeBSD
1. Copy the 'comet' binary to /usr/local/bin/ or your preferred
location.
2. Copy COMET.SAM to /etc/comet/comet.cfg (or wherever you
prefer) and edit it for your system.
3. Create the required directories:
mkdir -p /var/lib/fidonet/{inbound,secure,outbound,temp}
4. Generate an ED25519 keypair:
comet keygen
5. Add the PrivateKey to your config file.
6. Start the daemon:
comet -c /etc/comet/comet.cfg
7. To run as a system service, create a systemd unit file or
add to /etc/rc.local.
3.2. DOS
1. Copy COMET.EXE and CWSDPMI.EXE to your Comet directory
(e.g., F:\COMET\).
2. Copy COMET.SAM to COMET.CFG and edit it.
3. Create directories: INBOUND, SECURE, OUTBOUND, TEMP
4. Configure WATTCP.CFG for your network:
my_ip = dhcp
or:
my_ip = 192.168.1.100
netmask = 255.255.255.0
gateway = 192.168.1.1
5. Set the environment variable:
SET WATTCP.CFG=F:\COMET
6. Load your packet driver:
NE2000 0x60 3 0x300
7. Run Comet:
COMET CALL 1:213/723
3.3. Signal Handling (Unix)
SIGTERM / SIGINT : Clean shutdown (waits for active sessions)
SIGHUP : Reload configuration
SIGPIPE : Ignored (handled internally)
======================================================================
4. QUICK START
======================================================================
4.1. Daemon Mode (listen + poll outbound)
comet
comet -c /path/to/comet.cfg
4.2. Single Call
comet call 1:213/723
comet -c /path/to/comet.cfg call 1:218/720
4.3. Generate ED25519 Keypair
comet keygen
This generates a random keypair and displays:
- PrivateKey: add to [System] section of your config
- PublicKey: give to remote nodes (they add to [Node:] section)
4.4. Show Version
comet -v
4.5. Show Help
comet -h
4.6. Debug Mode
comet -d
comet -d call 1:213/723
Enables verbose protocol-level logging.
======================================================================
5. COMMAND REFERENCE
======================================================================
comet [options] [command]
Commands:
(none) Run as daemon (listen + poll outbound)
call <address> Single outbound call to a FidoNet node
keygen Generate ED25519 keypair
showkey Print public key from configured private key
Options:
-c <file> Use specified config file (default: comet.cfg)
-d Enable debug/trace logging
-q Quiet mode (errors only on console)
-v Show version and exit
-h, --help Show help and exit
Address format:
zone:net/node e.g., 1:213/723
zone:net/node.point e.g., 1:213/723.1
======================================================================
6. CONFIGURATION REFERENCE
======================================================================
Configuration is read from comet.cfg (or the file specified with
-c). The file uses INI-style sections with key = value pairs.
Lines beginning with ; are comments.
Comet monitors the config file for changes and automatically
reloads when modifications are detected (checked every 5 seconds).
On Unix, you can also send SIGHUP to force a reload.
6.1. [System] Section
Address = 1:218/720
Your FidoNet address. Multiple addresses supported (one per
line). First address is primary.
SysOp = Ken Johnson
Your name as system operator.
SystemName = The Danger Zone
Your system/BBS name.
Location = Sparks, NV
City, state/country.
PrivateKey = <64 hex characters>
Your ED25519 private key. Generate with: comet keygen
Keep this SECRET. Never share it.
6.2. [Network] Section
Port = 24554
TCP port to listen on. Default: 24554.
Bind =
Network interface to bind to. Empty = all interfaces.
MaxSessions = 5
Maximum simultaneous sessions. Range: 1-32.
MaxSessionsPerIP = 0
Optional cap on simultaneous inbound sessions from a single
IP address. 0 = no per-IP cap (default). Applies only to
inbound — outbound calls are unaffected.
MaxSessionTime = 0
Optional wall-clock limit (in seconds) per session. Any
session running longer than this is force-terminated. 0 =
no limit (default). Protects against stuck transfers and
slow-drip attackers.
PollInterval = 60
Outbound poll interval in seconds. 0 = no polling.
6.3. [Paths] Section
Inbound = /var/lib/fidonet/inbound/
Directory for files from unauthenticated nodes.
SecInbound = /var/lib/fidonet/secure/
Directory for files from password-verified nodes.
Outbound = /var/lib/fidonet/outbound/
BSO outbound base directory.
Temp = /var/lib/fidonet/temp/
Temporary files during transfer.
LogFile = /var/log/fidonet/comet.log
Main log file path.
DebugLog = /var/log/fidonet/comet-debug.log
Debug/trace log (very verbose).
Nodelist = /var/lib/fidonet/nodelist/
Directory containing nodelist files.
Freq = /var/lib/fidonet/freq/
Directory for file request searches.
Flags = /var/lib/fidonet/flags/
Semaphore/flag files directory.
6.4. [DriveMap] Section (Linux/FreeBSD only)
Maps DOS drive letters to Unix paths for reading .FLO files
written by DOS mailers.
C = /home/user/dos/c
D = /home/user/dos/d
6.5. [Protocol] Section
MaxBlockSize = 65528
Maximum data block size in bytes (512-65528).
WindowSize = 8
Sliding window depth (1-16).
Timeout = 120
Braindead timeout in seconds.
HandshakeTimeout = 30
Handshake timeout in seconds.
NoSHA256 = no
Disable SHA-256 file verification (NOT recommended).
NoFREQ = no
Disable file request support.
6.6. [BinkP] Section
Enabled = yes
Enable BinkP fallback for outbound connections.
Port = 24554
Default BinkP port for fallback connections.
6.7. [Logging] Section
FileLevel = info
Minimum log level for file: debug, info, warning, error, fatal
ConsoleLevel = info
Minimum log level for console.
Debug = no
Enable debug/trace logging.
6.8. [Events] Section
flag <flagfile> <pattern>
Create a semaphore file when files matching pattern are received.
exec "<command>" <pattern>
Run external command when files matching pattern are received.
Variables for exec commands:
*A = Remote FidoNet address
*I = Inbound directory path
*N = Number of files received
Examples:
flag /var/lib/fidonet/flags/toss.now *.pkt
exec "hpt toss" *.pkt
exec "allfix RP -SRIF *S" *.req
6.9. [Node:address] Sections
Per-node configuration. Section name includes the address:
[Node:1:213/723]
[Node:21:1/100]
Password = SECRET
Session password (CRAM-MD5 authenticated, never sent in clear).
PublicKey = <64 hex characters>
Remote node's ED25519 public key (from their sysop).
Host = bbs.example.com
IP address or hostname.
Port = 24554
Outbound port override. If not set, the port from the
nodelist (IBN flag) is used. Default: 24554.
NoBinkp = no
Disable BinkP fallback for this node.
NoComet = no
Use only BinkP for this node (skip Comet protocol).
CallWindows = mon-fri 09:00-17:00
Time restriction for outbound polls. Empty (default) = always
allowed. Format: "dayspec HH:MM-HH:MM" with multiple windows
separated by ';'.
Days (case-insensitive): sun mon tue wed thu fri sat, or '*'
for every day. Ranges "mon-fri" and lists "sat,sun" are
supported. Times are 24-hour local time. Wrap-around
"22:00-02:00" (overnight) is supported.
Examples:
"*" every day, all hours (default)
"mon-fri 09:00-17:00" weekdays 9am-5pm
"* 22:00-06:00" overnight only
"mon-fri 09:00-17:00; sat 10:00-15:00"
mixed schedule
The poller and the [Scheduler] poll_all action honour this
restriction. Manual "Call" from the WebUI bypasses it, so the
operator can always force a call when needed.
HookPreCall = command
HookOnSuccess = command
HookOnFail = command
Per-node external program hooks. Each is a shell command
template; variables are substituted at fire time and
shell-quoted on Unix to prevent injection.
Variables:
*A = node address (e.g. 1:218/720)
*N = remote system name (after handshake)
*S = remote sysop name
*I = inbound directory
*F = files received this session
*X = files sent this session
*B = total bytes (sent + received)
*R = remote mailer name
HookPreCall fires before the call attempt — *F/*X/*B will be
0. Use for pre-flight notifications, dial-up activation,
session logging.
HookOnSuccess fires after a successful session, after files
have been moved into the inbound directory. Use for tossing
mail, syncing files, success notifications.
HookOnFail fires after a connect failure, handshake reject,
or session-thread exception. Use for alerts, retry counters,
escalation.
These hooks are independent of the global [Events] section,
which fires for ALL sessions. Per-node hooks fire only for
the matching address.
Examples:
HookPreCall = logger -t comet "Calling *A"
HookOnSuccess = /usr/local/bin/toss-incoming.sh *I *F
HookOnFail = /usr/local/bin/page-oncall.sh *A "Comet failed"
======================================================================
7. AUTHENTICATION
======================================================================
Comet supports three authentication methods, tried in order of
strength:
7.1. ED25519 Digital Signatures (Strongest)
Public-key authentication. Each node generates a keypair:
- Private key: stays on your system, never shared
- Public key: given to remote nodes
Setup:
1. Run: comet keygen
2. Add PrivateKey to your [System] section
3. Give your PublicKey to remote sysops
4. They add it to their [Node:your_address] section
5. They give you their PublicKey
6. You add it to your [Node:their_address] section
No passwords needed. The private key never leaves your system
and cannot be derived from the public key.
7.2. CRAM-MD5 (Shared Secret)
Challenge-response authentication per RFC 2104. Both sides must
know the same password. The password is NEVER sent in plain text.
Setup:
1. Agree on a password with the remote sysop
2. Add Password = SECRETWORD to both [Node:] sections
7.3. No Authentication
If neither ED25519 keys nor passwords are configured, the session
proceeds without authentication. Files are placed in the unsecure
Inbound directory. This is logged as a warning.
7.4. Authentication Fallback
When connecting, Comet tries methods in order:
ED25519 -> CRAM-MD5 -> Plain password -> No auth
Only available methods are advertised to the remote. If ED25519
is not configured, it is not offered.
7.5. Session Encryption
When both sides support encryption and ED25519 authentication
succeeds, Comet automatically encrypts the session using
X25519 ephemeral key exchange and ChaCha20 stream cipher.
- Encryption is automatic: no additional configuration needed
beyond setting up ED25519 keys (see 7.1 above).
- Forward secrecy: each session generates fresh ephemeral keys,
so compromising a long-term private key does not reveal the
content of past sessions.
- Encryption requires ED25519: CRAM-MD5 and passwordless
sessions are not encrypted because there is no identity
verification to prevent man-in-the-middle attacks.
- Interoperable: if the remote does not support encryption,
the session proceeds unencrypted with no configuration
changes needed.
======================================================================
8. NODELIST INTEGRATION
======================================================================
Comet reads standard FidoNet raw text nodelists (FTS-5 format).
8.1. Setup
Set the Nodelist path in [Paths]:
Nodelist = /var/lib/fidonet/nodelist/
Place nodelist files (e.g., NODELIST.081) in this directory.
Comet automatically loads the most recent one.
8.2. How It Works
When polling outbound or making a call, if no [Node:] section
exists for the target address, Comet looks it up in the nodelist.
It extracts:
- INA: flag for IP address/hostname
- IBN: flag for BinkP port (default 24554)
[Node:] entries always take priority over nodelist data.
8.3. Nodelist Reload
The nodelist is reloaded when the config file changes (hot-reload).
======================================================================
9. FILE REQUESTS (FREQ)
======================================================================
9.1. BinkP FREQ
Standard .REQ file mechanism. The .REQ file is sent as a regular
file. When the remote receives it, they process the patterns and
send matching files back.
To make a request: create a .REQ file in the outbound directory
for the target node (e.g., 00D502D3.REQ for 1:213/723).
Format: one filename pattern per line.
9.2. Comet FREQ (Inline)
Comet protocol supports Hydra-style inline file requests via
NPKT_FREQ packets. Requests are processed immediately during the
session without requiring a separate .REQ file transfer.
9.3. FREQ Processing
When a file request is received, Comet first checks the
[FreqAliases] table for a case-insensitive name match. If found,
the alias's pattern is served (which may live anywhere on disk —
aliases are trusted config data). If not, Comet searches the Freq
directory for matching files.
Pattern sanitization rejects any remote-supplied pattern containing
path separators (/, \), parent-dir refs (..), control characters,
or absolute paths. After matching, every served file is verified
to actually live inside FreqDir as a defense-in-depth check
against symlinks or unusual filesystem layouts.
Limits: 10 files, 10 MB per request.
For advanced FREQ processing (file areas, per-link access control),
use an external FREQ processor via [Events] exec rules with SRIF.
9.4. Setup
Set the Freq path in [Paths]:
Freq = /var/lib/fidonet/freq/
Place files you want to make available in this directory.
9.5. FREQ Aliases (Magic Names)
Define short alias names that remote nodes can request via FREQ
instead of knowing the real filename:
[FreqAliases]
NODELIST = >/var/lib/fidonet/nodelist/nodelist.*
POINTLIST = >/var/lib/fidonet/nodelist/points24.*
ARCHIVE = /var/lib/fidonet/archive/*.zip
MAGIC = /var/lib/fidonet/freq/welcome.txt
FILES = /var/lib/fidonet/freq/files.bbs
PUBKEY = /etc/comet/pubkey.txt
Aliases are matched case-insensitively. The pattern can be a
literal path or a glob (* and ? wildcards). Anything not matching
an alias falls back to a normal FreqDir search.
Glob behaviour matches Argus and BTXE:
- By default, all files matching the glob are sent, up to the
10-file / 10 MB per-request limits.
- Prefix the pattern with '>' to serve only the NEWEST file(s)
by mtime. Use this for NODELIST-style magic names where the
remote wants today's file, not every archived copy.
- A literal path serves that one file as-is.
Extended form — per-alias password, limits, and external
processor (xenia-mailer compatible feature set):
ALIAS = <pattern> | pw=<password> | max_size=<N>
| max_count=<N> | exec=<command>
pw=X Password-protect the alias. Remote must send
"ALIAS!password" on the request line (FTN
convention). Without the password, the alias
returns nothing.
max_size=X Per-alias total size cap. Accepts K/M/G suffix
(e.g. 5M = 5,242,880 bytes). Overrides the
global 10MB default when lower.
max_count=N Per-alias file count cap. Overrides the
global 10-file default when lower.
exec=CMD External FREQ processor (SRIF-style). When set,
the pattern field is ignored. CMD is spawned
with three arguments: the alias name, the
password the remote supplied (or empty), and
the remote address. Whatever absolute file
paths the program prints on stdout (one per
line) are served back. Lines beginning with
'#' are comments. Stdout is capped at 256KB
and the per-alias / global size+count limits
still apply.
Use this for database-backed file areas,
dynamic "latest build" generators, custom
access control, or wrappers around legacy SRIF
processors.
On Unix the command goes through /bin/sh -c so
pipes and redirection work in the config. On
Windows it is passed directly to ExecuteProcess.
Examples:
[FreqAliases]
PRIVATE = /data/private/*.zip | pw=sekret | max_size=50M
LATEST = | exec=/usr/local/bin/comet-latest-build.sh
FILEAREA = | exec=/usr/local/bin/comet-freq-db.py | max_count=20
Aliases are TRUSTED config data and may point to files outside
the FreqDir — useful for serving system files (nodelist, pubkey)
without exposing the whole directory.
======================================================================
10. POST-SESSION EVENTS
======================================================================
After a session completes and files are received, Comet can:
10.1. Create Flag Files
[Events]
flag /var/lib/fidonet/flags/toss.now *.pkt
Creates the specified file when files matching the pattern are
received. Your tosser watches for this flag.
10.2. Run External Commands
[Events]
exec "hpt toss" *.pkt
exec "allfix RP -SRIF *S" *.req
Commands run AFTER the session is closed - they never block
active transfers.
======================================================================
11. BINKP COMPATIBILITY
======================================================================
Comet is fully compatible with BinkP/1.1 (FTS-1026).
11.1. Auto-Detection
Both Comet and BinkP connections are accepted on the same TCP
port. The protocol is auto-detected from the first bytes received.
11.2. Fallback
When calling a node, Comet connects on the port from the
nodelist (IBN flag) or the per-node Port setting. Both
protocols are auto-detected by the remote.
Set NoComet = yes for nodes that only speak BinkP.
11.3. Features in BinkP Mode
- CRAM-MD5 authentication (FTS-1027)
- ED25519 authentication (via OPT extension)
- M_GET file resume
- FREQ via .REQ file transfer
- OPT negotiation (NR, EXTCMD)
11.4. Tested Against
- binkd (reference BinkP implementation)
- Radius/Argus 4.010 (Windows GUI mailer)
======================================================================
12. CROSS-PLATFORM NOTES
======================================================================
12.1. Linux (Primary Platform)
Full support. Production ready.
12.2. FreeBSD
Cross-compiled from Linux. Same features as Linux.
Build: make freebsd
12.3. DOS (FreeDOS)
Requires CWSDPMI.EXE and Watt-32 TCP/IP with packet driver.
Single-session mode (no threading).
Build: make dos
12.4. Windows
WinSock2 code written, needs type compatibility fixes.
Build: make win64 (work in progress)
12.5. OS/2
Should work via FPC's OS/2 Sockets unit. Untested.
12.6. Path Handling
Comet handles DOS/Unix path conversion automatically:
- Backslash <-> forward slash conversion
- Drive letter mapping via [DriveMap]
- Case-insensitive file lookup on Unix
======================================================================
13. TROUBLESHOOTING
======================================================================
13.1. "Cannot listen on port 24554"
Another process is using the port, or a previous Comet instance
hasn't fully shut down. Wait a few seconds and try again, or
check with: netstat -tlnp | grep 24554
13.2. "No host configured for X:XXX/XXX"
The node has no [Node:] section and isn't in the nodelist.
Add a [Node:] section with Host, or set the Nodelist path.
13.3. "Password mismatch"
CRAM-MD5 passwords don't match. Verify both sides have the
same password configured.
13.4. "ED25519 verification failed"
The public key configured for the remote node doesn't match
the private key they're using. Verify the public key was
copied correctly.
13.5. "NO PACKET DRIVER FOUND" (DOS)
Watt-32 can't find the packet driver. Make sure:
- Packet driver is loaded (NE2000 0x60 3 0x300)
- WATTCP.CFG exists with IP configuration
- SET WATTCP.CFG=<directory> is set
13.6. CPU usage is high
Check for stale .BSY files in the outbound directory.
Comet auto-cleans BSY files older than 10 minutes.
13.7. Config changes not taking effect
Comet checks the config file every 5 seconds. On Unix,
you can force a reload with: kill -HUP $(pidof comet)
======================================================================
14. SECURITY NOTES
======================================================================
14.1. Configuration File Permissions
The config file (comet.cfg) may contain passwords and private
keys. On Unix, set permissions to 0600: chmod 600 comet.cfg
Comet will warn at startup if the config file is world-readable.
14.2. Input Validation
Comet validates all data received from remote peers:
- Filenames are stripped of path components (/ and \) to
prevent directory traversal attacks. A malicious remote
cannot write files outside the inbound directory.
- File sizes, resume offsets, and reposition offsets are all
validated against actual file boundaries before any seek
operation.
- Received data is checked against the declared file size.
A remote cannot send more data than advertised in FINFO.
- FREQ (file request) patterns are sanitized to prevent
requests for files outside the designated FREQ directory.
- Protocol strings from INIT packets are capped to prevent
memory exhaustion from oversized payloads.
14.3. Authentication Rate Limiting
Comet tracks failed authentication attempts per IP address.
After 5 failures within 5 minutes, further connections from
that IP are temporarily rejected. This prevents brute-force
password guessing.
14.4. Password Handling
Passwords and key material are wiped from memory (zeroed)
after authentication completes. This limits the window for
memory-based credential theft.
14.5. Post-Session Command Safety
Variables substituted into Exec commands (*A, *I, *N) are
shell-quoted on Unix to prevent command injection.
14.6. WebUI
The WebUI has its own separate security and configuration model,
including a SQLite-backed user database, API keys, webhooks,
and WebAuthn passwordless login. See COMETWEB.DOC for the
complete WebUI operator manual.
======================================================================
15. DATA COMPRESSION
======================================================================
Comet supports optional per-block zlib compression. When both
sides advertise the COPT_ZLIB capability during INIT negotiation,
each DATA block is individually compressed before transmission.
15.1. How It Works
- Each block is compressed with zlib/deflate (level 6)
- If the compressed block is smaller than the original, it is
sent with compression type 1 (zlib)
- If compression doesn't help (already-compressed data), the
block is sent uncompressed with compression type 0
- The decision is made per-block, not per-file
15.2. What Compresses Well
- .PKT files (netmail/echomail packets): 50-70% reduction
- Text files, .TIC files: good compression
- .ZIP, .RAR, .7Z, .GZ attachments: sent uncompressed
(no overhead beyond the 1-byte type field)
15.3. Configuration
Compression is enabled by default. To disable:
[Protocol]
Compression = none
When the remote node does not support compression, transfers
proceed uncompressed with the original wire format. No
configuration is needed for interoperability.
15.4. Wire Format
Without compression: offset(4) + data(N)
With compression: offset(4) + comp_type(1) + data(N)
comp_type: 0 = uncompressed, 1 = zlib
======================================================================
16. TECHNICAL REFERENCE
======================================================================
16.1. BSO Outbound Structure
outbound/ Default zone
outbound.NNN/ Zone NNN (3-digit hex)
outbound/XXXXXXXX.PNT/ Point directory
File naming: NNNNNNNN.ext where NNNNNNNN = (net<<16)|node in hex
16.2. Flow File (.FLO) Format
/path/to/file Send normally
^/path/to/file Delete after send
#/path/to/file Truncate after send
~/path/to/file Already sent (skip)
;comment Ignored
16.3. Comet Frame Format
+--------+------+-----+------------------+-------+
| LEN | TYPE | SEQ | PAYLOAD | CRC32 |
| 4B LE | 1B | 1B | 0..65528 bytes | 4B LE |
+--------+------+-----+------------------+-------+
See FSP-COMET.001 for complete wire protocol specification.
16.4. Default Ports
Comet: 24554 (standard BinkP port)
BinkP: 24554 (standard)
16.5. Source Code
29 Pascal source files, approximately 16,000 lines.
Built with Free Pascal Compiler (FPC) 3.2.x.
Core units:
cometdef.pas Protocol definitions and types
comettcp.pas Cross-platform TCP socket abstraction
cometfrm.pas Length-prefixed frame layer
cometses.pas Session handshake (banner, INIT, INITACK)
cometxfer.pas Bidirectional file transfer engine
cometbinkp.pas BinkP/1.1 protocol implementation
cometdaemon.pas Multi-session daemon core
cometbso.pas BSO outbound directory handler
cometnodelist.pas FTS-5 nodelist parser
cometcfg.pas Configuration file parser
cometio.pas Stream-based file I/O provider interface
cometfile.pas File transfer helpers
cometpath.pas Cross-platform path operations
cometlog.pas Logging subsystem
Crypto units:
cometcrypt.pas Session encryption (X25519 + ChaCha20)
cometmd5.pas MD5 message digest (RFC 1321)
cometcram.pas CRAM-MD5 authentication (RFC 2104)
cometsha.pas SHA-256/384/512 (FIPS 180-4)
cometcrc.pas CRC-32 checksum
cometed25519.pas ED25519 digital signatures
cometed25519sc.pas Scalar arithmetic mod l
cometed25519ge.pas Group element operations
cometed25519bp.pas Precomputed base point table
DOS-specific:
cometwatt.pas Watt-32 TCP/IP bindings
cometlibc.pas DJGPP libc emulation
======================================================================
For bug reports and updates:
FidoNet: 1:218/720
Email: ken@rail-city.net
Comet is free software released under the GPL-2.0 license.
======================================================================
======================================================================
17. EMBEDDING IN HOST APPLICATIONS (BBS SOFTWARE)
======================================================================
Comet is designed to be embedded in BBS and mailer software as a
library. Two callback mechanisms allow the host application to
intercept all output and receive structured session events.
17.1. Log Callback
By default, Comet writes to log files and stdout. To route log
output to your application instead:
CometLogSetCallback(@MyLogHandler);
CometLogSetBuiltin(False); { Suppress file/console output }
Callback signature:
procedure MyLogHandler(Level: TCometLogLevel; const Msg: string);
Levels: cllDebug, cllInfo, cllWarning, cllError, cllFatal.
All LogInfo/LogError/LogWarning calls route through this callback.
17.2. Event Callback
For structured session events (file progress, session status):
CometLogSetEventCallback(@MyEventHandler);
Callback signature:
procedure MyEventHandler(const Event: TCometEventData);
Event types (TCometEventType):
cetSessionStart Remote identified. Fields: RemoteName,
RemoteAddr, RemoteMailer, Protocol.
cetSessionAuth Authentication result. Fields: AuthMethod
(AUTH_ED25519, AUTH_CRAM, AUTH_NOPWD),
Encrypted (Boolean).
cetFileStart File transfer starting. Fields: FileName,
FileSize, Sending (True=TX, False=RX),
Protocol.
cetFileProgress Transfer progress update (max 4/sec).
Fields: FileName, FileSize, Position,
CPS, Sending.
cetFileEnd File transfer complete. Fields: FileName,
FileSize, Position, CPS, Sending.
cetFileSkip File skipped by remote. Fields: FileName.
cetSessionEnd Session complete. Fields: FilesSent,
FilesRecvd, BytesSent, BytesRecvd, Success.
The Protocol field is 'Comet' or 'BinkP' — same events for both.
17.3. Minimal Integration Example
uses cometdef, cometlog, cometses, cometxfer, cometbinkp, cometio;
procedure BBSLog(Level: TCometLogLevel; const Msg: string);
begin
BBSActivityLog(Msg); { Your BBS logging }
end;
procedure BBSEvent(const Event: TCometEventData);
begin
case Event.EventType of
cetFileProgress:
BBSStatusBar(Event.FileName, Event.Position,
Event.FileSize, Event.CPS);
cetSessionEnd:
BBSSessionComplete(Event.FilesSent, Event.FilesRecvd);
end;
end;
{ During initialization: }
CometLogSetCallback(@BBSLog);
CometLogSetBuiltin(False);
CometLogSetEventCallback(@BBSEvent);
{ Create a file provider for the protocol engines: }
FileIO := TCometLocalFileProvider.Create;
{ Then call CometHandshake/CometTransfer or BinkpRunOutbound,
passing FileIO — all output routes through your callbacks. }
{ Comet: }
CometXferInit(XS, State, InboundDir, TempDir, AbortLogPath, FileIO);
CometTransfer(XS, FileName, '');
{ BinkP: }
BResult := BinkpRunOutbound(Sock, Cfg, Addr, InboundDir, TempDir,
SendEntries, FileIO);
{ Clean up when done: }
FileIO.Free;
17.4. Authentication Constants
AUTH_NONE = 0 Not yet determined
AUTH_ED25519 = 1 ED25519 signature verified (encryption eligible)
AUTH_CRAM = 2 CRAM-MD5 password verified (no encryption)
AUTH_NOPWD = 3 Passwordless via COPT_NOPWD cap (no encryption)
AUTH_PLAIN = 4 Plain text password (BinkP only)
Encryption (X25519 + ChaCha20) is only enabled when AuthMethod
is AUTH_ED25519. All other methods run unencrypted.
17.5. File I/O Provider (cometio.pas)
As of version 1.2, both Comet and BinkP protocol engines access
files through a stream provider interface (TCometFileProvider).
The protocol code never touches the filesystem directly — it
reads and writes TStream objects returned by the provider.
The default provider (TCometLocalFileProvider) uses TFileStream
for local filesystem access. BBS software can subclass
TCometFileProvider to route file data anywhere — a database,
a WebSocket connection, in-memory buffers, etc.
Abstract class: TCometFileProvider
function OpenForSend(const Path: string): TStream;
Open a file for sending (TX). Returns a readable TStream
positioned at byte 0. Returns nil on failure.
Caller owns and frees the returned stream.
function CreateForReceive(const TempDir: string;
out TempPath: string): TStream;
Create a new file for receiving (RX). Returns a writable
TStream. TempPath receives the temporary file path for
later finalization. Returns nil on failure.
Caller owns and frees the returned stream.
function OpenForResume(const Path: string;
Offset: Int64): TStream;
Open an existing partial file for resume. Returns a
read/write TStream positioned at Offset. Returns nil if
the file doesn't exist or Offset exceeds file size.
Caller owns and frees the returned stream.
function FinalizeReceive(const TempPath, InboundDir,
OrigName: string; FileTime: LongInt): string;
Move a completed temp file to its final location.
Sets the file timestamp. Returns the final path on
success, '' on failure.
procedure CleanupReceive(const TempPath: string);
Delete a failed/incomplete temp file.
function GetFileSize(const Path: string): Int64;
Returns file size in bytes, or -1 on failure.
function GetFileTime(const Path: string): LongInt;
Returns file modification time (Unix timestamp).
function HashFile(const Path: string;
out Digest: TSHA256Digest): Boolean;
Compute SHA-256 of a file. Used before sending to
populate the FINFO packet hash field.
function HashStream(Stream: TStream; Len: Int64;
out Digest: TSHA256Digest): Boolean;
Compute SHA-256 of the first Len bytes of a stream.
Used at EOF to verify received file integrity.
Reads from position 0 and restores original position.
function StreamSize(Stream: TStream): Int64;
Returns the size of a stream. Default implementation
returns Stream.Size.
17.6. Custom File Provider Example
To implement a custom provider (e.g., for a BBS that stores
files in a database), subclass TCometFileProvider:
type
TMyBBSFileProvider = class(TCometFileProvider)
public
function OpenForSend(const Path: string): TStream;
override;
function CreateForReceive(const TempDir: string;
out TempPath: string): TStream; override;
function OpenForResume(const Path: string;
Offset: Int64): TStream; override;
function FinalizeReceive(const TempPath, InboundDir,
OrigName: string; FileTime: LongInt): string;
override;
procedure CleanupReceive(const TempPath: string);
override;
function GetFileSize(const Path: string): Int64;
override;
function GetFileTime(const Path: string): LongInt;
override;
function HashFile(const Path: string;
out Digest: TSHA256Digest): Boolean; override;
function HashStream(Stream: TStream; Len: Int64;
out Digest: TSHA256Digest): Boolean; override;
end;
function TMyBBSFileProvider.OpenForSend(
const Path: string): TStream;
begin
{ Return any TStream descendant — TFileStream, TMemoryStream,
a custom TDatabaseBlobStream, a TWebSocketStream, etc.
The protocol engine just calls Read() on it. }
Result := TFileStream.Create(Path,
fmOpenRead or fmShareDenyNone);
end;
{ ... implement remaining methods ... }
Then pass your provider to the protocol engines:
FileIO := TMyBBSFileProvider.Create;
try
CometXferInit(XS, State, InDir, TmpDir, AbortLog, FileIO);
{ ... transfer files ... }
finally
FileIO.Free;
end;
The same provider instance works for both Comet and BinkP.
Both protocols use the identical interface.
======================================================================
17.7. Triggering Outbound Calls (CallNodeExternal)
Host applications can trigger outbound calls programmatically
using the CallNodeExternal method on TCometDaemon.
Simple form (uses BSO outbound scan):
Daemon.CallNodeExternal(Addr);
Daemon.CallNodeExternal(Addr, 'bbs.example.com', 24554);
Full form (host-provided file list + stream provider):
var
Entries: array of TBinkpSendEntry;
MyIO: TMyFileProvider;
begin
SetLength(Entries, 3);
Entries[0].FilePath := 'netmail_001.pkt';
Entries[0].Action := csaDelete;
Entries[1].FilePath := 'echomail_042.pkt';
Entries[1].Action := csaDelete;
Entries[2].FilePath := 'nodediff.z42';
Entries[2].Action := csaNone;
MyIO := TMyFileProvider.Create;
Daemon.CallNodeExternal(Addr, Host, Port, Entries, MyIO);
{ Session runs in a thread. MyIO must stay valid until
the session completes. Do NOT free MyIO here — wait
for cetSessionEnd event. }
end;
Parameters:
Addr FidoNet address to call (TCometAddress)
Host Hostname/IP (empty = look up from config/nodelist)
Port Port (0 = use config default)
SendEntries Pre-built file list (bypasses BSO scan)
FileIO Custom stream provider (host retains ownership)
The SendEntries FilePath values are passed to
FileIO.OpenForSend(). They can be real paths, database keys,
URLs, or any identifier your FileIO implementation understands.
Protocol selection (Comet vs BinkP) is automatic based on the
node's config (NoComet setting) and remote detection.
Thread-safe. Each call spawns a new session thread.
TBinkpSendEntry record:
FilePath string Passed to FileIO.OpenForSend()
FloPath string .FLO file for cleanup ('' if N/A)
FloLine string Original .FLO line for marking
Action TCometSendAction csaNone, csaDelete, csaTruncate
18. CHANGE LOG
======================================================================
Version 1.2.1 (2026-04-21)
---------------------------
BinkP embedder symmetry with Comet native:
- TBinkpPostAuthCallback fires at the bsTransfer entry point —
the equivalent of binkd's complete_login / select_inbound
(protocol.c:1544-1573). Host applications can route secure
vs unsecure inbound BEFORE any M_FILE arrives, matching the
pre-transfer decision point Comet native already offers via
its phased CometHandshake / CometXferInit API.
- TCometBinkpResult gains Authenticated and AuthMethod fields
for consumers that prefer to route post-session.
- Daemon itself uses the new callback: InDir starts as
Cfg.Inbound (unsecure) and promotes to Cfg.SecInbound only
after the peer authenticates — fixes a pre-1.2.1 bug where
SecInbound was selected unconditionally when configured,
causing unauthenticated sessions to land in the secure dir.
Version 1.2-2 (2026-04-21)
---------------------------
BinkP full Argus/binkd parity. The BinkP stack now implements
every major extension either reference mailer supports:
- PLZ compression (Argus zlib, bit-14 frame flag) with adaptive
block sizing that grows from 4KB toward the 14-bit PLZ max
(16383 B) as the observed compression ratio warrants.
- NR mode (FSP-1029) resume is fully honored: crashed transfers
leave a `<target>.bkp-part` partial in the temp directory,
and the next session's M_FILE "-1" probe sends M_GET with the
partial size so the peer resumes from there.
- ND / NDA (FSP-1038) deferred cleanup: when the peer advertises
ND we stash M_GOT receipts in a pending list and only apply
the destructive cleanup after the session closes cleanly. A
mid-session abort preserves the outbound for the next attempt.
- MBT multi-batch (second-EOB handshake): a FREQ response
served after the first EOB now rides the same session instead
of being queued for the next poll.
- M_NUL TRF <netmail> <files> advisory (FRL-1026) and M_NUL
FREQ advisory when a .req file is queued.
- M_NUL NDL / PHN info strings driven by new [System] config
fields (NodelistFlags, Phone).
- RFC 2822 date format for M_NUL TIME (replaces the previous
locale-dependent format).
- Strict M_GET validation: name/size/time must match the active
TX file before we honor the resume offset.
- Duplicate-file pre-check: incoming M_FILE that names a file
already in the inbound with matching size + modtime is
acknowledged immediately with M_GOT, skipping the receive.
- EXTCMD kept in the OPT advertisement (binkd's prerequisite
for GZ/BZ2 compression; Argus ignores it).
Comet native protocol augmentations. The goal: keep Comet
"truly augmented" relative to BinkP — if a feature exists in
BinkP, Comet should match or beat it.
- INIT payload extended (backward-compatible trailing strings)
with Location, Time, Phone, and NodelistFlags. Older peers
stop reading at Mailer and leave the new fields empty.
- CometRFCDateStr shared between BinkP TIME and Comet INIT.Time
so the two protocols' time strings cannot drift.
- LST: new packet types NPKT_LSTREQ / NPKT_LSTITEM / NPKT_LSTEND
provide a structured file-listing query. Server enumerates
the FREQ directory and aliases for each request. New
capability flag COPT_LST.
- Transactional file cleanup (ND-equivalent). Destructive
cleanup for successfully transferred files is deferred to
session close; XFER_ABORT during the transfer loop preserves
the outbound for the next attempt.
Shared infrastructure:
- TCometFileProvider gains GetPartialSize and OpenForReceiveNamed
for NR-mode partial tracking. Providers that don't implement
them fall back to the random-name scheme automatically — no
plugin breakage.
- Config: new Phone and NodelistFlags fields in [System].
- WebUI lands: /src/web/ HTML+JS+OpenAPI + /src/webui/ backend.
Modeled after the Argus GUI: live session activity,
outbound polls, FREQ requests, nodelist browser, config
editor, scheduler. SSE event stream keeps the dashboard
updated in real time; OpenAPI spec exposes the same
endpoints for host applications.
Tested against:
- Argus (PLZ, MBT, ND, CRYPT)
- binkd (NR, CRAM-MD5)
Version 1.2 (2026-04-09)
-------------------------
BinkP state machine aligned with Argus/Radius (p_binkp.pas):
- CRAM-MD5 challenge sent in separate M_NUL (FTS-1027 compliance)
- Answerer defers M_ADR until originator's M_ADR received
- CRAM challenge parsed once as raw bytes (not reparsed from string)
- Strip trailing NUL bytes from BinkP command frames (Argus compat)
- State guard on OPT parsing failure
BinkP multi-file transfer fix:
- Added SentFiles tracking list (matches binkd's sent_fls).
After sending all data for a file, close immediately and start
next file — don't wait for M_GOT. Fixes frame ordering bug
where M_FILE for the next file was sent before the last data
frame of the previous file.
- M_GOT/M_SKIP now match by filename against SentFiles list
- EOB only sent when send queue empty AND all SentFiles confirmed
- M_FILE receive handler: finalize complete files, skip only
if actually incomplete (was always skipping)
Daemon improvements:
- Unified cetSessionEnd events: identical for BinkP and Comet,
fired from daemon. Catch-all for failed handshakes prevents
WFC node leaks in host applications.
- Replace Sleep(100) with select() on listener socket — inbound
connections accepted immediately instead of up to 100ms delay.
Host application API:
- New CallNodeExternal method on TCometDaemon for triggering
outbound calls from host applications. Accepts optional
pre-built send queue and custom TCometFileProvider.
- FFileIO changed to TCometFileProvider base class (was hardcoded
to TCometLocalFileProvider). Host-provided FileIO not freed
by session thread (host retains ownership).
New commands:
- showkey: print public key derived from configured private key
- keygen: updated output format (PublicKey shown as config comment)
Tested against:
- Argus/Radius 4.010 (CRAM-MD5 + CRYPT, 48 files inbound)
- BinkIT/sbbs 2.42 (CRAM-MD5)
- 5-node concurrent bidirectional: 45 files SHA256 verified
Version 1.01 (2026-04-02)
-------------------------
- Fixed bidirectional file transfers hanging or timing out,
particularly over internet links and on FreeBSD systems.
- Fixed file transfer resume reliability.
- Improved BinkP inbound bidirectional support — the daemon now
sends queued files back to the caller during inbound sessions.
- Various stability improvements for long-running daemon sessions.
Version 1.00 (2026-03-30)
-------------------------
- Initial release.