Files
comet/COMET.DOC
Ken Johnson 8aeef4d323 Release 1.01: docs, all-platform binaries, and TIC files
Updated COMET.DOC, COMET.QA, COMET.SAM to version 1.01 with changelog.
Fixed DOS build: removed StrUtils dependency, guarded SO_SNDBUF for
Watt-32, added IFDEF UNIX around CometTcpRecv EINTR retry loop for
Windows. Release archives and TIC files for all 5 platforms.
2026-04-02 07:59:19 -07:00

911 lines
27 KiB
Plaintext

======================================================================
COMET MAILER - OPERATOR MANUAL
Version 1.01
Direct TCP File Transfer for FidoNet
Copyright (C) 2026 Ken Johnson, 1:218/720
======================================================================
TABLE OF CONTENTS
1. Introduction
2. System Requirements
3. Installation
4. Quick Start
5. Command Reference
6. Configuration Reference
7. Authentication
8. Nodelist Integration
9. File Requests (FREQ)
10. Post-Session Events
11. BinkP Compatibility
12. Cross-Platform Notes
13. Troubleshooting
14. Security Notes
15. Data Compression
16. Technical Reference
17. Embedding in Host Applications
18. Change Log
======================================================================
1. INTRODUCTION
======================================================================
Comet is a modern FidoNet TCP mailer daemon designed for the 2020s.
It provides:
- Direct TCP file transfer with SHA-256 verification
- Bidirectional transfer with sliding window flow control
- ED25519 public-key authentication (no shared secrets needed)
- CRAM-MD5 password authentication (passwords never sent in clear)
- Full BinkP/1.1 compatibility (FTS-1026) with auto-detection
- Inline file requests (Hydra-style, within active sessions)
- FTS-5 nodelist integration (automatic IP/port lookup)
- BSO outbound scanning with point directory support
- Multi-session daemon with per-node BSY locking
- Config hot-reload (edit config while Comet is running)
- Cross-platform: Linux, FreeBSD, DOS, Windows, OS/2
Comet uses TCP port 24554, the standard BinkP port. Both
Comet native and BinkP connections are accepted on the same port.
Both Comet and BinkP connections are accepted on the same port.
The protocol specification is in FSP-COMET.001.
======================================================================
2. SYSTEM REQUIREMENTS
======================================================================
Linux / FreeBSD:
- Any modern x86 or x86-64 system
- TCP/IP networking
- Recommended: /dev/urandom for ED25519 key generation
DOS (FreeDOS recommended):
- 386 or better with DPMI (CWSDPMI.EXE)
- Packet driver for your network card
- Watt-32 TCP/IP (WATTCP.CFG)
- Approximately 1 MB RAM for the executable
Windows:
- Windows 7 or later (64-bit)
- WinSock2 (included with Windows)
OS/2:
- OS/2 Warp or eComStation with TCP/IP stack
======================================================================
3. INSTALLATION
======================================================================
3.1. Linux / FreeBSD
1. Copy the 'comet' binary to /usr/local/bin/ or your preferred
location.
2. Copy COMET.SAM to /etc/comet/comet.cfg (or wherever you
prefer) and edit it for your system.
3. Create the required directories:
mkdir -p /var/lib/fidonet/{inbound,secure,outbound,temp}
4. Generate an ED25519 keypair:
comet keygen
5. Add the PrivateKey to your config file.
6. Start the daemon:
comet -c /etc/comet/comet.cfg
7. To run as a system service, create a systemd unit file or
add to /etc/rc.local.
3.2. DOS
1. Copy COMET.EXE and CWSDPMI.EXE to your Comet directory
(e.g., F:\COMET\).
2. Copy COMET.SAM to COMET.CFG and edit it.
3. Create directories: INBOUND, SECURE, OUTBOUND, TEMP
4. Configure WATTCP.CFG for your network:
my_ip = dhcp
or:
my_ip = 192.168.1.100
netmask = 255.255.255.0
gateway = 192.168.1.1
5. Set the environment variable:
SET WATTCP.CFG=F:\COMET
6. Load your packet driver:
NE2000 0x60 3 0x300
7. Run Comet:
COMET CALL 1:213/723
3.3. Signal Handling (Unix)
SIGTERM / SIGINT : Clean shutdown (waits for active sessions)
SIGHUP : Reload configuration
SIGPIPE : Ignored (handled internally)
======================================================================
4. QUICK START
======================================================================
4.1. Daemon Mode (listen + poll outbound)
comet
comet -c /path/to/comet.cfg
4.2. Single Call
comet call 1:213/723
comet -c /path/to/comet.cfg call 1:218/720
4.3. Generate ED25519 Keypair
comet keygen
This generates a random keypair and displays:
- PrivateKey: add to [System] section of your config
- PublicKey: give to remote nodes (they add to [Node:] section)
4.4. Show Version
comet -v
4.5. Show Help
comet -h
4.6. Debug Mode
comet -d
comet -d call 1:213/723
Enables verbose protocol-level logging.
======================================================================
5. COMMAND REFERENCE
======================================================================
comet [options] [command]
Commands:
(none) Run as daemon (listen + poll outbound)
call <address> Single outbound call to a FidoNet node
keygen Generate ED25519 keypair
Options:
-c <file> Use specified config file (default: comet.cfg)
-d Enable debug/trace logging
-q Quiet mode (errors only on console)
-v Show version and exit
-h, --help Show help and exit
Address format:
zone:net/node e.g., 1:213/723
zone:net/node.point e.g., 1:213/723.1
======================================================================
6. CONFIGURATION REFERENCE
======================================================================
Configuration is read from comet.cfg (or the file specified with
-c). The file uses INI-style sections with key = value pairs.
Lines beginning with ; are comments.
Comet monitors the config file for changes and automatically
reloads when modifications are detected (checked every 5 seconds).
On Unix, you can also send SIGHUP to force a reload.
6.1. [System] Section
Address = 1:218/720
Your FidoNet address. Multiple addresses supported (one per
line). First address is primary.
SysOp = Ken Johnson
Your name as system operator.
SystemName = The Danger Zone
Your system/BBS name.
Location = Sparks, NV
City, state/country.
PrivateKey = <64 hex characters>
Your ED25519 private key. Generate with: comet keygen
Keep this SECRET. Never share it.
6.2. [Network] Section
Port = 24554
TCP port to listen on. Default: 24554.
Bind =
Network interface to bind to. Empty = all interfaces.
MaxSessions = 5
Maximum simultaneous sessions. Range: 1-32.
PollInterval = 60
Outbound poll interval in seconds. 0 = no polling.
6.3. [Paths] Section
Inbound = /var/lib/fidonet/inbound/
Directory for files from unauthenticated nodes.
SecInbound = /var/lib/fidonet/secure/
Directory for files from password-verified nodes.
Outbound = /var/lib/fidonet/outbound/
BSO outbound base directory.
Temp = /var/lib/fidonet/temp/
Temporary files during transfer.
LogFile = /var/log/fidonet/comet.log
Main log file path.
DebugLog = /var/log/fidonet/comet-debug.log
Debug/trace log (very verbose).
Nodelist = /var/lib/fidonet/nodelist/
Directory containing nodelist files.
Freq = /var/lib/fidonet/freq/
Directory for file request searches.
Flags = /var/lib/fidonet/flags/
Semaphore/flag files directory.
6.4. [DriveMap] Section (Linux/FreeBSD only)
Maps DOS drive letters to Unix paths for reading .FLO files
written by DOS mailers.
C = /home/user/dos/c
D = /home/user/dos/d
6.5. [Protocol] Section
MaxBlockSize = 65528
Maximum data block size in bytes (512-65528).
WindowSize = 8
Sliding window depth (1-16).
Timeout = 120
Braindead timeout in seconds.
HandshakeTimeout = 30
Handshake timeout in seconds.
NoSHA256 = no
Disable SHA-256 file verification (NOT recommended).
NoFREQ = no
Disable file request support.
6.6. [BinkP] Section
Enabled = yes
Enable BinkP fallback for outbound connections.
Port = 24554
Default BinkP port for fallback connections.
6.7. [Logging] Section
FileLevel = info
Minimum log level for file: debug, info, warning, error, fatal
ConsoleLevel = info
Minimum log level for console.
Debug = no
Enable debug/trace logging.
6.8. [Events] Section
flag <flagfile> <pattern>
Create a semaphore file when files matching pattern are received.
exec "<command>" <pattern>
Run external command when files matching pattern are received.
Variables for exec commands:
*A = Remote FidoNet address
*I = Inbound directory path
*N = Number of files received
Examples:
flag /var/lib/fidonet/flags/toss.now *.pkt
exec "hpt toss" *.pkt
exec "allfix RP -SRIF *S" *.req
6.9. [Node:address] Sections
Per-node configuration. Section name includes the address:
[Node:1:213/723]
[Node:21:1/100]
Password = SECRET
Session password (CRAM-MD5 authenticated, never sent in clear).
PublicKey = <64 hex characters>
Remote node's ED25519 public key (from their sysop).
Host = bbs.example.com
IP address or hostname.
Port = 24554
Outbound port override. If not set, the port from the
nodelist (IBN flag) is used. Default: 24554.
NoBinkp = no
Disable BinkP fallback for this node.
NoComet = no
Use only BinkP for this node (skip Comet protocol).
======================================================================
7. AUTHENTICATION
======================================================================
Comet supports three authentication methods, tried in order of
strength:
7.1. ED25519 Digital Signatures (Strongest)
Public-key authentication. Each node generates a keypair:
- Private key: stays on your system, never shared
- Public key: given to remote nodes
Setup:
1. Run: comet keygen
2. Add PrivateKey to your [System] section
3. Give your PublicKey to remote sysops
4. They add it to their [Node:your_address] section
5. They give you their PublicKey
6. You add it to your [Node:their_address] section
No passwords needed. The private key never leaves your system
and cannot be derived from the public key.
7.2. CRAM-MD5 (Shared Secret)
Challenge-response authentication per RFC 2104. Both sides must
know the same password. The password is NEVER sent in plain text.
Setup:
1. Agree on a password with the remote sysop
2. Add Password = SECRETWORD to both [Node:] sections
7.3. No Authentication
If neither ED25519 keys nor passwords are configured, the session
proceeds without authentication. Files are placed in the unsecure
Inbound directory. This is logged as a warning.
7.4. Authentication Fallback
When connecting, Comet tries methods in order:
ED25519 -> CRAM-MD5 -> Plain password -> No auth
Only available methods are advertised to the remote. If ED25519
is not configured, it is not offered.
======================================================================
8. NODELIST INTEGRATION
======================================================================
Comet reads standard FidoNet raw text nodelists (FTS-5 format).
8.1. Setup
Set the Nodelist path in [Paths]:
Nodelist = /var/lib/fidonet/nodelist/
Place nodelist files (e.g., NODELIST.081) in this directory.
Comet automatically loads the most recent one.
8.2. How It Works
When polling outbound or making a call, if no [Node:] section
exists for the target address, Comet looks it up in the nodelist.
It extracts:
- INA: flag for IP address/hostname
- IBN: flag for BinkP port (default 24554)
[Node:] entries always take priority over nodelist data.
8.3. Nodelist Reload
The nodelist is reloaded when the config file changes (hot-reload).
======================================================================
9. FILE REQUESTS (FREQ)
======================================================================
9.1. BinkP FREQ
Standard .REQ file mechanism. The .REQ file is sent as a regular
file. When the remote receives it, they process the patterns and
send matching files back.
To make a request: create a .REQ file in the outbound directory
for the target node (e.g., 00D502D3.REQ for 1:213/723).
Format: one filename pattern per line.
9.2. Comet FREQ (Inline)
Comet protocol supports Hydra-style inline file requests via
NPKT_FREQ packets. Requests are processed immediately during the
session without requiring a separate .REQ file transfer.
9.3. FREQ Processing
When a file request is received, Comet searches the Freq directory
for matching files. Limits: 10 files, 10 MB per request.
For advanced FREQ processing (magic names, file areas), use
an external FREQ processor via [Events] exec rules with SRIF.
9.4. Setup
Set the Freq path in [Paths]:
Freq = /var/lib/fidonet/freq/
Place files you want to make available in this directory.
======================================================================
10. POST-SESSION EVENTS
======================================================================
After a session completes and files are received, Comet can:
10.1. Create Flag Files
[Events]
flag /var/lib/fidonet/flags/toss.now *.pkt
Creates the specified file when files matching the pattern are
received. Your tosser watches for this flag.
10.2. Run External Commands
[Events]
exec "hpt toss" *.pkt
exec "allfix RP -SRIF *S" *.req
Commands run AFTER the session is closed - they never block
active transfers.
======================================================================
11. BINKP COMPATIBILITY
======================================================================
Comet is fully compatible with BinkP/1.1 (FTS-1026).
11.1. Auto-Detection
Both Comet and BinkP connections are accepted on the same TCP
port. The protocol is auto-detected from the first bytes received.
11.2. Fallback
When calling a node, Comet connects on the port from the
nodelist (IBN flag) or the per-node Port setting. Both
protocols are auto-detected by the remote.
Set NoComet = yes for nodes that only speak BinkP.
11.3. Features in BinkP Mode
- CRAM-MD5 authentication (FTS-1027)
- ED25519 authentication (via OPT extension)
- M_GET file resume
- FREQ via .REQ file transfer
- OPT negotiation (NR, EXTCMD)
11.4. Tested Against
- binkd (reference BinkP implementation)
- Radius/Argus 4.010 (Windows GUI mailer)
- Fimail 0.1.0 (FidoNet mailer)
======================================================================
12. CROSS-PLATFORM NOTES
======================================================================
12.1. Linux (Primary Platform)
Full support. Production ready.
12.2. FreeBSD
Cross-compiled from Linux. Same features as Linux.
Build: make freebsd
12.3. DOS (FreeDOS)
Requires CWSDPMI.EXE and Watt-32 TCP/IP with packet driver.
Single-session mode (no threading).
Build: make dos
12.4. Windows
WinSock2 code written, needs type compatibility fixes.
Build: make win64 (work in progress)
12.5. OS/2
Should work via FPC's OS/2 Sockets unit. Untested.
12.6. Path Handling
Comet handles DOS/Unix path conversion automatically:
- Backslash <-> forward slash conversion
- Drive letter mapping via [DriveMap]
- Case-insensitive file lookup on Unix
======================================================================
13. TROUBLESHOOTING
======================================================================
13.1. "Cannot listen on port 24554"
Another process is using the port, or a previous Comet instance
hasn't fully shut down. Wait a few seconds and try again, or
check with: netstat -tlnp | grep 24554
13.2. "No host configured for X:XXX/XXX"
The node has no [Node:] section and isn't in the nodelist.
Add a [Node:] section with Host, or set the Nodelist path.
13.3. "Password mismatch"
CRAM-MD5 passwords don't match. Verify both sides have the
same password configured.
13.4. "ED25519 verification failed"
The public key configured for the remote node doesn't match
the private key they're using. Verify the public key was
copied correctly.
13.5. "NO PACKET DRIVER FOUND" (DOS)
Watt-32 can't find the packet driver. Make sure:
- Packet driver is loaded (NE2000 0x60 3 0x300)
- WATTCP.CFG exists with IP configuration
- SET WATTCP.CFG=<directory> is set
13.6. CPU usage is high
Check for stale .BSY files in the outbound directory.
Comet auto-cleans BSY files older than 10 minutes.
13.7. Config changes not taking effect
Comet checks the config file every 5 seconds. On Unix,
you can force a reload with: kill -HUP $(pidof comet)
======================================================================
14. SECURITY NOTES
======================================================================
14.1. Configuration File Permissions
The config file (comet.cfg) may contain passwords and private
keys. On Unix, set permissions to 0600: chmod 600 comet.cfg
Comet will warn at startup if the config file is world-readable.
14.2. Input Validation
Comet validates all data received from remote peers:
- Filenames are stripped of path components (/ and \) to
prevent directory traversal attacks. A malicious remote
cannot write files outside the inbound directory.
- File sizes, resume offsets, and reposition offsets are all
validated against actual file boundaries before any seek
operation.
- Received data is checked against the declared file size.
A remote cannot send more data than advertised in FINFO.
- FREQ (file request) patterns are sanitized to prevent
requests for files outside the designated FREQ directory.
- Protocol strings from INIT packets are capped to prevent
memory exhaustion from oversized payloads.
14.3. Authentication Rate Limiting
Comet tracks failed authentication attempts per IP address.
After 5 failures within 5 minutes, further connections from
that IP are temporarily rejected. This prevents brute-force
password guessing.
14.4. Password Handling
Passwords and key material are wiped from memory (zeroed)
after authentication completes. This limits the window for
memory-based credential theft.
14.5. Post-Session Command Safety
Variables substituted into Exec commands (*A, *I, *N) are
shell-quoted on Unix to prevent command injection.
======================================================================
15. DATA COMPRESSION
======================================================================
Comet supports optional per-block zlib compression. When both
sides advertise the COPT_ZLIB capability during INIT negotiation,
each DATA block is individually compressed before transmission.
15.1. How It Works
- Each block is compressed with zlib/deflate (level 6)
- If the compressed block is smaller than the original, it is
sent with compression type 1 (zlib)
- If compression doesn't help (already-compressed data), the
block is sent uncompressed with compression type 0
- The decision is made per-block, not per-file
15.2. What Compresses Well
- .PKT files (netmail/echomail packets): 50-70% reduction
- Text files, .TIC files: good compression
- .ZIP, .RAR, .7Z, .GZ attachments: sent uncompressed
(no overhead beyond the 1-byte type field)
15.3. Configuration
Compression is enabled by default. To disable:
[Protocol]
Compression = none
When the remote node does not support compression, transfers
proceed uncompressed with the original wire format. No
configuration is needed for interoperability.
15.4. Wire Format
Without compression: offset(4) + data(N)
With compression: offset(4) + comp_type(1) + data(N)
comp_type: 0 = uncompressed, 1 = zlib
======================================================================
16. TECHNICAL REFERENCE
======================================================================
14.1. BSO Outbound Structure
outbound/ Default zone
outbound.NNN/ Zone NNN (3-digit hex)
outbound/XXXXXXXX.PNT/ Point directory
File naming: NNNNNNNN.ext where NNNNNNNN = (net<<16)|node in hex
14.2. Flow File (.FLO) Format
/path/to/file Send normally
^/path/to/file Delete after send
#/path/to/file Truncate after send
~/path/to/file Already sent (skip)
;comment Ignored
14.3. Comet Frame Format
+--------+------+-----+------------------+-------+
| LEN | TYPE | SEQ | PAYLOAD | CRC32 |
| 4B LE | 1B | 1B | 0..65528 bytes | 4B LE |
+--------+------+-----+------------------+-------+
See FSP-COMET.001 for complete wire protocol specification.
14.4. Default Ports
Comet: 24554 (standard BinkP port)
BinkP: 24554 (standard)
14.5. Source Code
29 Pascal source files, approximately 16,000 lines.
Built with Free Pascal Compiler (FPC) 3.2.x.
Core units:
cometdef.pas Protocol definitions and types
comettcp.pas Cross-platform TCP socket abstraction
cometfrm.pas Length-prefixed frame layer
cometses.pas Session handshake (banner, INIT, INITACK)
cometxfer.pas Bidirectional file transfer engine
cometbinkp.pas BinkP/1.1 protocol implementation
cometdaemon.pas Multi-session daemon core
cometbso.pas BSO outbound directory handler
cometnodelist.pas FTS-5 nodelist parser
cometcfg.pas Configuration file parser
cometfile.pas File transfer helpers
cometpath.pas Cross-platform path operations
cometlog.pas Logging subsystem
Crypto units:
cometmd5.pas MD5 message digest (RFC 1321)
cometcram.pas CRAM-MD5 authentication (RFC 2104)
cometsha.pas SHA-256/384/512 (FIPS 180-4)
cometcrc.pas CRC-32 checksum
cometed25519.pas ED25519 digital signatures
cometed25519sc.pas Scalar arithmetic mod l
cometed25519ge.pas Group element operations
cometed25519bp.pas Precomputed base point table
DOS-specific:
cometwatt.pas Watt-32 TCP/IP bindings
cometlibc.pas DJGPP libc emulation
======================================================================
For bug reports and updates:
FidoNet: 1:218/720
Email: ken@rail-city.net
Comet is free software released under the GPL-2.0 license.
======================================================================
======================================================================
17. EMBEDDING IN HOST APPLICATIONS (BBS SOFTWARE)
======================================================================
Comet is designed to be embedded in BBS and mailer software as a
library. Two callback mechanisms allow the host application to
intercept all output and receive structured session events.
17.1. Log Callback
By default, Comet writes to log files and stdout. To route log
output to your application instead:
CometLogSetCallback(@MyLogHandler);
CometLogSetBuiltin(False); { Suppress file/console output }
Callback signature:
procedure MyLogHandler(Level: TCometLogLevel; const Msg: string);
Levels: cllDebug, cllInfo, cllWarning, cllError, cllFatal.
All LogInfo/LogError/LogWarning calls route through this callback.
17.2. Event Callback
For structured session events (file progress, session status):
CometLogSetEventCallback(@MyEventHandler);
Callback signature:
procedure MyEventHandler(const Event: TCometEventData);
Event types (TCometEventType):
cetSessionStart Remote identified. Fields: RemoteName,
RemoteAddr, RemoteMailer, Protocol.
cetSessionAuth Authentication result. Fields: AuthMethod
(AUTH_ED25519, AUTH_CRAM, AUTH_NOPWD),
Encrypted (Boolean).
cetFileStart File transfer starting. Fields: FileName,
FileSize, Sending (True=TX, False=RX),
Protocol.
cetFileProgress Transfer progress update (max 4/sec).
Fields: FileName, FileSize, Position,
CPS, Sending.
cetFileEnd File transfer complete. Fields: FileName,
FileSize, Position, CPS, Sending.
cetFileSkip File skipped by remote. Fields: FileName.
cetSessionEnd Session complete. Fields: FilesSent,
FilesRecvd, BytesSent, BytesRecvd, Success.
The Protocol field is 'Comet' or 'BinkP' — same events for both.
17.3. Minimal Integration Example
uses cometdef, cometlog, cometses, cometxfer, cometbinkp;
procedure BBSLog(Level: TCometLogLevel; const Msg: string);
begin
BBSActivityLog(Msg); { Your BBS logging }
end;
procedure BBSEvent(const Event: TCometEventData);
begin
case Event.EventType of
cetFileProgress:
BBSStatusBar(Event.FileName, Event.Position,
Event.FileSize, Event.CPS);
cetSessionEnd:
BBSSessionComplete(Event.FilesSent, Event.FilesRecvd);
end;
end;
{ During initialization: }
CometLogSetCallback(@BBSLog);
CometLogSetBuiltin(False);
CometLogSetEventCallback(@BBSEvent);
{ Then call CometHandshake/CometTransfer or BinkpRunOutbound
as normal — all output routes through your callbacks. }
17.4. Authentication Constants
AUTH_NONE = 0 Not yet determined
AUTH_ED25519 = 1 ED25519 signature verified (encryption eligible)
AUTH_CRAM = 2 CRAM-MD5 password verified (no encryption)
AUTH_NOPWD = 3 Passwordless via COPT_NOPWD cap (no encryption)
AUTH_PLAIN = 4 Plain text password (BinkP only)
Encryption (X25519 + ChaCha20) is only enabled when AuthMethod
is AUTH_ED25519. All other methods run unencrypted.
======================================================================
18. CHANGE LOG
======================================================================
Version 1.01 (2026-04-02)
-------------------------
- Fixed bidirectional file transfers hanging or timing out,
particularly over internet links and on FreeBSD systems.
- Fixed file transfer resume reliability.
- Improved BinkP inbound bidirectional support — the daemon now
sends queued files back to the caller during inbound sessions.
- Various stability improvements for long-running daemon sessions.
Version 1.00 (2026-03-30)
-------------------------
- Initial release.