BizzFed ActivityPub Tester

Automated ActivityPub protocol compatibility testing platform for BizzFed.

Detects Fediverse instance software, runs protocol compliance tests, and tracks regressions over time.

Features

• Software Detection - Identifies instance software (Mastodon, Pleroma, Akkoma, Misskey, GoToSocial, Pixelfed, etc.) via NodeInfo and API probing
• Protocol Testing - Tests NodeInfo, WebFinger (incl. actor consistency + publicKey checks), ActivityPub Core, Software APIs, Auth, and Server-to-Server endpoints
• Regression Tracking - Automatically compares test runs and detects changes
• Diff Severity - Each detected change is classified (info/low/medium/high/critical) by configurable rules; display and notification thresholds can be set independently
• Notifications - Optional ntfy/webhook alerts when changes are detected
• Cron Automation - Ready for unattended monitoring with logging
• Rich Terminal Output - Colored tables and diff display

Installation

git clone https://github.com/bizzfed/bizzfed-activitypub-tester.git
cd bizzfed-activitypub-tester
pip install -e ".[dev]"

Quick Start

# Run tests against an instance
aptester run https://example.com

# Use config file
aptester run -c config/default.yaml https://example.com

# Interactive menu
aptester interactive

CLI Commands

Command	Description
aptester run <url>	Run all tests against target instance
aptester history	Show past test runs
aptester compare <run1> <run2>	Compare two test runs
aptester report	Show detailed report of last run
aptester interactive	Start interactive menu (Textual TUI)

Run aptester --help for an overview of all commands and their flags, or aptester <command> --help for detailed options of a single command.

Common run options:

Option	Description
-c, --config <path>	Config file path
-v, --verbose	Repeatable: -v response bodies, -vv bodies + headers
-q, --quiet	Suppress all stdout output (results are still saved)
-t, --tests <cats>	Only run selected categories, comma-separated: nodeinfo,webfinger,core,software_api,auth,s2s

aptester run exits with code 0 if all tests passed/skipped and 1 if any test failed or errored — usable in cron jobs and CI. Result files older than storage.keep_last per instance are pruned automatically.

Configuration

Default configuration is in config/default.yaml. CLI arguments override config file values.

target:
  url: ""  # Override via CLI

timeouts:
  connect: 10
  read: 30
  total: 60
  retries: 2     # Retry transient failures (timeout/connect errors)
  retry_delay: 0.5  # Delay between retry attempts (seconds)

tests:
  nodeinfo:
    enabled: true
  webfinger:
    enabled: true
  activitypub_core:
    enabled: true
  software_api:
    enabled: true
  auth:
    enabled: false  # Requires token
  s2s:
    enabled: false  # Requires signing

storage:
  results_dir: "results"
  keep_last: 100  # Oldest result files beyond this per instance are pruned

reporting:
  show_diff: true     # Show diffs against previous run after each run
  verbose: 0          # 0=normal, 1=response bodies, 2=headers+bodies
  # min_display_severity: "medium"  # only render diffs at/above this severity

severity:
  enabled: true
  default: "medium"              # fallback when no rule matches
  # min_notification_severity: "high"  # only notify at/above this level
  rules: []                      # see docs/04-severity-guide.md

notifications:
  enabled: false
  type: "ntfy"        # "ntfy" (topic URL) or "webhook" (generic JSON POST)
  url: ""             # e.g. https://ntfy.sh/your-topic or your webhook URL
  token: ""           # Optional bearer token
  on_change_only: true  # Only notify when changes vs previous run detected

Cron Automation

# Run every 6 hours
0 */6 * * * /path/to/scripts/cron_runner.sh --url https://example.com

# With custom config
0 */6 * * * /path/to/scripts/cron_runner.sh --config /path/to/config.yaml

Project Structure

bizzfed-activitypub-tester/
├── pyproject.toml           # Dependencies and CLI entry
├── requirements.lock        # Pinned runtime deps (hashes)
├── requirements-dev.lock    # Pinned runtime + dev deps (hashes)
├── config/default.yaml      # Default configuration
├── scripts/cron_runner.sh   # Cron wrapper script
├── src/aptester/
│   ├── cli.py              # CLI interface
│   ├── tui.py              # Textual TUI (interactive mode)
│   ├── runner.py           # Test orchestration
│   ├── config.py           # YAML + CLI config merging
│   ├── models.py           # Data models (TestRun, EndpointResult, etc.)
│   ├── detection.py        # Software detection
│   ├── severity.py         # Severity rule engine + diff filtering
│   ├── compare.py          # Regression detection
│   ├── notifier.py         # ntfy / webhook notifications
│   ├── storage.py          # JSON persistence
│   ├── reporter.py         # Terminal output (rich)
│   ├── login.py            # OAuth 2.0 authorization code flow
│   ├── version.py          # Instance version fetch
│   └── tests/              # Test modules
├── tests/                  # pytest unit tests
├── docs/                   # Documentation
│   ├── 01-implementation-plan.md
│   ├── 02-architecture.md
│   ├── 03-risk-analysis.md
│   └── 04-severity-guide.md
└── results/                # Test results (JSON)

Documentation

• Implementation Plan - Project phases and structure
• Architecture - arc42 architecture documentation
• Risk Analysis - Technical risks and mitigations
• Severity Rules Guide - Severity levels, rules, notifications, and display filtering
• Changelog - Version history between release tags

Development

# Install dev dependencies (exactly from lockfile, hashes verified)
pip-sync requirements-dev.lock

# Or install editable without lockfile pinning
pip install -e ".[dev]"

# Check for available dependency upgrades (regenerates the lockfiles)
pip-compile --upgrade pyproject.toml --generate-hashes --output-file requirements.lock
pip-compile --upgrade --extra dev pyproject.toml --generate-hashes --output-file requirements-dev.lock

# Security audit of resolved dependencies
pip-audit -r requirements-dev.lock

# Run linter
ruff check src/

# Type check
mypy src/

# Fix lint issues
ruff check --fix src/

# Format code
ruff format src/

# Run tests with coverage (fails below 60%)
pytest

# pre-commit hooks (ruff + mypy, run on every commit)
pre-commit install

License

See at the file LICENSE