Bridges help users connect to external content (RSS, ActivityPub) but
are not real Org Social accounts. They no longer appear in /feeds/,
cannot be registered via POST, are skipped by both feed discovery
tasks, and do not count in /stats/. A data migration removes bridge
URLs already registered as feeds.
Expose external accounts as virtual Org Social feeds that any client
can follow with a plain #+FOLLOW: line:
- /bridge/activitypub/@{user}@{instance}/ bridges a Mastodon or any
ActivityPub account (WebFinger, actor and paginated outbox; only
public top-level notes, with CW, hashtags, language and attachments)
- /bridge/rss/?url={feed} bridges any RSS/Atom feed (entry title as
*** sub-heading, converted body and link to the original article)
Registration is implicit on first GET. Bridged data is stored in the
existing Profile/Post tables so /profile/, /search/ and the rest of
the API work on bridged feeds. Active bridges are refreshed every 15
minutes; bridges unrequested for 90 days are cleaned up.
Remote HTML is converted to Org text, escaping headline-like lines so
external content cannot inject posts. Fetches enforce the Webmention
SSRF protections, a 10s timeout and a 5MB size cap. Bridged posts
never queue Webmentions nor publish notifications.
Implements the sender side of https://www.w3.org/TR/webmention/. During
feed scans, external URLs found in new posts (and links added by edits)
are queued as OutgoingWebmention rows; a periodic task discovers each
target's endpoint and delivers the notification. The unique
(source, target) pair guarantees a webmention is sent at most once, no
matter how many times a feed is rescanned. Targets without an endpoint
are marked permanently, failures retry with exponential backoff, and
endpoints resolving to loopback/private addresses are rejected.
Receiving webmentions is out of scope: plain text feeds cannot
advertise an endpoint.
manage.py check booted a full Django interpreter every 30s (~1.5s CPU
per run) and never verified the server was actually responding. Probe
the HTTP endpoint directly instead and relax the interval.
- Drop malformed #+BIRTHDAY values (not YYYY-MM-DD) instead of letting
them reach the Profile DateField and abort the whole feed scan
- Use a (connect, read) timeout of (3.05, 5) when fetching feeds so dead
hosts are dropped faster without penalizing slow-but-alive servers
- Add parser and scan_feeds regression tests
Sync generators blocked uvicorn's event loop preventing chunks from
being flushed. Converts to async generators with redis.asyncio so
streaming works correctly under ASGI. Adds /sse/ nginx location with
proxy_buffering off and 1h read timeout.
Without ?feed=, the endpoint subscribes to all notification channels via
Redis psubscribe("notifications:*") and adds target_feed to each event.
With ?feed=, behavior is unchanged. Updates README and root _links.
New GET /profile/?feed={url} endpoint that returns the list of feed URLs
that follow the given profile, using the existing Follow model.
Includes cache support, 400/404 error handling, and 8 unit tests.
The previous startup probe ran a 1s TCP connect to Redis at module import
time and silently fell back to DummyCache when it failed. In docker compose
the huey container could finish importing settings.py just before Redis
finished warming up, locking the daemon to DummyCache for its entire lifetime
and breaking cache.delete/cache.clear inside scan_feeds.
Drop the probe; in DEBUG use DummyCache, otherwise always RedisCache.
Changed node labels from 'Node X (relay-list.txt)' to simply 'Relay X'
for cleaner visualization. Also updated Concepts section to use 'Relay'
terminology consistently throughout the documentation.
The relay list is now decentralized - each node maintains its own
local relay-list.txt file. Nodes share feeds directly with each other
in a P2P fashion, without needing a centralized list endpoint.
Changes:
- Remove 'List nodes' from mermaid diagram
- Add relay-list.txt notation to each node
- Update 'Share Users' to 'Share Feeds' for accuracy
- Simplify Concepts section to reflect decentralized architecture
The relay-list.txt file is now part of this repository, so the
instructions for making a relay public have been updated to reflect
that users should make PRs to this repo instead of the external
org-social repository.
When PRs are merged, all relay nodes that update their code will
automatically get the updated relay list.
Added test coverage for:
- Basic 301 redirect detection and parsing
- Feed validation with redirects
- Feed URL updates when only old URL exists
- Feed merging when both old and new URLs exist
- Mention migration without UNIQUE constraint errors
The mention migration test specifically validates the fix for the
production bug where feeds with 301 redirects were failing due to
duplicate (post, mentioned_profile) combinations.
All 23 parser tests pass successfully.
The feed redirect merge logic was attempting a bulk update of mentions,
which failed when duplicate (post, mentioned_profile) combinations existed.
This caused feeds with 301 redirects to fail merging properly.
Now mentions are migrated individually, checking for existing duplicates
before updating, similar to how Follow and PollVote relationships are handled.
This fixes the UNIQUE constraint errors seen in production for feeds like:
- thesolarprincess.github.io -> thesolarprincess.site
- haiverin.scot -> www.haiverin.scot
- teoten.com -> www.teoten.com
New features from Org Social v1.6:
- Add LOCATION, BIRTHDAY, LANGUAGE, PINNED fields to Profile model
- Support post ID in header (** 2025-05-01T12:00:00+0100)
- Header ID takes priority over property drawer ID
- Parse and store all new v1.6 metadata fields
Changes:
- Updated Profile model with 4 new fields
- Updated parser to extract new metadata fields
- Updated parser to support ID in post headers
- Updated tasks.py to save new profile fields
- Added database migration 0010
- Added 3 new tests for v1.6 features
- Renamed SKILL.md to CLAUDE.MD
All tests passing (58/58)
The SSE endpoint was using pubsub.listen() which blocks indefinitely
waiting for messages. This prevented heartbeats from being sent when
there was no activity, causing connections to timeout and close.
Changed to use pubsub.get_message(timeout=1) in a while loop, which
allows heartbeats to be sent every 30 seconds even when there are no
new notifications, keeping connections alive.
This fixes the issue where SSE connections would break and users
wouldn't receive real-time notifications.
When merging duplicate feeds with redirects, Follow relationships
can generate UNIQUE constraint errors if the relationship already
exists between the same profiles.
This fix:
- Iterates Follow relationships individually
- Checks for existing relationships before updating
- Deletes duplicates safely
- Adds comprehensive error handling
Similar approach to PollVote fix.
Resolves issue with feeds like haiverin.scot redirect.
Signed-off-by: Andros Fenollosa <hi@andros.dev>
