I want to tell you why #Ditto was slow and how we fixed it.
There were 2 issues: Postgres performance and nip05 lookups.
Postgres performance was caused by profile lookups, eg:
{ kinds: [0], authors: [a, b, c, d, ...] }
The issue was ORDER BY clauses on those queries. This caused Postgres to do a "sequence scan" instead of using the correct index:
"nostr_events_replaceable_idx" UNIQUE, btree (kind, pubkey) WHERE kind >= 10000 AND kind < 20000 OR (kind = ANY (ARRAY[0, 3]))
Logging query errors exposed this. This is why logging matters, people! Now I am switching Ditto to use JSON logging and incorporating Loki + Promtail so can easily track these problems in the future.
As for nip05 lookups - Ditto actually does nip05 verification in realtime as feeds are being rendered. This is obviously very bad. It does have an in-memory LRU cache, but restarting Ditto resets it. As a result, feeds can take up to 3 seconds to load (that's the timeout I set for fetching nip05). And if fetching fails, usernames are not displayed correctly.
To solve this (with a bandaid solution), I configured squid (HTTP proxy) to aggressively cache nip05 responses with this:
refresh_pattern /.well-known/nostr.json 3600 0% 259200
I also had to configure Ditto and squid with self-signed SSL certs so it can mitm and actually cache:
My TODO is to cache nip05 verifications in Postgres, to solve the root problem for self-hosters and increase performance even more.
tl;dr Ditto is fast now.