nostr-rs-relay/docs/database-maintenance.md
Paul Rollo 2331c881d7 docs: typo in database-maintenance.md
Add a missing `"`
2023-01-20 10:49:03 -06:00

4.0 KiB

Database Maintenance

nostr-rs-relay uses the SQLite embedded database to minimize dependencies and overall footprint of running a relay. If traffic is light, the relay should just run with very little need for intervention. For heavily trafficked relays, there are a number of steps that the operator may need to take to maintain performance and limit disk usage.

This maintenance guide is current as of version 0.7.14. Future versions may incorporate and automate some of these steps.

Backing Up the Database

To prevent data loss, the database should be backed up regularly. The recommended method is to use the sqlite3 command to perform an "Online Backup". This can be done while the relay is running, queries can still run and events will be persisted during the backup.

The following commands will perform a backup of the database to a dated file, and then compress to minimize size:

BACKUP_FILE=/var/backups/nostr/`date +%Y%m%d_%H%M`.db
sqlite3 -readonly /apps/nostr-relay/nostr.db ".backup $BACKUP_FILE"
sqlite3 $BACKUP_FILE "vacuum;"
bzip2 -9 $BACKUP_FILE

Nostr events are very compressible. Expect a compression ratio on the order of 4:1, resulting in a 75% space saving.

Vacuuming the Database

As the database is updated, it can become fragmented. Performing a full vacuum will rebuild the entire database file, and can reduce space. Running this may reduce the size of the database file, especially if a large amount of data was updated or deleted.

vacuum;

Clearing Hidden Events

When events are deleted, either through deletion events, metadata or follower updates, or a replaceable event kind, the event is not actually removed from the database. Instead, a flag HIDDEN is set to true for the event, which excludes it from search results. The original intent was to ensure that subsequent rebroadcasts of the event would be easily detected as having been deleted, and would not need to be stored again. In practice, this decision causes excessive growth of the tags table, since all the previous followers are retained for those HIDDEN events.

The event and especially the tag table can be significantly reduced in size by running these commands:

PRAGMA foreign_keys = ON;
delete from event where HIDDEN=true;

Manually Removing Events

For a variety of reasons, an operator may wish to remove some events from the database. The only way of achieving this today is with manually run SQL commands.

It is recommended to have a good backup prior to manually running SQL commands!

In all cases, it is mandatory to enable foreign keys, and this must be done for every connection. Otherwise, you will likely orphan rows in the tag table.

Deleting Specific Event

PRAGMA foreign_keys = ON;
delete from event where event_hash=x'00000000000c1271675dc86e3e1dd1336827bccabb90dc4c9d3b4465efefe00e';

Deleting All Events for Pubkey

PRAGMA foreign_keys = ON;
delete from event where author=x'000000000002c7831d9c5a99f183afc2813a6f69a16edda7f6fc0ed8110566e6';

Deleting All Events of a Kind

PRAGMA foreign_keys = ON;
delete from event where kind=70202;

Deleting Old Events

In this scenario, we wish to delete any event that has been stored by our relay for more than 1 month. Crucially, this is based on when the event was stored, not when the event says it was created. If an event has a created field of 2 years ago, but was first sent to our relay yesterday, it would not be deleted in this scenario. Keep in mind, we do not track anything for re-broadcast events that we already have, so this is not a very effective way of implementing a "least recently seen" policy.

PRAGMA foreign_keys = ON;
TODO!

Delete Profile Events with No Recent Events

Many users create profiles, post a "hello world" event, and then never appear again (likely using an ephemeral keypair that was lost in the browser cache). We can find these accounts and remove them after some time.

PRAGMA foreign_keys = ON;
TODO!