Server tech specs

Software

We're running a fork based on latest Mastodon nightly from source, main branch. On top of the Mastodon features, we have bunch of our own. See Instance features.

Current optimizations

• Puma is on its own server
• PostgreSQL is on its own server
• Database is on its own scalable SSD volume (storage for the next 100 years)
• ElasticSearch is on its own server
• Media uses fast Cloudflare R2 Zero Egress Distributed Object Storage (practically endless storage)
• Assets delivered by brotli
• Nginx optimized for resources
• RAM optimized for services
• Sidekiq splitted to services and optimized for plenty of enough RAM
• More than enough resources for every server and service

Server infrastructure

This is where the magic happens. All Mementomori.social servers are running on UpCloud cloud servers, located in Helsinki, Finland. UpCloud sponsors our infrastructure. Every server is built with Ansible, and the playbooks are public.

UpCloud - Global Cloud PlatformHigh-performance cloud infrastructure, built for developers and businesses. Our servers run in UpCloud's Helsinki data centre.upcloud.com

Puma server

The server specifications:

• 8 vcpus
• 32 GB RAM
• 20 GB system disk
• 20 GB data volume

On top of the regular Mastodon dependencies, the server software has

• Latest nginx
• Brotli
• Valkey and all the other cool Mastodon dependencies

PostgreSQL server

The server specifications:

• 6 vcpus
• 24 GB RAM
• 300 GB SSD volume for the PostgreSQL database, scalable further when needed
• 260 GB volume for database backups

ElasticSearch server

The server specifications:

• 4 vcpus
• 16 GB RAM
• 40 GB system disk
• 35 GB data volume

S3 Object storage for media

Media storage is provided by Cloudflare R2, Zero Egress Distributed Object Storage.

Tweaks

There are some extensive improvements to the default installation.

Sidekiq services and jobs

Sidekiq background jobs have been separated into their own systemd services. We decided not to have them on a different server for now, we have not found the benefit in it yet.

mastodon@mastodon-0:~$ ls /etc/systemd/system/ | grep mastodon-sidekiq
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]

Files (you get the idea):

[Unit]
Description=mastodon-sidekiq-%j-queue
After=network.target

[Service]
Type=simple
User=mastodon
WorkingDirectory=/data/mastodon/live
Environment="RAILS_ENV=production"
Environment="DB_POOL=%i"
Environment="MALLOC_ARENA_MAX=2"
Environment="LD_PRELOAD=libjemalloc.so"
ExecStart=/data/mastodon/.rbenv/shims/bundle exec sidekiq -c %i -q %j
TimeoutSec=15
Restart=always
# Proc filesystem
ProcSubset=pid
ProtectProc=invisible
# Capabilities
CapabilityBoundingSet=
# Security
NoNewPrivileges=true
# Sandboxing
ProtectSystem=strict
PrivateTmp=true
PrivateDevices=true
PrivateUsers=true
ProtectHostname=true
ProtectKernelLogs=true
ProtectKernelModules=true
ProtectKernelTunables=true
ProtectControlGroups=true
RestrictAddressFamilies=AF_INET
RestrictAddressFamilies=AF_INET6
RestrictAddressFamilies=AF_NETLINK
RestrictAddressFamilies=AF_UNIX
RestrictNamespaces=true
LockPersonality=true
RestrictRealtime=true
RestrictSUIDSGID=true
RemoveIPC=true
PrivateMounts=true
ProtectClock=true
# System Call Filtering
SystemCallArchitectures=native
SystemCallFilter=~@cpu-emulation @debug @keyring @ipc @mount @obsolete @privileged @setuid
SystemCallFilter=@chown
SystemCallFilter=pipe
SystemCallFilter=pipe2
ReadWritePaths=/data/mastodon/live

[Install]
WantedBy=multi-user.target

Scheduled jobs

Health checks, preview card refetching, FediFetcher, the weekly search index rebuild and the nightly database dump run as systemd timers. See the Server runbook for the full list.

Server status and monitoring

The admin gets a phone call from Better Uptime automation if anything mentioned on the status page goes down, by the minute.

Services status can be followed at status.mementomori.social. Status page, monitors and heart beats are provided by Better Stack.

Backups

• Nightly database dump to a separate backup volume, see the Server runbook
• Manual backups each time anything is performed