Deep dive: How we use quantum safe encrypted SQLite mailboxes for our privacy-focused and secure email service
Unlike other email services, we ensure that only you have access to your mailbox at all times.
tldr; Our email service is 100% open-source and privacy-focused through secure and encrypted SQLite mailboxes.
Until we launched IMAP support, we used MongoDB for our persistent data storage needs.
This technology is amazing and we still use it today – but in order to have encryption-at-rest with MongoDB you need to use a provider that offers MongoDB Enterprise, such as Digital Ocean or Mongo Atlas – or pay for an enterprise license (and subsequently have to work with sales team latency).
Our team at Forward Email needed a developer-friendly, scalable, reliable, and encrypted storage solution for IMAP mailboxes. As open-source developers, using a technology you need to pay a license fee in order to get the encryption-at-rest feature was against our principles – and so we experimented, researched, and developed a new solution from scratch to solve these needs.
Instead of using a shared database to store your mailboxes, we individually store and encrypt your mailboxes with your password (which only you have). Our email service is so secure that if you forget your password, then you lose your mailbox (and need to recover with offline backups or start over).
We are the only 100% open-source and privacy-focused email service provider that stores individually encrypted SQLite mailboxes, offers unlimited domains, aliases, and users, and has outbound SMTP, IMAP, and POP3 support:
Unlike other email providers, you do not need to pay for storage on a per domain or alias basis with Forward Email. Storage is shared across your entire account – so if you have multiple custom domain names and multiple aliases on each, then we are the perfect solution for you. Note that you can still enforce storage limits if desired on a per domain or alias basis.
|Provider||Open Source||Quantum Safe Encrypted SQLite Mailboxes||Unlimited Custom Domains and Aliases||Outbound SMTP||IMAP||POP3||Basic IMAP Pricing||Basic IMAP Storage|
|Forward Email ⭐||✅||✅||✅||✅||✅||✅||$3/mo||10 GB pooled (pay as you go)|
|Apple||❌||❌||❌||✅||✅||❌ iCloud does not support POP||$0.99/mo||5 GB free or 50 GB basic pooled|
|Fastmail||❌||❌||❌||✅||✅||✅||$3/mo||2 GB per mailbox|
|Gandi||❌||❌||❌||✅||✅||✅||$3.99/mo||10 GB per mailbox|
|❌||❌||❌||✅||✅||✅||$7.20/mo||30 GB pooled|
|Microsoft 365||❌||❌||❌||✅||✅||❌||$6/mo billed annually ($72/year)||1 TB pooled|
|Proton Mail||❌ Front-end only||❌||❌||❌ Proton Mail Bridge is required||❌ Proton Mail does not support POP3||❌ Proton Mail Bridge is required||$12.99/mo||500 GB pooled|
|Tutanota||❌ Front-end only||❌||❌||✅||❌||❌||❌||❌|
The comparison table above was last updated in December 2023 and may contain errors or inaccurate data. We are not affiliated, associated, authorized, endorsed by, or in any way officially connected with any of the Providers, or any of their subsidiaries or their affiliates.
Using your email client such as Apple Mail, Thunderbird, Gmail, or Outlook – you connect to our secure IMAP servers using your username and password:
- Your username is your full alias with your domain such as
- Your password is randomly generated and only displayed to you for 30 seconds when you click Generate Password from My Account Domains Aliases.
- Your username is your full alias with your domain such as
Once connected, your email client will send IMAP protocol commands to our IMAP server to keep your mailbox in sync. This includes writing and storing draft emails and other actions you might do (e.g. label an email as Important or flag an email as Spam/Junk Mail).
Mail exchange servers (commonly known as "MX" servers) receive new inbound email and store it to your mailbox. When this happens your email client will get notified and sync your mailbox. Our mail exchange servers can forward your email to one or more recipients (including webhooks), store your email for you in your encrypted IMAP storage with us, or both!
Behind the scenes, our secure email storage design works in two ways to keep your mailboxes encrypted and only accessible by you:
When new mail is received for you from a sender, our mail exchange servers write to an individual, temporary, and encrypted mailbox for you.sequenceDiagram autonumber actor Sender Sender->>MX: Inbound message received for your alias (e.g. firstname.lastname@example.org). MX->>SQLite: Message is stored in a temporary mailbox. Note over MX,SQLite: Forwards to other recipients and webhooks configured. MX->>Sender: Success!
When you connect to our IMAP server with your email client, your password is then encrypted in-memory and used to read and write to your mailbox. Your mailbox can only be read from and written to with this password. Keep in mind that since you are the only one with this password, only you can read and write to your mailbox when you are accessing it. The next time your email client attempts to poll for mail or syncs, your new messages will be transferred from this temporary mailbox and stored in your actual mailbox file using your supplied password. Note that this temporary mailbox is purged and deleted afterwards so that only your password protected mailbox has the messages.
If you are connected to IMAP (e.g. using an email client such as Apple Mail or Thunderbird), then we do not need to write to temporary disk storage. Your in-memory encrypted IMAP password is instead fetched and used. In real-time, when a message is attempting to be delivered to you, we send a WebSocket request to all IMAP servers asking them if they have an active session for you (this is the fetch part), and then subsequently will pass along that encrypted in-memory password – so we don't need to write to a temporary mailbox, we can write to your actual encrypted mailbox using your encrypted password.sequenceDiagram autonumber actor You You->>IMAP: You connect to IMAP server using an email client. IMAP->>SQLite: Transfer message from temporary mailbox to your alias' mailbox. Note over IMAP,SQLite: Your alias' mailbox is only available in-memory using IMAP password. SQLite->>IMAP: Retrieves messages as requested by email client. IMAP->>You: Success!
Compressed backups of your encrypted mailboxes are made hourly. You can also request a new backup at any time or download the latest backup from My Account Domains Aliases. If you decide to switch to another email service, then you can easily migrate, download, export, and purge your mailboxes and backups at anytime.
We explored other possible database storage layers, however none satisfied our requirements as much as SQLite did:
|Database||Encryption-at-rest||Sandboxed Mailboxes||License||Used Everywhere|
|SQLite ⭐||✅ Yes with SQLite3MultipleCiphers||✅||✅ Public Domain||✅|
|MongoDB||❌ "Available in MongoDB Enterprise only"||❌ Relational database||❌ AGPL and
|rqlite||❌ Network only||❌ Relational database||✅
|dqlite||❌ Untested and not yet supported?||❌ Untested and not yet supported?||✅
|PostgreSQL||✅ Yes||❌ Relational database||✅
|MariaDB||✅ For InnoDB only||❌ Relational database||✅
|CockroachDB||❌ Enterprise-only feature||❌ Relational database||❌
Here is a blog post that compares several SQLite database storage options in the table above.
At all times we use encryption-at-rest (AES-256), encryption-in-transit (TLS), DNS over HTTPS ("DoH") using 🍊 Tangerine, and sqleet (ChaCha20-Poly1305) encryption on mailboxes. Additionally we use token-based two-factor authentication (as opposed to SMS which is suspectible to man-in-the-middle-attacks), rotated SSH keys with root access disabled, exclusive access to servers through restricted IP addresses, and more.
In the event of an evil maid attack or rogue employee from a third-party vendor, your mailbox can still only be opened with your generated password. Rest assured, we don't rely upon any third-party vendors other than our SOC Type 2 complaint server providers of Cloudflare, Digital Ocean, and Vultr.
Our goal is to have as few single point of failures as possible.
tldr; Our IMAP servers use individually encrypted SQLite databases for each of your mailboxes.
For example, on our encrypted servers there's a SQLite database mailbox for
email@example.com and so on – one for each as a
.sqlite database file. We don't name the database files with the email address either – instead we use BSON ObjectID and unique UUID's generated which do not share who the mailbox belongs to or which email address it is under (e.g.
Each of these databases are encrypted themselves using your password (which only you have) using sqleet (ChaCha20-Poly1305). This means that your mailboxes are individually encrypted, self-contained, sandboxed, and portable.
We have fine-tuned SQLite with the following PRAGMA:
||ChaCha20-Poly1305 SQLite database encryption. Reference
||This is your decrypted in-memory only password that gets passed through your email client's IMAP connection to our server. New database instances are created and closed for each read and write session (in order to ensure sandboxing and isolation).|
||Write-ahead-log ("WAL") which boosts performance and allows concurrent read access.|
||Prevents write-lock errors while other writes are taking place.|
||Increases durability of transactions without data corruption risk.|
||Enforces that foreign key references (e.g. a relation from one table to another) are enforced. By default this is not turned on in SQLite, but for validation and data integrity it should be enabled.|
||Default encoding to use to ensure developer sanity.|
All other defaults are from SQLite as specified from the official PRAGMA documentation.
tldr; We use
WebSocketfor concurrent reads and writes to your encrypted SQLite mailboxes.
Your email client on your phone may resolve
imap.forwardemail.net to one of our Digital Ocean IP addresses – and your desktop client may resolve a separate IP from a different provider altogether.
Regardless of which IMAP server your email client connects to, we want the connection to read from your database in real-time with 100% accuracy:
This is accomplished by using
--vfs-cache-mode off(the default).
Instead of using local disk cache, the cache is read directly from the remote mount (your database) in real-time.
In the event that the local file cannot be found, this indicates that
rclonefailed to mount or has an issue. In this case we use a
WebSocketfallback for reads (which slightly decreases performance, but still maintains the integrity of the service).
Each of our servers is configured to mount with consistency and alerts us in real-time of any errors.
Writing to your database is a bit different – since SQLite is an embedded database and your mailbox lives in a single file by default.
We had explored options such as
dqlite below – however none of these satisfied our requirements.
To accomplish writes with write-ahead-logging ("WAL") enabled – we need to ensure that only one server ("Primary") is responsible for doing so. WAL drastically speeds up concurrency and allows one writer and multiple readers.
The Primary is running on the data servers with the mounted volumes containing the encrypted mailboxes. From a distribution standpoint, you could consider all the individual IMAP servers behind
imap.forwardemail.net to be secondary servers ("Secondary").
We accomplish two-way communication with WebSockets:
- Primary servers use an instance of ws's
- Secondary servers use an instance of ws's
WebSocketclient that is wrapped with websocket-as-promised and reconnecting-websocket. These two wrappers ensure that the
WebSocketreconnects and can send and receive data for specific database writes.
tldr; Compressed backups of your encrypted mailboxes are made every hour. You can also instantly request a new backup or download the latest backup at anytime from My Account Domains Aliases.
For backups, we simply run the SQLite
VACUUM INTO command every hour during IMAP command processing, which leverages your encrypted password from an in-memory IMAP connection. Backups are stored if no existing backup is detected or if the SHA-256 hash has changed on the file as compared to the most recent backup.
Note that we use the
VACUUM INTO command as opposed to the built-in
backup command because if a page is modified during a
backup command operation, then it has to start over. The
VACUUM INTO command will take a snapshot. See these comments on GitHub and Hacker News for more insight.
Additionally we use
VACUUM INTO as opposed to
backup, because the
backup command would leave the database unencrypted for a brief period until
rekey is invoked (see this GitHub comment for insight).
The Secondary will instruct the Primary over the
WebSocket connection to execute the backup – and the Primary will then receive the command to do so and will subsequently:
- Connect to your encrypted mailbox.
- Run the
- Compress the resulting backup file with
- Upload it to Cloudflare R2 for storage (or your own provider if specified).
Remember that your mailboxes are encrypted – and while we have IP restrictions and other authentication measures in place for WebSocket communication – in the event of a bad actor, you can rest assured that unless the WebSocket payload has your IMAP password, it cannot open your database.
Only one backup is stored per mailbox at this time, but in the future we may offer point-in-time-recovery ("PITR").
Our IMAP servers support the
SEARCH command with complex queries, regular expressions, and more.
Indices are also stored for all properties that are in search queries.
Here's a table outlining projects we use in our source code and development process (sorted alphabetically):
|Ansible||DevOps automation platform for maintaing, scaling, and managing our entire fleet of servers with ease.|
|Lad||Node.js framework which powers our entire architecture and engineering design with MVC and more.|
|MongoDB||NoSQL database solution that we use for storing all other data outside of mailboxes (e.g. your account, settings, domains, and alias configurations).|
|Mongoose||MongoDB object document modeling ("ODM") which we use across our entire stack. We wrote special helpers that allow us to simply continue using Mongoose with SQLite 🎉|
|Nodemailer||Node.js package for sending emails, creating connections, and more. We are an official sponsor of this project.|
|Redis||In-memory database for caching, publish/subscribe channels, and DNS over HTTPS requests.|
|SQLite3MultipleCiphers||Encryption extension for SQLite to allow entire database files to be encrypted (including the write-ahead-log ("WAL"), journal, rollback, …).|
|SQLiteStudio||Visual SQLite editor (which you could also use) to test, download, and view development mailboxes.|
|SQLite||Embedded database layer for scalable, self-contained, fast, and resilient IMAP storage.|
|Spam Scanner||Node.js anti-spam, email filtering, and phishing prevention tool (our alternative to Spam Assassin and rspamd).|
|Tangerine||DNS over HTTPS requests with Node.js and caching using Redis – which ensures global consistency and much more.|
|Thunderbird||Our development team uses this (and recommends this too) as the preferred email client to use with Forward Email.|
|UTM||Our development team uses this create virtual machines for iOS and macOS in order to test different email clients (in parallel) with our IMAP and SMTP servers.|
|Ubuntu||Modern open-source Linux-based server operating system which powers all of our infrastructure.|
|WildDuck||IMAP server library – see its notes on attachment de-duplication and IMAP protocol support.|
|better-sqlite3-multiple-ciphers||Fast and simple API library for Node.js to interact with SQLite3 programmatically.|
|email-templates||Developer-friendly email framework to create, preview, and send custom emails (e.g. account notifications and more).|
|json-sql||SQL query builder using Mongo-style syntax. This saves our development team time since we can continue to write in Mongo-style across the entire stack with a database agnostic approach. It also helps to avoid SQL injection attacks by using query parameters.|
|knex-schema-inspector||SQL utility to extract information about existing database schema. This allows us to easily validate that all indices, tables, columns, constraints, and more are valid and are
|knex||SQL query builder which we use only for database migrations and schema validation through
|mandarin||Automatic i18n phrase translation with support for Markdown using Google Cloud Translation API.|
|mx-connect||Node.js package to resolve and establish connections with MX servers and handle errors.|
|pm2||Node.js production process manager with built-in load balancer (fine-tuned for performance).|
|smtp-server||SMTP server library – we use this for our mail exchange ("MX") and outbound SMTP servers.|
|ImapTest||Useful tool for testing IMAP servers against benchmarks and RFC specification IMAP protocol compatibility. This project was created by the Dovecot team (an active open-source IMAP and POP3 server from July 2002). We extensively tested our IMAP server with this tool.|
You can find other projects we use in our source code on GitHub.
|Cloudflare||DNS provider, health checks, load balancers, and backup storage using Cloudflare R2.|
|Digital Ocean||Dedicated server hosting, SSD block storage, and managed databases.|
|Vultr||Dedicated server hosting and SSD block storage.|
Forward Email is designed according to these principles:
- Always be developer-friendly, security and privacy-focused, and transparent.
- Adhere to MVC, Unix, KISS, DRY, YAGNI, Twelve Factor, Occam's razor, and dogfooding
- Target the scrappy, bootstrapped, and ramen-profitable developer
tldr; Ultimately using S3-compatible object storage and/or Virtual Tables are not technically feasible for performance reasons and prone to error due to memory limitations.
We have done a few experiments leading up to our final SQLite solution as discussed above.
One of these was to try using rclone and SQLite together with an S3-compatible storage layer.
That experiment led us to further understand and discover edge cases surrounding rclone, SQLite, and VFS usage:
- If you enable
--vfs-cache-mode writesflag with rclone, then reads will be OK, however writes will get cached.
- If you have multiple IMAP servers distributed globally, then the cache will be off across them unless you have a single writer and multiple listeners (e.g. a pub/sub approach).
- This is incredibly complex and adding any additional complexity like this will result in more single points of failure.
- S3-compatible storage providers do not support partial file changes – which means any change of the
.sqlitefile will result in a complete change and re-upload of the database.
- Other solutions like
rsyncexist, but they are not focused on write-ahead-log ("WAL") support – so we ended up reviewing Litestream. Fortunately our encryption usage already encrypts the WAL files for us, so we do not need to rely on Litestream for that. However we weren't yet confident in Litestream for production-use and have a few notes below on that.
- Using this option of
--vfs-cache-mode writes(the only way to use SQLite over
rclonefor writes) will attempt to copy the entire database from scratch in-memory – handling one 10 GB mailbox is OK, however handling multiple mailboxes with exceedingly high storage will cause the IMAP servers to run into memory limitations and
ENOMEMerrors, segmentation faults, and data corruption.
- If you attempt to use SQLite Virtual Tables (e.g. using s3db) in order to have data live on an S3-compatible storage layer, then you will run into several more issues:
- Read and writes will be extremely slow as S3 API endpoints will need to be hit with HTTP
- Development tests showed that exceeding 500K-1M+ records on fiber internet is still limited by the throughput of writing and reading to S3-compatible providers. For example, our developers ran
forloops to do both sequential SQL
INSERTstatements and ones that bulk wrote large amounts of data. In both cases the performance was staggeringly slow.
- Virtual tables cannot have indexes,
ALTER TABLEstatements, and other limitations – which leads to delays upwards of 1-2 minutes or more depending on the amount of data.
- Objects were stored unencrypted and no native encryption support is readily available.
- Read and writes will be extremely slow as S3 API endpoints will need to be hit with HTTP
- We also explored using sqlite-s3vfs which is similar conceptually and technically to the previous bullet point (so it has the same issues). A possibility would be to use a custom
sqlite3build wrapped with encryption such as wxSQLite3 (which we currently use in our solution above) through editing the setup file.
- Another potential approach was to use the multiplex extension, however this has a limitation of 32 GB and would require complex building and development headaches.
ALTER TABLEstatements are required (so this completely rules out using Virtual Tables). We need
ALTER TABLEstatements in order for our hook with
knex-schema-inspectorto work properly – which ensures that data is not corrupted and rows retrieved can be converted to valid documents according to our
mongooseschema definitions (which includes constraint, variable type, and arbitrary data validation).
- Compression libraries such as sqlite-zstd (see comments) look promising, but may not yet be ready for production usage. Instead application-side compression on data types such as
Bufferis going to be a cleaner and easier approach (and is easier to migrate too, since we could store a
Booleanflag or column – or even use
user_version=1for compression or
user_version=0for no compression as database metadata).
- Fortunately we already have attachment de-duplication implemented in our IMAP server storage – therefore every message with the same attachment won't keep a copy of the attachment – instead a single attachment is stored for multiple messages and threads in a mailbox (and a foreign reference is subsequently used).
- The project Litestream, which is a SQLite replication and backup solution is very promising and we will most likely use it in the future.
- Backup restoration needs to be frictionless and trivial. Using a solution such as MongoDB with
mongoexportis not only tedious, but time intensive and has configuration complexity.
- SQLite databases make it simple (it's a single file).
- We wanted to design a solution where users could take their mailbox and leave at any moment.
- Simple Node.js commands to
fs.unlink('mailbox.sqlite'))and it's permanently erased from disk storage.
- We can similarly use an S3-compatible API with HTTP
DELETEto easily remove snapshots and backups for users.
- Simple Node.js commands to
- SQLite was the simplest, fastest, and most cost-effective solution.
To our knowledge, no other email services are designed this way nor are they open-source.
We think this might be due to existing email services having legacy technology in production with spaghetti code 🍝.
Most if not all of existing email service providers are either closed-source or advertise as open-source, but in reality only their front-end is open-source.
The most sensitive part of email (the actual storage/IMAP/SMTP interaction) is all done on the back-end (server), and not on the front-end (client).
Sign up today at https://forwardemail.net! 🚀