Back to Homepage
Services

Backend Development, Security Engineering

Industry

B2B Marketplace

Year

2024

Encryption at rest, without changing a single download

The legal team of a B2B marketplace asked one question: if someone reads the disk, what do they get? The honest answer was everything. Thousands of NDAs, financial statements and company valuations sat in Django's MEDIA_ROOT as plain files, readable by anyone with a leaked SSH key or a copy of a backup. Under GDPR, that answer was a liability.

THE CHALLENGE

Plain files in MEDIA_ROOT

Uploads were regular files in the media directory, named after what they were. Read access to the file system was all an attacker needed, and there were three plausible ways to get it: a compromised dependency, a leaked SSH key, or a backup that landed somewhere it should not. Any of them exposed every client's documents at once. There was no access log, so a breach could not even be sized afterwards, and for the regulatory audit there was nothing to show for data-at-rest protection. Legal filed it as a critical GDPR liability. The product side was just as firm: nothing about uploading or downloading a document was allowed to change.

The same directory, before and after

What someone with read access sees. Drag the divider: the listing as it was on the left, the same files after the encryption layer on the right, names included.

Plaintext
Encrypted
PlaintextEncrypted
THE SOLUTION

Fernet, decrypted per request

The library offered two routes. Raw AES-GCM is faster and authenticates in one primitive, but it hands you the nonce, and one reused nonce under the same key breaks the whole scheme. Fernet is the same library's high-level recipe: AES-128-CBC plus HMAC-SHA256, a random IV per token, a version byte and a timestamp, in an API that is hard to misuse. I chose Fernet and paid for it with two passes over every file and no streaming: a document is decrypted whole, in memory, for the length of one HTTP response, and the file on disk is never anything but ciphertext. The key is derived from a master secret with PBKDF2-HMAC over 100,000 iterations and lives only in an environment variable, never in the database, so it can be rotated without a deploy.

Key derivation and encryption. The master secret comes from the environment; the derived key is what Fernet uses:

Python
import base64
from cryptography.fernet import Fernet
from cryptography.hazmat.primitives.kdf.pbkdf2 import PBKDF2HMAC
from cryptography.hazmat.primitives import hashes

def get_fernet_key(master_secret: str) -> bytes:
    """Derive a Fernet key from the master secret via PBKDF2."""
    kdf = PBKDF2HMAC(
        algorithm=hashes.SHA256(),
        length=32,
        salt=b"stable-salt-from-settings",
        iterations=100_000,
    )
    return base64.urlsafe_b64encode(
        kdf.derive(master_secret.encode())
    )

def encrypt_file(file_bytes: bytes, key: bytes) -> bytes:
    """Encrypt raw file bytes with Fernet."""
    return Fernet(key).encrypt(file_bytes)

Inside a Fernet token

Type a short text and encrypt it. The token is built in your browser with WebCrypto, the same construction the server uses: version, timestamp, IV, ciphertext, HMAC. Then decrypt and watch the order: the HMAC is checked first, so a flipped byte is rejected before AES runs. Rotate the key and the old token still opens, because MultiFernet tries every key it holds.

Runs in your browser with WebCrypto. Nothing leaves this page.

Version 0x80
1 bytes
Timestamp
8 bytes · 2024-10-27T14:10:00Z
IV
16 bytes
Ciphertext
32 bytes
HMAC-SHA256
32 bytes

Token bytes by segment

Token as stored on disk (base64url)

Decrypt path

  1. Verify HMAC over version, timestamp, IV and ciphertext · waiting

Key ring (MultiFernet)

  • Key #10f5e2476current

New tokens use the first key. Decryption tries each key in order until one HMAC matches.

Token size
89 B
Plaintext
29 B
PKCS7 padding
3 B
Overhead
60 B
Encrypted with
Key #1
Result
Ready

The serving view: path check, key lookup with role-specific errors, in-memory decrypt, MIME sniffed from the bytes:

Python
class FetchCompanyDocumentsView(LoginRequiredMixin, View):

    def get(self, request, path):
        full_path = os.path.normpath(
            os.path.join(settings.MEDIA_ROOT, path)
        )

        # Path traversal protection
        if not full_path.startswith(settings.MEDIA_ROOT):
            raise SuspiciousFileOperation("Invalid path.")

        try:
            key = get_fernet_key(settings.ENCRYPTION_SECRET)
        except AttributeError:
            if request.user.is_staff:
                return HttpResponse("Encryption key not configured.", status=503)
            return HttpResponse("Please try again later.", status=503)

        encrypted = open(full_path, "rb").read()
        decrypted = Fernet(key).decrypt(encrypted)

        # Sniff MIME from decrypted content, not extension
        mime = magic.from_buffer(decrypted, mime=True)
        return HttpResponse(decrypted, content_type=mime)
THE RESULT

What the audit was shown

Nothing changed for the people using the platform: the same links, the same file names, the same speed. What changed is what the disk holds, and what the team could put in front of the auditor. The question of what happens when someone reads the disk now has a short answer, ciphertext, and a longer one that fits on a page, because the whole mechanism sits in one Django view and its settings.

What the audit could be shown

  • Algorithm: Fernet, AES-128-CBC with HMAC-SHA256 authentication, a random IV per file
  • Key derivation: PBKDF2-HMAC-SHA256, 100,000 iterations, from a master secret held only in the environment
  • Key rotation: a management command re-encrypts file by file, without downtime and without a deploy
  • Access path: one login-protected view, path normalised and confined to MEDIA_ROOT, decryption in memory only
  • Access logging: every document request recorded
  • Track record: more than 10,000 documents served, no decryption failures, under 15 ms added per download
CLIENT FEEDBACK

When legal asked for proof of data-at-rest protection, we sent them the view and the key parameters the same afternoon. Nobody in the product noticed the change; downloads look and behave exactly as before.

CTO

B2B marketplace, enterprise platform

FOR YOUR PROJECT

  • When it applies

    You store documents a stranger must not read, on a server or in backups other people can reach, and an auditor may ask how they are protected. Encrypting in the application covers the disk, the backup and the stray copy in one move.

  • What to check

    Where uploads live today and who can read that path: deploy users, backup jobs, CI. Whether your downloads rely on file extensions for the content type; after encryption they cannot. And where a key could live that is neither the database nor the repository.

  • What it needs

    Changes in the upload and download path, an environment variable or secret manager for the master key, a rotation command, and a one-time pass over the files already on disk. Write the parameters down where legal can find them; that is the audit answer.

FAQ

TECHNOLOGY STACK

Django
Python
PostgreSQL

We have enjoyed working with Daniel for 10 years now. We highly appreciate his fast response times around the clock and his all-round knowledge. Whether server configurations or programming, he always has the right solution.

Manuel Kasbarian - CEO, SophistiX

Join Manuel and launch your next project with confidence.

Get In TouchGet In Touch