Backend Development, Security Engineering
B2B Marketplace
2024
A B2B marketplace handled thousands of sensitive documents daily: NDAs, financial statements, company valuations, and proprietary contracts. All stored on disk in plaintext. A server breach would have exposed confidential data for every client on the platform. They needed encryption at rest that was transparent to users, manageable by admins, and verifiable under GDPR audit.
Every NDA, financial statement, and proprietary contract was stored as a regular file on disk. One server breach would have exposed everything.
The platform stored uploaded documents as regular files in Django's MEDIA_ROOT. Any attacker who gained read access to the file system - through a compromised dependency, an SSH key leak, or a misconfigured backup - would have immediate access to every NDA and financial document on the platform. The client's legal team flagged this as a critical GDPR liability: personal and commercial data stored unencrypted, with no access logging, and no way to prove data-at-rest protection in a regulatory audit.
The same directory on the server - before and after the encryption layer was deployed.
I implemented a symmetric encryption layer using Fernet, which wraps AES-128-CBC with HMAC-SHA256 authentication. Every file is encrypted before being written to disk, using a key derived from a master secret via PBKDF2-HMAC with 100,000 iterations. The key never touches the database - it lives exclusively in an environment variable, rotatable without re-deploying code. When a user requests a document, the view decrypts it in-memory only for the duration of the HTTP response. The file on disk remains encrypted at all times. After decryption, python-magic sniffs the actual MIME type from the binary content, not the file extension, because encrypted files have no meaningful extension. Path traversal protection ensures that only files within MEDIA_ROOT can be served, using os.path.normpath with a prefix check.
Documents are encrypted with Fernet before writing to disk. The key is derived via PBKDF2:
import base64
from cryptography.fernet import Fernet
from cryptography.hazmat.primitives.kdf.pbkdf2 import PBKDF2HMAC
from cryptography.hazmat.primitives import hashes
def get_fernet_key(master_secret: str) -> bytes:
"""Derive a Fernet key from the master secret via PBKDF2."""
kdf = PBKDF2HMAC(
algorithm=hashes.SHA256(),
length=32,
salt=b"stable-salt-from-settings",
iterations=100_000,
)
return base64.urlsafe_b64encode(
kdf.derive(master_secret.encode())
)
def encrypt_file(file_bytes: bytes, key: bytes) -> bytes:
"""Encrypt raw file bytes with Fernet."""
return Fernet(key).encrypt(file_bytes)Hover over each node to trace how a document moves from upload to secure delivery.
User uploads a confidential document through the Django application.
A cryptographic key is derived from the master secret using PBKDF2-HMAC with 100,000 iterations.
The document bytes are encrypted with Fernet (AES-128-CBC + HMAC-SHA256) and written to disk.
An authorized user requests the document. The view verifies permissions and decrypts in-memory only.
python-magic detects the actual MIME type from decrypted bytes, not the file extension, for correct delivery.
The view decrypts in-memory, sniffs the real MIME type, and serves with path traversal protection:
class FetchCompanyDocumentsView(LoginRequiredMixin, View):
def get(self, request, path):
full_path = os.path.normpath(
os.path.join(settings.MEDIA_ROOT, path)
)
# Path traversal protection
if not full_path.startswith(settings.MEDIA_ROOT):
raise SuspiciousFileOperation("Invalid path.")
try:
key = get_fernet_key(settings.ENCRYPTION_SECRET)
except AttributeError:
if request.user.is_staff:
return HttpResponse("Encryption key not configured.", status=503)
return HttpResponse("Please try again later.", status=503)
encrypted = open(full_path, "rb").read()
decrypted = Fernet(key).decrypt(encrypted)
# Sniff MIME from decrypted content, not extension
mime = magic.from_buffer(decrypted, mime=True)
return HttpResponse(decrypted, content_type=mime)The encryption layer was invisible to end users - documents downloaded exactly as before, with the same speed and the same file names. Behind the scenes, every file was now encrypted at rest with authenticated encryption, removing the plaintext exposure risk. When the GDPR audit came, the team could demonstrate key derivation parameters, encryption algorithm, key rotation capability, and access logging - all from a single Django view. The system processed over 10,000 documents without a single decryption failure. Admin staff received specific error messages if the encryption key was missing from the environment, while regular users saw a friendly retry message.
"We sleep better knowing every document is encrypted. When our legal team asked for proof of data-at-rest protection, we had it ready in five minutes, not five weeks."
CTO
B2B Marketplace · Enterprise Platform