​

SPEC 023 — File Storage

---

​

1. Feature Purpose

File Storage is the foundational module that all file-handling features in Equa depend on. It manages upload, storage, retrieval, and deduplication of files using AWS S3 as the backing store and a content-addressed hashing scheme for deduplication. Every document uploaded to Data Rooms (SPEC 008), agreement PDFs (SPEC 005), generated reports (SPEC 014), and Google Drive synced files (SPEC 016) ultimately flows through this module. A secondary Microsoft file storage integration handles files managed through the Microsoft 365 connector (SPEC 017).

---

​

2. Current State (Verified)

​

2.1 Storage Backend

​

2.2 Upload Flow

​

2.3 Microsoft File Storage

​

2.4 Environment Variables

---

​

3. Data Model

​

Files

​

HashedFiles

​

Relationships

• Files → HashedFiles: Many-to-one. Multiple Files records (different filenames, owners, or contexts) can reference the same HashedFiles row when content is identical.
• Files → Users: Many-to-one via owner. Tracks who uploaded the file.
• DirectoryItems → Files: Many-to-one via DirectoryItems.file (SPEC 008). Data Room entries reference Files.

---

​

4. API Endpoints

​

Upload Processing Pipeline

1. Client sends multipart/form-data via postMultipart() (Axios)
2. Server receives file stream, computes SHA-256 hash
3. Check HashedFiles for existing hash
4. If hash exists → skip S3 upload (dedup), create new Files record pointing to existing hash
5. If hash is new → upload to S3, create HashedFiles row, create Files record
6. Return Files record with id, url, and metadata

---

​

5. Frontend Components

​

Frontend Behavior

• Multipart upload — postMultipart(url, formData, onProgress) sends files as multipart/form-data with Content-Type header set automatically by Axios.
• Progress tracking — onUploadProgress receives Axios progress events with loaded and total bytes for percentage calculation.
• Size validation — Frontend validates file size against AWS_S3_UPLOAD_SIZE_LIMIT_MB before initiating upload; over-limit files show an error toast.
• Multiple files — Batch uploads send files sequentially (not parallel) to avoid overwhelming the server; each file gets its own progress bar.
• Retry — No automatic retry on upload failure; users see an error and can retry manually.

---

​

6. Business Rules

1. Content-addressed deduplication — Files with identical content (same SHA-256 hash) are stored once in S3; additional uploads create new Files metadata records pointing to the same HashedFiles entry.
2. Upload size limit — Files exceeding AWS_S3_UPLOAD_SIZE_LIMIT_MB (default 10 MB) are rejected at the server with a 413 response. The frontend also validates before upload.
3. Hash as foreign key — Files.hash references HashedFiles.hash; the HashedFiles row must exist before the Files row is created.
4. Owner tracking — Every file upload records the authenticated user as Files.owner for attribution and access control.
5. S3 key structure — S3 object keys include the content hash to enable direct dedup lookup; the exact key format is {prefix}/{hash}/{filename}.
6. Soft delete — Deleting a Files record does not immediately remove the S3 object; orphaned HashedFiles entries (no remaining Files references) are cleaned up by a background job.
7. MIME type detection — Content type is determined from the file extension and validated against the file’s magic bytes where possible.
8. Static file URL — STATIC_FILE_URL is prepended to file paths for client-facing URLs; this allows CDN or proxy configuration without changing stored paths.
9. Microsoft file passthrough — Files originating from Microsoft 365 (via SPEC 017) flow through the same upload pipeline; Microsoft-specific metadata is preserved in the Files record.
10. No direct S3 access — Clients never interact with S3 directly; all uploads and downloads are proxied through the API server for access control.

---

​

7. Acceptance Criteria

• File upload via multipart form-data succeeds and returns file metadata with ID and URL
• Uploading the same file content twice creates two Files records but only one HashedFiles / S3 object
• Files exceeding the size limit are rejected with a 413 error and descriptive message
• Frontend validates file size before upload and shows error toast for over-limit files
• Upload progress bar reflects actual upload progress via onUploadProgress
• File download streams content from S3 with correct Content-Type and Content-Disposition
• File metadata endpoint returns correct filename, size, extension, content type, and owner
• Deleting a file removes the Files record; S3 object persists if other records reference the same hash
• Organization-scoped file listing returns only files belonging to that organization
• STATIC_FILE_URL is correctly prepended to file URLs in API responses
• Microsoft-sourced files are stored and retrievable through the same file endpoints
• File owner is correctly set to the authenticated user on upload

---

​

8. Risks