40.1 Introduction Chapters 34–39 established the database, authentication, authorization, and API foundations of the ACAI platform.
The next major subsystem is file storage.
An AI platform may process many types of user-owned data: These objects should not normally be stored directly inside PostgreSQL.
Instead, the system should separate: The resulting architecture is: This separation improves scalability, performance, lifecycle management, and security. 40.2 Database vs Object Storage PostgreSQL is appropriate for structured information such as: Object storage is appropriate for: The database therefore acts as the authoritative metadata layer. 40.3 File Architecture The basic flow is: For larger files, the architecture can use a direct or multipart upload pattern: The client should not receive unrestricted storage credentials. 40.4 Storage Key Design A file should have an internal storage key rather than relying only on its original filename.
For example: A generated asset could use: The exact format can vary, but the key should be: unique; predictable only where necessary; independent from user-provided filenames; free of secrets; compatible with lifecycle policies. 40.5 Original Filename vs Storage Key Suppose the user uploads: The application can store: while using an internal key such as: This separation prevents user-provided filenames from becoming the primary identity of the stored object. 40.6 File Metadata The model introduced earlier can be extended.
Example: Potential metadata includes: Only appropriate metadata should be stored. 40.7 File Lifecycle A file should have an explicit lifecycle.
Example: Failure path: Deletion: This allows the application to distinguish between: and: 40.8 Upload Validation A secure upload system should validate at least: The client-provided filename or MIME type should not automatically be treated as authoritative. 40.9 File Size Limits Different resources can have different limits.
For example: The exact limits should be defined by product requirements, infrastructure capacity, and abuse testing.
The server must enforce the limit rather than relying only on the UI. 40.10 MIME Type Validation A browser may report: but the server should not blindly trust that declaration.
Where appropriate, the processing pipeline should inspect the actual file characteristics.
The principle is: 40.11 Extension Handling User-controlled filenames can contain unusual or misleading extensions.
The application should not construct sensitive storage behavior directly from an untrusted filename.
Instead: The storage key should be generated by the application. 40.12 File Ownership Every private file must have an ownership boundary.
Conceptually: Before access: Only after successful authorization should the application provide access. 40.13 Private Storage Private user files should generally remain inaccessible through anonymous public URLs.
The preferred pattern is: This prevents possession of an arbitrary URL from automatically becoming permanent access. 40.14 Temporary Access For private objects, the application can issue a short-lived authorized access mechanism when supported by the storage platform.
Conceptually: The lifetime should be limited to the actual use case. 40.15 Upload Workflow A robust upload sequence is: This creates a traceable lifecycle. 40.16 Processing Pipeline After upload: For a document: For an image: 40.17 Malware and Unsafe File Handling File-processing infrastructure should assume that uploaded files are untrusted.
The processing environment should therefore be isolated from privileged infrastructure as much as practical.
Important principles include: The exact security controls depend on deployment architecture. 40.18 Processing Isolation A file-processing worker should not automatically have access to: The worker should receive only the permissions required for its task.
This follows least-privilege design. 40.19 Temporary Files Processing may require temporary local storage.
For e