Dedicated Storage Configuration Outline

When Open Edx has installed, it has a default configuration of storing user related data files on the local filesystem. The local filesystem stores the data files on the instance/machine itself and occupies instance storage. These data files will be gathered from profile images, ORA uploads and Grades reports. As the data grows, it becomes less efficient to store on the local filesystem. One prominent solution is to use dedicated storage, also termed as object storage (eg: AWS S3, Azure blob).

Another use case of using dedicated storage is when you horizontally scale your application server / Open Edx instance to meet the needs to increase traffic. The approach of horizontal scaling is called auto-scaling and then distributes the traffic between multiple instances using Load balancer. So, storing data files on a single instance is not suitable. Instead, in this scenario we will use a centralized approach which can be achieved through dedicated storage.

Dedicated storage backend

The data files mainly stored on either staticfiles or media directory located in /edx/var/edxapp. To store it onto dedicated storage, a backend storage class need to be implemented. By default, Open Edx uses FileSystemStorage to store into local filesystem. Some of the class has already defined in django-storage package (eg: S3BotoStorage) and is inherited to override the default behaviour. Once the class is inherited, it will redefine the methods of the storing and retrieving files. The derived backend class thus created will be configured in environment files of the Open Edx repository (lms.env.json and cms.env.json).

For profile images, ORA uploads and Grades reports, you will need to work on their code to make compatibility with the derived backend class. Also the backend class will be dependent on third party libraries (for eg: AWS S3 uses boto) so version mismatch will be an issue to consider.

What assets rely on static storage?

• Profile Images
• Video Transcripts
• Course Exports (.tar.gzip files)
• Self-Graded Assessment Uploads
• ORA Uploads
• Data Downloads (e.g. Grade Report, Student Profile Data)
• Theming-related static files