By default, uploaded files (such as email attachments) are stored in the MySQL database. However, databases are not designed to store binary data like this efficiently.

Deskpro supports two alternative storage methods:

• Storing files in the filesystem of your server.

• Storing files using Amazon’s S3 cloud storage service.

For better performance, we recommend switching from database storage to either of these two methods.

Switching to filesystem storage

To switch to filesystem storage:

1. Go to Server > File Uploads.

2. Click Change Storage Mechanism.

3. In the pop-up window that opens, select Filesystem.

   

4. Click Save new storage mechanism.

Binary files will begin to be stored in /data/files within the directory where you installed Deskpro. Any files already stored in the database will gradually be transferred to the filesystem in small batches (a few files will be moved each time the once-per-minute cron job/Scheduled Task runs).

Server configuration and file size

Your server configuration determines the maximum file size Deskpro can handle; factors include the amount of memory available to PHP and the values of some settings in your php.ini file. This determines the size of file that Deskpro can handle as an upload from a web interface (agent, portal, chat widget etc) or as an email attachment.

See Server Settings > File Uploads for details of which settings affect the file size limit, and how to change it.

Raising the file size settings in Setup > Settings (under User Attachments and Agent Attachments) will not work if it exceeds the maximum file size determined by the server configuration.

Filesystem storage with multiple web servers

If you have multiple web servers, you must ensure that they are all accessing the same shared filesystem (e.g. using Network Attached Storage). If each web server is storing files to its own filesystem, then when web server A stores a file, web server B will not be able to access it, and you will experience errors.

You can set a custom location for filesystem storage by editing the dir_data variable in the Deskpro config.php file. Use this to set the location of the shared filesystem.

Another way to deal with the issue of file storage with multiple servers is to use Amazon S3. All the web servers will be storing and retrieving their files from a single S3 ‘bucket’.

Switching to Amazon S3 storage

Switching to Amazon S3 cloud storage has two main advantages:

• S3 storage is highly available and highly scalable. This reduces your server administration burden, since you don’t have to ensure that your servers have disk space for all the files.

• If your Deskpro installation uses multiple web servers, it’s easy to have them share the same S3 storage. Using filesystem storage with multiple webservers requires extra configuration overhead.

To use Amazon S3, you must register with Amazon Web Services at http://aws.amazon.com/s3/.

1. Open the Services menu in the top left corner of the screen and click on S3.

2. Click on Create Bucket and choose a bucket name; for compatibility the name should:

   • Have 3 to 63 characters.

   • Contain only lower case letters (at least one), numbers and -.

1. Select a location that is closest to the location of your Deskpro On-Premise webserver.

2. Select your new bucket in the list and click Properties tab in the right pane. Expand the Permissions section and click Add Bucket Policy and paste the following policy into the box:

{
	"Version": "2008-10-17",
	"Statement": [
		{
			"Sid": "AllowPublicRead",
			"Effect": "Allow",
			"Principal": {
				"AWS": "*"
			},
			"Action": [
				"s3:GetObject"
			],
			"Resource": [
				"arn:aws:s3:::YOUR_BUCKET_NAME/*"
			]
		}
	]
}
 copy

(Replace “YOUR_BUCKET_NAME” with the name of the bucket you just created). This policy makes the bucket public, so Deskpro can serve binary objects or “blobs” directly from the Amazon S3 servers.

1. In the top right corner of the AWS site, you will see your name with a pull-down menu control. Use this to access the AWS Security Credentials screen.

2. Click the + icon next to the Access Keys section. Click Create New Access Key and save the Access Key ID and Secret Access Key. It’s very important to save the Secret Access Key now; you will not be able to retrieve it once you close the window.

   For security, you should create a separate IAM user with specific permissions instead of using an access key from your main AWS user (which has full AWS permission to do everything). See the section IAM user for S3 storage below for details.

3. To enable S3 storage within Deskpro, in the admin interface go to Server> File Uploads. Click the Storage Mechanism Options button and switch the mechanism to Amazon S3.

   Enter the Access Key, Secret Key and S3 Bucket Name you configured above, then click Save.

Deskpro will automatically move existing files to S3. This is done gradually, in small batches, as part of the default once-per-minute scheduled task (the same one that is responsible for processing email).

If you do not see files being added to the bucket, see Troubleshooting S3 below.

This process can take a long time, especially if you have a lot of data. If you would like to speed up the process, you can use the dedicated command-line tool:

$ cd /path/to/deskpro
$ php bin/console dp:move-blobs          # Shows an overview
$ php bin/console dp:move-blobs --run    # Actually starts moving files
 copy

It may seem insecure to have every file within your helpdesk (including email attachments) hosted publicly on S3. However, the URL for each file on S3 includes a very long string of random and unguessable characters that essentially acts as a password and ensures that people or bots browsing the web cannot find the ‘public’ files.

Only people who are authorized to see the file can get the URL. While a user or agent who is authorized to see the file could copy the URL and pass it to somebody else, they could also copy the file itself and pass it on. So enabling S3 doesn’t make your files any more accessible than if you use filesystem or database storage.

Note that in the unlikely event that the S3 service becomes unavailable, links to previously-added files will not work. For example, if you had added an image to a Knowledgebase article while you were using S3, that image would not load in the browser while S3 was down. However, your helpdesk as a whole will remain operational. If S3 is unavailable, Deskpro automatically switches back to database storage for newly-created attachments.

IAM user for S3 storage

It is best security practice not to use your root account credentials to set up S3. It is more secure to create a dedicated IAM user within AWS, and use that for S3 access instead. That way, an attacker who steals your Secret Key only gets access to one S3 bucket, instead of your whole AWS account.

See the AWS IAM best practices for an introduction to how IAM works.

You will need to grant the IAM user access to your Deskpro bucket.

Use this policy, either attached inline to the user, or granted through a group:

{
   "Version":"2012-10-17",
   "Statement":[
      {
         "Effect":"Allow",
         "Action":[
            "s3:ListBucket",
            "s3:GetBucketLocation"
         ],
         "Resource":"arn:aws:s3:::YOUR_BUCKET_NAME"
      },
      {
         "Effect":"Allow",
         "Action":[
           "s3:PutObject",
           "s3:GetObject",
           "s3:DeleteObject",
           "s3:PutObjectAcl"
         ],
         "Resource":"arn:aws:s3:::YOUR_BUCKET_NAME/*"
      }
   ]
}
 copy

After you have set this up, you should then generate a Secret Key for the IAM user, and enter that into the S3 settings in Deskpro.

Note

Older keys generated for the same IAM user may not work. Generate a fresh key only after you have set up the bucket and added the policy above to the IAM user.

Troubleshooting S3

If files are not being added to the S3 bucket after you have switched to S3 storage, add this to your config.php file:

$DP_CONFIG['enable_blobstorage_log'] = true;

Then try adding another file to Deskpro, then check the log at data/logs/blob_storage.log. The error messages logged there will give you more information about the problem.

If you see this error:

The request signature we calculated does not match the signature you provided. Check your key and signing method.

the problem is likely due to the Access Key, Secret Key or S3 Bucket Name being incorrectly entered in Deskpro.