Assets
File assets
There is two types of assets. Files that are uploaded and URLs. We will cover Uploading file assets here, for URL assets view this page.
Uploading files
When uploading files they go directly to S3 (or service that implements the S3 API, eg. Minio, DigitalOcean Spaces, Linode Object Storage, etc...)
Firstly we need to send the metadata about the file to the app. In the response will be signed URLs to allow sending the file in chunks directly to S3. Once the file has been uploaded we will need to notify the app that the file has completed and pass back the id for each chunk that was uploaded.
File size
There is no file size limit when uploading. The only limit is on your plans allocated storage. If the file is larger than your plans' allocation you will receive a 402 error.
For more storage space you'll have to delete files or upgrade your plan.
Step 1. Sending the metadata
Send a POST request with metadata.
POST https://app.digital-downloads.com/api/v1/assets/signed
{
"name": "MyAmazingFile.zip",
"size": 303287472, // filesize in bytes
"mime": "application/zip"
}
Response
{
"chunk_size": 100000000,
"upload_id": "56f30001-2548-489b-be0b-068be4b8102d",
"urls": [
{
"start": 0,
"end": 100000000,
"part": 1,
"url": "https://minio.massivemonkey.io/digital-assets/my-store.myshopify.com/8b0228e8-4f54-4beb-ae84-da8cc149bc60?uploadId=56f30001-2548-489b-be0b-068be4b8102d&partNumber=1&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=massivemonkey%2F20220629%2Fnyc3%2Fs3%2Faws4_request&X-Amz-Date=20220629T015644Z&X-Amz-SignedHeaders=host&X-Amz-Expires=43200&X-Amz-Signature=5145422d9dcfef6374eb4939619735505c3c6e2a171945f917f74294812ec6e3"
},
{
"start": 100000000,
"end": 200000000,
"part": 2,
"url": "https://minio.massivemonkey.io/digital-assets/my-store.myshopify.com/8b0228e8-4f54-4beb-ae84-da8cc149bc60?uploadId=56f30001-2548-489b-be0b-068be4b8102d&partNumber=2&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=massivemonkey%2F20220629%2Fnyc3%2Fs3%2Faws4_request&X-Amz-Date=20220629T015644Z&X-Amz-SignedHeaders=host&X-Amz-Expires=43200&X-Amz-Signature=4f56dbbebc3002a8df0037f5a9192d00490179e23af4f0e72894dd61e93c87ea"
},
{
"start": 200000000,
"end": 300000000,
"part": 3,
"url": "https://minio.massivemonkey.io/digital-assets/my-store.myshopify.com/8b0228e8-4f54-4beb-ae84-da8cc149bc60?uploadId=56f30001-2548-489b-be0b-068be4b8102d&partNumber=3&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=massivemonkey%2F20220629%2Fnyc3%2Fs3%2Faws4_request&X-Amz-Date=20220629T015644Z&X-Amz-SignedHeaders=host&X-Amz-Expires=43200&X-Amz-Signature=447af2a5eef971882a2028fcfd6c1945b37b8c76dd2edd4d3be2def08abe1447"
},
{
"start": 300000000,
"end": 303287472,
"part": 4,
"url": "https://minio.massivemonkey.io/digital-assets/my-store.myshopify.com/8b0228e8-4f54-4beb-ae84-da8cc149bc60?uploadId=56f30001-2548-489b-be0b-068be4b8102d&partNumber=4&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=massivemonkey%2F20220629%2Fnyc3%2Fs3%2Faws4_request&X-Amz-Date=20220629T015644Z&X-Amz-SignedHeaders=host&X-Amz-Expires=43200&X-Amz-Signature=485b0bee23986b5add37dfaf0422a95e897949789f0ec64c0c880237f39ccf2b"
}
],
"id": "e784c8cb-7192-4c6c-9004-17fe31a859fc",
"file_url": "https://app.digital-downloads.com/assets/e784c8cb-7192-4c6c-9004-17fe31a859fc"
}
The urls that are generated are chunked into 100MB parts (but this could change at any time, the chunk size is provided in the payload). Using the URLs send a PUT request to each with the section of the file using the start and end bytes. These chunks do not need to be uploaded in order or wait until the previous part has finished.
Signed URLs expiry
The S3 signed URL are only valid for 12 hours.
Step 2. Uploading file parts to S3
Send a PUT request with chunk of the file. See JavaScript example.
for (let part of urls) {
// get only the part of the file that is required
let blob = file.slice(part.start, part.end);
// create a new http request and remove the content type
const a = axios.create();
delete a.defaults.headers.put['Content-Type'];
a.put(part.url, blob).then((response) => {
// from the header we need to etag, this is S3's id for each part
// of file so we can re construct it when all pieces are uploaded
return {
ETag: response.headers.etag.split('"').join(''),
PartNumber: part.part,
};
});
}
Step 3. Notifying the app that the file has uploaded
Once each chunk has been uploaded to S3 send a POST request with each chunk details back to the app.
POST https://app.digital-downloads.com/api/v1/assets/:id/uploaded
{
"parts":[
{
"ETag": "0d5bf6a8d41ed4d3d8813fa27ee67b6f",
"PartNumber": 1
},
{
"ETag": "4f0d43ac4cd8dc4ec545572c44b8642b",
"PartNumber": 2
},
{
"ETag": "00dbc7cf67bef6ae9eb5c47936f1f289",
"PartNumber": 3
},
{
"ETag": "dd2b66242dba17ac0b7e33b7a70e716b",
"PartNumber": 4
}
],
"upload_id": "56f30001-2548-489b-be0b-068be4b8102d"
}
JavaScript example of the flow
This is an example of the code used within the UI of the app to upload files.
async function upload(file) {
// generate the signed urls
let signedResponse = await axios.post('https://app.digital-downloads.com/api/v1/assets/signed', {
name: file.name,
size: file.size,
mime: file.mime,
}).then((r) => r.data);
// create a new http request and remove the content type
const a = axios.create();
delete a.defaults.headers.put['Content-Type'];
const promises = [];
for (let part of signedResponse.urls) {
// get the part of the file to send in this request
let blob = file.file.slice(part.start, part.end);
promises.push(
// send the PUT request to the url with part of the file
a.put(part.url, blob).then((response) => {
// from the header we need to etag, this is S3's id for each part
// of file so we can re construct it when all pieces are uploaded
return {
ETag: response.headers.etag.split('"').join(''),
PartNumber: part.part,
};
})
);
}
// once all parts have uploaded then we need to send
// the parts with the joined S3 etag back to the app
Promise.all(promises).then((parts) => {
axios.put(`https://app.digital-downloads.com/api/v1/assets/${signedResponse.id}/uploaded`, {
parts: parts,
upload_id: signedResponse.upload_id
}).then((r) => {
// completed
});
});
}
Canceling an upload
If a chunk fails or something goes wrong, or you want to cancel the upload. We need to remove the chunks that got uploaded but are no longer needed. Send a POST request to the asset with the upload id (received from the signed request).
POST https://app.digital-downloads.com/api/v1/assets/:id/cancel
{
"upload_id": "56f30001-2548-489b-be0b-068be4b8102d"
}