Skip to content

Commit 7d615da

Browse files
author
Kelvin Luck
committed
fix: Send Prefix to aws to limit files that we need to scan
Prior to this change, when you tried to activate a revision we list *all* revisions to try and discover if the passed argument is valid. In our case this ends up fetching tens of thousands of revisions from s3 which is kinda slow... It turns out that the AWS `listObjects` function takes a prefix (this is already used in the code to somewhat limit the results). We can pass the entire expected filename and retrieve exactly one record (which is of course much much faster)
1 parent 3c38b98 commit 7d615da

File tree

1 file changed

+12
-3
lines changed

1 file changed

+12
-3
lines changed

lib/s3.js

Lines changed: 12 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -131,9 +131,8 @@ module.exports = CoreObject.extend({
131131
params.ServerSideEncryption = serverSideEncryption;
132132
}
133133

134-
return this.fetchRevisions(options).then(function(revisions) {
135-
var found = revisions.map(function(element) { return element.revision; }).indexOf(options.revisionKey);
136-
if (found >= 0) {
134+
return this.findRevision(options).then(function(found) {
135+
if (found !== undefined) {
137136
return copyObject(params).then(function() {
138137
plugin.log('✔ ' + revisionKey + " => " + indexKey);
139138
});
@@ -143,6 +142,16 @@ module.exports = CoreObject.extend({
143142
});
144143
},
145144

145+
findRevision: function(options) {
146+
var client = this._client;
147+
var listObjects = RSVP.denodeify(client.listObjects.bind(client));
148+
var bucket = options.bucket;
149+
var prefix = options.prefix;
150+
var revisionPrefix = joinUriSegments(prefix, options.filePattern + ":" + options.revisionKey);
151+
152+
return listObjects({ Bucket: bucket, Prefix: revisionPrefix }).then((response) => response.Contents[0]);
153+
},
154+
146155
fetchRevisions: function(options) {
147156
var client = this._client;
148157
var bucket = options.bucket;

0 commit comments

Comments
 (0)