Piping uploads direct to S3 with node.js and express
UPDATE: I packed up this functionality into some middleware: multipart-pipe
Recently, express found a small vulnerability in their body parsers where an attacker could DDOS an express app by sending a bunch of uploaded files to the multipart parser, which creates temporary files.
Here’s how you can setup express to pipe uploads direct to S3 without ever writing to disk.
First we add the key piece of middleware:
app.use(express.multipart({ defer: true }));
Make sure you do not have app.use(express.bodyParser())
in your
middleware chain because this causes a redundant (and wrong) multipart
parser.
Here’s the core piece of code:
The req.form
object and its event 'part'
are created by the deferred
multipart middleware; each file will have a part and the data stream can
be piped directly to S3 using the streams2 from node 0.10+.
And whala! No temporary file vulnerability, and all uploads go directly to S3!
Note: cross-blogged at make.rafflecopter.com