Double URL Encoding Bug In Presigned URLs

by Luna Greco 42 views

Hey everyone, let's dive into a tricky bug we've encountered with presigned URLs when dealing with filenames that have special characters. This issue can be a real headache, especially when you're trying to provide secure access to your files stored in S3. So, let's break down what's happening, why it's happening, and how it impacts your applications.

Understanding the Bug

When using presigned URLs (presignedUrl: true) with special characters in the filename such as brackets, spaces, or other characters that are sensitive to URLs are being double-encoded. What this means is that these URLs become malformed and inaccessible, which can be a major roadblock for your users. This bug specifically affects filenames with special characters when using presigned URLs. Presigned URLs are crucial for granting temporary access to your files, making this a high-priority issue.

Bug Details

  • Affected Version: Current version
  • Component: getFileLocation() method in index.js
  • Severity: High - Breaks file access for filenames with special characters when using presigned URLs

Problem Description

The heart of the issue lies within the getFileLocation() method. Here's a breakdown of the process that leads to the double encoding:

  1. Initial Encoding: The filename undergoes an initial encoding process using filename.split('/').map(encodeURIComponent).join('/'). This step is intended to ensure that the filename is properly formatted for use in a URL.
  2. S3 Key Creation: The encoded filename is then used to construct the S3 key for generating the presigned URL. Think of this key as the address of your file within your S3 bucket.
  3. Double Encoding by AWS SDK: The AWS SDK's getSignedUrl() function steps in to generate the presigned URL. However, it internally encodes the key again, leading to the double encoding. This is where things go south, as the URL becomes overly encoded and, thus, invalid.

Let's illustrate this with an example. Say you have a filename like doc[123].pdf:

  • Expected Behavior: The filename should be encoded to doc%5B123%5D.pdf in the presigned URL. This is the standard URL encoding for the brackets.
  • Actual Behavior: The filename gets double-encoded, resulting in doc%255B123%255D.pdf. Yikes! Here's how it happens:
    • First encoding: [123] → %5B123%5D
    • Second encoding by AWS SDK: %5B123%5D → %255B123%255D

Root Cause Analysis

The culprit is the combination of pre-encoding the filename and the AWS SDK's built-in encoding. Here's the snippet of code where the issue manifests:

// Line 242: Pre-encodes the filename
const fileName = filename.split('/').map(encodeURIComponent).join('/');

// Line 247: Uses pre-encoded filename for S3 key
const fileKey = `${this._bucketPrefix}${fileName}`;

// Line 251: AWS SDK encodes the already-encoded key again
const params = { Bucket: this._bucket, Key: fileKey };
presignedUrl = await this.getFileSignedUrl(this._s3Client, command, options);

On Line 242, the filename is pre-encoded. Then, on Line 247, this pre-encoded filename is used to create the S3 key. Finally, on Line 251, the AWS SDK encodes the key once more, leading to our double-encoding problem. The key takeaway here is that the AWS SDK's encoding is redundant when we've already encoded the filename.

Impact of the Bug

The impact of this bug can be quite significant:

  • Inaccessible Presigned URLs: Any presigned URLs generated for files with special characters become inaccessible. This means users won't be able to download or access these files using the generated URLs.
  • Affected Filenames: This issue affects a wide range of filenames, specifically those containing characters like [], (), spaces, and other URL-sensitive characters. These characters are commonly used in filenames, making the bug quite pervasive.
  • Regular S3 URLs Work Fine: Interestingly, regular S3 URLs (i.e., non-presigned URLs) are not affected by this bug. This is because they don't go through the AWS SDK's internal encoding process. So, if you're not using presigned URLs, you might not even notice this issue.

Steps to Reproduce the Bug

If you want to see this bug in action, here's how you can reproduce it:

  1. Configure S3 Adapter: Set up your S3 adapter with the presignedUrl: true option enabled. This tells the adapter to generate presigned URLs.
  2. Upload a File: Upload a file with special characters in its name. A good example would be `