starred/pocket-id
1
0
Fork 0
mirror of https://github.com/pocket-id/pocket-id.git synced 2026-02-05 00:39:38 +03:00
Code Issues 61 Packages Projects Releases 10 Wiki Activity

🐛 Bug Report: GCS S3 Compatibility: SignatureDoesNotMatch error with AWS SDK v2 (v1.16.0) #561

New Issue
Closed
opened 2026-02-04 20:28:04 +03:00 by OVERLORD · 1 comment
OVERLORD commented 2026-02-04 20:28:04 +03:00
Owner
Copy Link

Originally created by @plotikai on GitHub (Dec 6, 2025).

Reproduction steps

Hey team, I've spent a bunch of time trying to implement this with ai with no success, I got it to write what ive been trying and looking for help here. Forgive me as im just a humble vibecoding homelabber trying to get this running

Description

The application fails to initialize when configured to use Google Cloud Storage (GCS) as the S3 backend, failing with a SignatureDoesNotMatch error. This occurs despite valid credentials and correct region configuration verified via other tools.

Steps to Reproduce

  1. Create a Google Cloud Storage bucket (e.g., in us-east1).
  2. Generate HMAC keys (Access Key and Secret) for a Service Account with object admin permissions.
  3. Configure the Pocket-ID container with the following environment variables: 
    FILE_BACKEND=s3
S3_ENDPOINT=https://storage.googleapis.com
S3_BUCKET=<your-bucket-name>
S3_REGION=us-east1
S3_ACCESS_KEY_ID=<HMAC_KEY_ID>
S3_SECRET_ACCESS_KEY=<HMAC_SECRET>
S3_FORCE_PATH_STYLE=true # Issue persists with false as well
  4. Start the container (docker compose up).

Troubleshooting Performed

  1. Credential Verification: I verified the HMAC credentials and bucket access using a standalone Python script with boto3 from the same machine. The script successfully listed objects in the bucket using us-east-1 (and us-east1), confirming network access and permissions are correct.
  2. Region Variations:
    • us-east1 (Correct GCS region for my bucket): Result SignatureDoesNotMatch.
    • us-east-1: Result SignatureDoesNotMatch.
    • auto: Result SignatureDoesNotMatch.
    • Empty (S3_REGION=): Result A region must be set.
  3. Path Style: Toggled S3_FORCE_PATH_STYLE between true and false. The error persists (though false is generally recommended for GCS).

Analysis

The error operation error S3: ListObjectsV2 suggests the application is using AWS SDK for Go v2. GCS S3 interoperability has known issues with AWS SDK v2's default behavior, particularly regarding:

  1. ListObjectsV2: GCS has limited support for this API compared to V1.
  2. Header Handling: The Go SDK v2 often includes headers (like Accept-Encoding) in the signature calculation that GCS does not expect, causing the signature mismatch.

It seems the current implementation prevents GCS from being used as a backend. Support for GCS (handling the specific header quirks or falling back to ListObjectsV1) would be greatly appreciated.

Expected behavior

The application should successfully authenticate with the S3-compatible GCS endpoint, initialize the storage backend, and start up.

Actual Behavior

The application fails to start and enters a restart loop. The logs show a 403 Forbidden error with SignatureDoesNotMatch during the ListObjectsV2 operation:

Pocket ID Version

1.16.0

Database

PostgreSQL 17.6

OS and Environment

Ubuntu 24.04.3 LTS, Docker v29.0.2, Docker Compose v2.40.3
Traefik v3.6.1

Log Output

ERR Failed to run pocket-id app=pocket-id version=1.16.0 error="failed to initialize application images: failed to list application images: operation error S3: ListObjectsV2, https response error StatusCode: 403, RequestID: <ID>, HostID: <ID>, api error SignatureDoesNotMatch: Access denied."
Originally created by @plotikai on GitHub (Dec 6, 2025). ### Reproduction steps Hey team, I've spent a bunch of time trying to implement this with ai with no success, I got it to write what ive been trying and looking for help here. Forgive me as im just a humble vibecoding homelabber trying to get this running ## Description The application fails to initialize when configured to use Google Cloud Storage (GCS) as the S3 backend, failing with a `SignatureDoesNotMatch` error. This occurs despite valid credentials and correct region configuration verified via other tools. ## Steps to Reproduce 1. Create a Google Cloud Storage bucket (e.g., in `us-east1`). 2. Generate HMAC keys (Access Key and Secret) for a Service Account with object admin permissions. 3. Configure the Pocket-ID container with the following environment variables: ```bash FILE_BACKEND=s3 S3_ENDPOINT=https://storage.googleapis.com S3_BUCKET=<your-bucket-name> S3_REGION=us-east1 S3_ACCESS_KEY_ID=<HMAC_KEY_ID> S3_SECRET_ACCESS_KEY=<HMAC_SECRET> S3_FORCE_PATH_STYLE=true # Issue persists with false as well ``` 4. Start the container (`docker compose up`). ## Troubleshooting Performed 1. **Credential Verification**: I verified the HMAC credentials and bucket access using a standalone Python script with `boto3` from the same machine. The script **successfully** listed objects in the bucket using `us-east-1` (and `us-east1`), confirming network access and permissions are correct. 2. **Region Variations**: * `us-east1` (Correct GCS region for my bucket): Result `SignatureDoesNotMatch`. * `us-east-1`: Result `SignatureDoesNotMatch`. * `auto`: Result `SignatureDoesNotMatch`. * Empty (`S3_REGION=`): Result `A region must be set`. 3. **Path Style**: Toggled `S3_FORCE_PATH_STYLE` between `true` and `false`. The error persists (though `false` is generally recommended for GCS). ## Analysis The error `operation error S3: ListObjectsV2` suggests the application is using AWS SDK for Go v2. GCS S3 interoperability has known issues with AWS SDK v2's default behavior, particularly regarding: 1. **ListObjectsV2**: GCS has limited support for this API compared to V1. 2. **Header Handling**: The Go SDK v2 often includes headers (like `Accept-Encoding`) in the signature calculation that GCS does not expect, causing the signature mismatch. It seems the current implementation prevents GCS from being used as a backend. Support for GCS (handling the specific header quirks or falling back to ListObjectsV1) would be greatly appreciated. ### Expected behavior The application should successfully authenticate with the S3-compatible GCS endpoint, initialize the storage backend, and start up. ### Actual Behavior The application fails to start and enters a restart loop. The logs show a 403 Forbidden error with `SignatureDoesNotMatch` during the `ListObjectsV2` operation: ### Pocket ID Version 1.16.0 ### Database PostgreSQL 17.6 ### OS and Environment Ubuntu 24.04.3 LTS, Docker v29.0.2, Docker Compose v2.40.3 Traefik v3.6.1 ### Log Output ``` ERR Failed to run pocket-id app=pocket-id version=1.16.0 error="failed to initialize application images: failed to list application images: operation error S3: ListObjectsV2, https response error StatusCode: 403, RequestID: <ID>, HostID: <ID>, api error SignatureDoesNotMatch: Access denied." ```
OVERLORD commented 2026-02-04 20:28:18 +03:00
Author
Owner
Copy Link

@stonith404 commented on GitHub (Dec 17, 2025):

Thanks for the bug report. This seems like a common issue with the new AWS SDK: https://github.com/aws/aws-sdk-go-v2/issues/1816.

We would have to implement a custom middleware that ignores the Accept-Encoding header for GCS. In my opinion it's not worth to implement custom logic for a specific S3 provider, this should rather be fixed by Google or Amazon.

Here are the changes that would be necessary to add support for GCS to Pocket ID. If you really need GCS compatibility you can fork the project and apply those changes:
GCS compatibility.patch

@stonith404 commented on GitHub (Dec 17, 2025): Thanks for the bug report. This seems like a common issue with the new AWS SDK: https://github.com/aws/aws-sdk-go-v2/issues/1816. We would have to implement a custom middleware that ignores the `Accept-Encoding` header for GCS. In my opinion it's not worth to implement custom logic for a specific S3 provider, this should rather be fixed by Google or Amazon. Here are the changes that would be necessary to add support for GCS to Pocket ID. If you really need GCS compatibility you can fork the project and apply those changes: [GCS compatibility.patch](https://github.com/user-attachments/files/24210562/GCS.compatibility.patch)
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
breaking
bug
bug
bug
documentation
feature
needs more upvotes
open to pull requests
pull-request
Mirrored from GitHub Pull Request
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
OVERLORD
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/pocket-id#561
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?