immich-app/immich
1
0
Fork 0
mirror of https://github.com/immich-app/immich.git synced 2026-07-25 14:00:45 +03:00
Code Issues 783 Packages Projects Releases 22 Wiki Activity

Duplicate Detection Incorrectly Blocks Re-upload of Trashed Assets #8199

New Issue
Closed
opened 2026-02-05 13:35:28 +03:00 by OVERLORD · 3 comments
OVERLORD commented 2026-02-05 13:35:28 +03:00
Owner
Copy Link

Originally created by @richard1912 on GitHub (Jan 5, 2026).

Duplicate Detection Incorrectly Blocks Re-upload of Trashed Assets

Description

Immich's duplicate detection prevents re-uploading files that have been deleted/trashed, even though they are not visible in the library. The unique constraint on (owner_id, checksum) applies to all assets including trashed ones, making it impossible to re-upload a file that was previously deleted from an album.

Steps to Reproduce

  1. Upload a file to Immich (e.g., 20260101_122835_45c127f5.mp4)
  2. Delete the file from an album or from the library
  3. Try to re-upload the same file from your filesystem
  4. Observe the "duplicate detected" error

Expected Behavior

Users should be able to re-upload files that are in the trash, as trashed assets are not part of the active library and should not trigger duplicate detection.

Actual Behavior

  • Upload fails with "duplicate detected" message
  • Backend logs show: PostgresError: duplicate key value violates unique constraint "UQ_assets_owner_checksum"
  • The trashed asset is invisible in the library but still blocks new uploads
  • There's no way to force-upload or recover from this state without database intervention

Environment

  • Immich Version: release
  • Installation Method: Docker Compose
  • Database: PostgreSQL 14
  • OS: Windows 11

Technical Details

Root Cause

The unique constraint that enforces duplicate detection is:

CONSTRAINT "UQ_assets_owner_checksum" UNIQUE (ownerId, checksum) WHERE libraryId IS NULL

This constraint applies to ALL assets, regardless of their status. When an asset is deleted, it's marked as status = 'trashed' with a deletedAt timestamp, but the constraint still prevents new uploads with the same checksum.

Database Evidence

SELECT id, "originalFileName", status, visibility, "deletedAt"
FROM asset
WHERE "originalFileName" ILIKE '%20260101_122835_45c127f5%'
AND status = 'trashed';

Result shows 4 trashed copies of the same file, all blocking re-upload:

                  id                  |        originalFileName        | status  | visibility |         deletedAt
--------------------------------------+--------------------------------+---------+------------+----------------------------
 0132902b-2b7e-45f5-8660-39a1ecad2203 | 20260101_122835_45c127f5.mp4   | trashed | timeline   | 2026-01-04 08:03:48.577+00
 56605628-6d5b-458c-b2ea-368228884c22 | 20260101_122835_45c127f5_1.mp4 | trashed | timeline   | 2026-01-04 08:03:48.577+00
 59ebfe12-77b5-40bb-b2b5-ee3aa14c86d1 | 20260101_122835_45c127f5.mp4   | trashed | timeline   | 2026-01-04 08:03:48.577+00
 dec7fdfa-162a-4af9-9e86-fddc3a100cb3 | 20260101_122835_45c127f5.mp4   | trashed | timeline   | 2026-01-04 08:03:48.577+00

Server Logs

[Nest] 7  - 01/05/2026, 12:05:51 PM  DEBUG [Microservices:DuplicateService] Found 1 duplicate for asset 9dc5e9db-27cc-45bc-a6c9-58cd7678e14a
error: PostgresError: duplicate key value violates unique constraint "UQ_assets_owner_checksum"

Proposed Solution

Modify the duplicate detection logic to exclude trashed assets. The DuplicateService should:

  1. Update the unique constraint to exclude trashed assets:

    CONSTRAINT "UQ_assets_owner_checksum" UNIQUE (ownerId, checksum)
WHERE libraryId IS NULL AND status != 'trashed'

    OR

  2. Modify the duplicate detection query in the DuplicateService to exclude trashed assets:

    // When checking for duplicates, only consider active assets
WHERE ownerId = ? AND checksum = ? AND status != 'trashed'

This way:

  • Users can re-upload files that are in trash
  • Trashed files won't interfere with new uploads
  • The trash system remains intact
  • If a user restores a file from trash, duplicate detection will work correctly against it

Impact

  • Users cannot recover from accidental deletions by re-uploading
  • Deleted files become "zombie" entries - invisible but blocking
  • Creates confusion as search finds nothing but upload says it's a duplicate
  • Workaround requires manual database intervention

Additional Notes

  • The system currently has 318+ trashed assets, suggesting this is a common scenario
  • Trash functionality is working (assets are properly marked as trashed)
  • The issue is only in the duplicate detection logic not respecting the trashed status
Originally created by @richard1912 on GitHub (Jan 5, 2026). # Duplicate Detection Incorrectly Blocks Re-upload of Trashed Assets ## Description Immich's duplicate detection prevents re-uploading files that have been deleted/trashed, even though they are not visible in the library. The unique constraint on `(owner_id, checksum)` applies to all assets including trashed ones, making it impossible to re-upload a file that was previously deleted from an album. ## Steps to Reproduce 1. Upload a file to Immich (e.g., `20260101_122835_45c127f5.mp4`) 2. Delete the file from an album or from the library 3. Try to re-upload the same file from your filesystem 4. Observe the "duplicate detected" error ## Expected Behavior Users should be able to re-upload files that are in the trash, as trashed assets are not part of the active library and should not trigger duplicate detection. ## Actual Behavior - Upload fails with "duplicate detected" message - Backend logs show: `PostgresError: duplicate key value violates unique constraint "UQ_assets_owner_checksum"` - The trashed asset is invisible in the library but still blocks new uploads - There's no way to force-upload or recover from this state without database intervention ## Environment - **Immich Version**: release - **Installation Method**: Docker Compose - **Database**: PostgreSQL 14 - **OS**: Windows 11 ## Technical Details ### Root Cause The unique constraint that enforces duplicate detection is: ```sql CONSTRAINT "UQ_assets_owner_checksum" UNIQUE (ownerId, checksum) WHERE libraryId IS NULL ``` This constraint applies to ALL assets, regardless of their status. When an asset is deleted, it's marked as `status = 'trashed'` with a `deletedAt` timestamp, but the constraint still prevents new uploads with the same checksum. ### Database Evidence ```sql SELECT id, "originalFileName", status, visibility, "deletedAt" FROM asset WHERE "originalFileName" ILIKE '%20260101_122835_45c127f5%' AND status = 'trashed'; ``` Result shows 4 trashed copies of the same file, all blocking re-upload: ``` id | originalFileName | status | visibility | deletedAt --------------------------------------+--------------------------------+---------+------------+---------------------------- 0132902b-2b7e-45f5-8660-39a1ecad2203 | 20260101_122835_45c127f5.mp4 | trashed | timeline | 2026-01-04 08:03:48.577+00 56605628-6d5b-458c-b2ea-368228884c22 | 20260101_122835_45c127f5_1.mp4 | trashed | timeline | 2026-01-04 08:03:48.577+00 59ebfe12-77b5-40bb-b2b5-ee3aa14c86d1 | 20260101_122835_45c127f5.mp4 | trashed | timeline | 2026-01-04 08:03:48.577+00 dec7fdfa-162a-4af9-9e86-fddc3a100cb3 | 20260101_122835_45c127f5.mp4 | trashed | timeline | 2026-01-04 08:03:48.577+00 ``` ### Server Logs ``` [Nest] 7 - 01/05/2026, 12:05:51 PM DEBUG [Microservices:DuplicateService] Found 1 duplicate for asset 9dc5e9db-27cc-45bc-a6c9-58cd7678e14a error: PostgresError: duplicate key value violates unique constraint "UQ_assets_owner_checksum" ``` ## Proposed Solution Modify the duplicate detection logic to exclude trashed assets. The DuplicateService should: 1. **Update the unique constraint** to exclude trashed assets: ```sql CONSTRAINT "UQ_assets_owner_checksum" UNIQUE (ownerId, checksum) WHERE libraryId IS NULL AND status != 'trashed' ``` OR 2. **Modify the duplicate detection query** in the DuplicateService to exclude trashed assets: ```typescript // When checking for duplicates, only consider active assets WHERE ownerId = ? AND checksum = ? AND status != 'trashed' ``` This way: - Users can re-upload files that are in trash - Trashed files won't interfere with new uploads - The trash system remains intact - If a user restores a file from trash, duplicate detection will work correctly against it ## Impact - Users cannot recover from accidental deletions by re-uploading - Deleted files become "zombie" entries - invisible but blocking - Creates confusion as search finds nothing but upload says it's a duplicate - Workaround requires manual database intervention ## Additional Notes - The system currently has 318+ trashed assets, suggesting this is a common scenario - Trash functionality is working (assets are properly marked as trashed) - The issue is only in the duplicate detection logic not respecting the trashed status
OVERLORD commented 2026-02-05 13:35:31 +03:00
Author
Owner
Copy Link

@github-actions[bot] commented on GitHub (Jan 5, 2026):

This issue has automatically been closed as it is likely a duplicate. We get a lot of duplicate threads each day, which is why we ask you in the template to confirm that you searched for duplicates before opening one. If you're sure this is not a duplicate, please leave a comment and we will reopen the thread if necessary.

@github-actions[bot] commented on GitHub (Jan 5, 2026): This issue has automatically been closed as it is likely a duplicate. We get a lot of duplicate threads each day, which is why we ask you in the template to confirm that you searched for duplicates before opening one. If you're sure this is not a duplicate, please leave a comment and we will reopen the thread if necessary.
OVERLORD commented 2026-02-05 13:35:32 +03:00
Author
Owner
Copy Link

@richard1912 commented on GitHub (Jan 5, 2026):

Request for Manual Review

This issue appears to be marked as a duplicate, but I believe it addresses a unique root cause that differs from the general duplicate detection constraint error in #22742.

Key Difference

While #22742 reports the constraint error in general, this issue specifically identifies that trashed assets are not excluded from duplicate detection, making it impossible to re-upload files that have been deleted.

Evidence

Database analysis shows 318+ trashed assets with and timestamps are still blocking uploads:

Trashed Assets Still Blocking Re-upload:

  • Asset 1: - status: trashed, deletedAt: 2026-01-04 08:03:48
  • Asset 2: - status: trashed, deletedAt: 2026-01-04 08:03:48
  • Plus 2 more copies of the same file

These are not visible in the library but prevent re-uploading the same file.

Proposed Fix

The duplicate detection logic should exclude assets with from the unique constraint check:

Option 1 (Database level):

CONSTRAINT "UQ_assets_owner_checksum" UNIQUE (ownerId, checksum) 
WHERE libraryId IS NULL AND status != 'trashed'

Option 2 (Application level):
Modify DuplicateService to exclude trashed assets from duplicate checks.

Why This Matters

This creates a user-facing bug where:

  1. Files deleted from albums become invisible (correct behavior)
  2. But they block re-uploading the same file (unexpected behavior)
  3. No way to recover without manual database intervention

This seems like a distinct issue from the general duplicate constraint error, as it specifically relates to how the trash system interacts with duplicate detection.

Would appreciate manual review to determine if this warrants its own issue thread.

@richard1912 commented on GitHub (Jan 5, 2026): ## Request for Manual Review This issue appears to be marked as a duplicate, but I believe it addresses a **unique root cause** that differs from the general duplicate detection constraint error in #22742. ### Key Difference While #22742 reports the constraint error in general, **this issue specifically identifies that trashed assets are not excluded from duplicate detection**, making it impossible to re-upload files that have been deleted. ### Evidence Database analysis shows 318+ trashed assets with and timestamps are still blocking uploads: **Trashed Assets Still Blocking Re-upload:** - Asset 1: - status: trashed, deletedAt: 2026-01-04 08:03:48 - Asset 2: - status: trashed, deletedAt: 2026-01-04 08:03:48 - Plus 2 more copies of the same file These are not visible in the library but prevent re-uploading the same file. ### Proposed Fix The duplicate detection logic should exclude assets with from the unique constraint check: Option 1 (Database level): ```sql CONSTRAINT "UQ_assets_owner_checksum" UNIQUE (ownerId, checksum) WHERE libraryId IS NULL AND status != 'trashed' ``` Option 2 (Application level): Modify DuplicateService to exclude trashed assets from duplicate checks. ### Why This Matters This creates a user-facing bug where: 1. Files deleted from albums become invisible (correct behavior) 2. But they block re-uploading the same file (unexpected behavior) 3. No way to recover without manual database intervention This seems like a distinct issue from the general duplicate constraint error, as it specifically relates to how the trash system interacts with duplicate detection. Would appreciate manual review to determine if this warrants its own issue thread.
OVERLORD commented 2026-02-05 13:35:33 +03:00
Author
Owner
Copy Link

@mmomjian commented on GitHub (Jan 5, 2026):

This is operating as intended.

@mmomjian commented on GitHub (Jan 5, 2026): This is operating as intended.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
accessibility
changelog:enhancement
changelog:security
changelog:skip
changelog:translation
cli
date-time
dependencies
documentation
external-library
format
good first issue
mobile-beta
mobile-beta
mobile-beta
needs-answer
nice to have
pull-request
Mirrored from GitHub Pull Request
 sharing
tech-debt
📱mobile
🖥️web
🗄️server
🧠machine-learning
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
OVERLORD
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: immich-app/immich#8199
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?