[BUG] Duplicated images in a source folder are uploaded twice with the CLI tool #825

Closed
opened 2026-02-04 22:50:33 +03:00 by OVERLORD · 5 comments
Owner

Originally created by @Kernald on GitHub (Apr 20, 2023).

The bug

I'm using Immich as a mirror of my main photo collection, and as such only upload through the CLI tool. My source of truth is Lightroom. I have a few photos that are in multiple albums in Lightroom, and match this layout on export. So if I have e.g. a photo Foo in two albums, this is what my FS looks like:

  • a/
  • a/Bar.jpg
  • a/Foo.jpg
  • b/
  • b/Foo.jpg
  • c/
  • c/Baz.jpg

My goal was to end up with the same album structure in Immich automatically. The result was however a bit different, and is actually different between the web and Android too:

  • On the web, I see the image twice in the feed. Both have the same file name, and each is in a distinct album (e.g. the first one is in a and the second one is in b). The navigation seems a bit off at times too - clicking on the first instance of the photo and then right will sometimes skip over the second one and display the following photo directly.
  • On Android, I can't easily see the album association but I assume this part is identical. What's different however, is that one is named Foo.jpg, while the other is named Foo+1.jpg.

I'm not too sure how much of this is a bug and how much is expected (at the very least, I expect the navigation issues on the web to be an actual bug, but the rest can arguably be expected behaviour). If it indeed is expected, I'm curious if there's another solution here?

The OS that Immich Server is running on

Docker/Arch

Version of Immich Server

v1.54.0

Version of Immich Mobile App

v1.54.0-build.77

Platform with the issue

  • Server
  • Web
  • Mobile

Your docker-compose.yml content

Copied from the doc

Your .env content

Copied from the doc

Reproduction steps

1. Make a folder hierarchy with the same photo present twice in different folders
2. Upload the folder with the CLI tool
3. Notice the behaviour described above

Additional information

No response

Originally created by @Kernald on GitHub (Apr 20, 2023). ### The bug I'm using Immich as a mirror of my main photo collection, and as such only upload through the CLI tool. My source of truth is Lightroom. I have a few photos that are in multiple albums in Lightroom, and match this layout on export. So if I have e.g. a photo Foo in two albums, this is what my FS looks like: - `a/` - `a/Bar.jpg` - `a/Foo.jpg` - `b/` - `b/Foo.jpg` - `c/` - `c/Baz.jpg` My goal was to end up with the same album structure in Immich automatically. The result was however a bit different, and is actually different between the web and Android too: - On the web, I see the image twice in the feed. Both have the same file name, and each is in a distinct album (e.g. the first one is in `a` and the second one is in `b`). The navigation seems a bit off at times too - clicking on the first instance of the photo and then right will sometimes skip over the second one and display the following photo directly. - On Android, I can't easily see the album association but I assume this part is identical. What's different however, is that one is named `Foo.jpg`, while the other is named `Foo+1.jpg`. I'm not too sure how much of this is a bug and how much is expected (at the very least, I expect the navigation issues on the web to be an actual bug, but the rest can arguably be expected behaviour). If it indeed is expected, I'm curious if there's another solution here? ### The OS that Immich Server is running on Docker/Arch ### Version of Immich Server v1.54.0 ### Version of Immich Mobile App v1.54.0-build.77 ### Platform with the issue - [X] Server - [X] Web - [X] Mobile ### Your docker-compose.yml content ```YAML Copied from the doc ``` ### Your .env content ```Shell Copied from the doc ``` ### Reproduction steps ```bash 1. Make a folder hierarchy with the same photo present twice in different folders 2. Upload the folder with the CLI tool 3. Notice the behaviour described above ``` ### Additional information _No response_
Author
Owner

@alextran1502 commented on GitHub (Apr 20, 2023):

Are they the same file but in different album or they are different file i.e was edited differently with the same name?

@alextran1502 commented on GitHub (Apr 20, 2023): Are they the same file but in different album or they are different file i.e was edited differently with the same name?
Author
Owner

@Kernald commented on GitHub (Apr 20, 2023):

They're the same photo exactly in Lightroom, exported twice with the exact same settings. I assume that the checksum might be different, given that last step - I'll check a bit later!

@Kernald commented on GitHub (Apr 20, 2023): They're the same photo exactly in Lightroom, exported twice with the exact same settings. I assume that the checksum might be different, given that last step - I'll check a bit later!
Author
Owner

@Kernald commented on GitHub (Apr 20, 2023):

I can confirm that they have different checksums. Removing everything from Immich, making sure the duplicated exports have the same checksum, and importing again behaves as expected. I wonder if a content-aware duplicate detection might be desirable? I know it's a feature I wished Google Photos had for quite a while too...

@Kernald commented on GitHub (Apr 20, 2023): I can confirm that they have **different** checksums. Removing everything from Immich, making sure the duplicated exports have the same checksum, and importing again behaves as expected. I wonder if a content-aware duplicate detection might be desirable? I know it's a feature I wished Google Photos had for quite a while too...
Author
Owner

@alextran1502 commented on GitHub (Apr 20, 2023):

@Kernald Content-aware is a more complicated feature that we will look into adding to the application down the road. Right now we only have hash deduplication. Sorry for the inconvenience!

@alextran1502 commented on GitHub (Apr 20, 2023): @Kernald Content-aware is a more complicated feature that we will look into adding to the application down the road. Right now we only have hash deduplication. Sorry for the inconvenience!
Author
Owner

@Kernald commented on GitHub (Apr 21, 2023):

@alextran1502 no worries, that's definitely fair enough. Knowing that I can just copy my export in the two album folders works for me for now. I'll try to pinpoint exactly what's causing the navigation issues on web - my hunch is that it's caused by the exact same timestamp, but I might be wrong - and open a new issue just for that one. Thanks!

@Kernald commented on GitHub (Apr 21, 2023): @alextran1502 no worries, that's definitely fair enough. Knowing that I can just copy my export in the two album folders works for me for now. I'll try to pinpoint exactly what's causing the navigation issues on web - my hunch is that it's caused by the exact same timestamp, but I might be wrong - and open a new issue just for that one. Thanks!
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: immich-app/immich#825