immich-app/immich
1
0
Fork 0
mirror of https://github.com/immich-app/immich.git synced 2026-02-25 03:12:21 +03:00
Code Issues 783 Packages Projects Releases 12 Wiki Activity

[BUG] Getting many duplicated photos #327

New Issue
Closed
opened 2026-02-04 19:40:14 +03:00 by OVERLORD · 5 comments
OVERLORD commented 2026-02-04 19:40:14 +03:00
Owner
Copy Link

Originally created by @nsantini on GitHub (Sep 23, 2022).

Describe the bug
I connected the app to the server and backed up all the photos by enabling auto backup. Then I downloaded my Google Photos and uploaded them to the server via the CLI tool.
I'm now seeing lots of "duplicates" both on the mobile app and on the web server.

Task List

Please complete the task list below. We need this information to help us reproduce the bug or point out problems in your setup. You are not providing enough info may delay our effort to help you.

  • [ x ] I have read thoroughly the README setup and installation instructions.
  • [ x ] I have included my docker-compose file.
  • [ x ] I have included my redacted .env file.
  • [ x ] I have included information on my machine, and environment.

To Reproduce
Steps to reproduce the behavior:

  1. Enable auto backup in the phone
  2. Upload photos via the CLI tool

Expected behavior
When the phone gets access to the CLI uploaded photos it does not duplicate them by re-uploading them

Screenshots
On the mobile app the duplicates have two different icons, one with a cloud and a tick, the other with a photo icon.
IMG_3DD2467656B4-1

System

  • Phone OS: iOS: 16.0
  • Server Version: v1.29.5
  • Mobile App Version: 1.29.4 build.59

Docker compose

version: "3.8"

services:
  immich-server:
    image: altran1502/immich-server:release
    entrypoint: ["/bin/sh", "./start-server.sh"]
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
    env_file:
      - .env
    environment:
      - NODE_ENV=production
    depends_on:
      - redis
      - database
    restart: always

  immich-microservices:
    image: altran1502/immich-server:release
    entrypoint: ["/bin/sh", "./start-microservices.sh"]
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
    env_file:
      - .env
    environment:
      - NODE_ENV=production
    depends_on:
      - redis
      - database
    restart: always

  immich-machine-learning:
    image: altran1502/immich-machine-learning:release
    entrypoint: ["/bin/sh", "./entrypoint.sh"]
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
    env_file:
      - .env
    environment:
      - NODE_ENV=production
    depends_on:
      - database
    restart: always

  immich-web:
    image: altran1502/immich-web:release
    entrypoint: ["/bin/sh", "./entrypoint.sh"]
    env_file:
   - .env
    environment:
      - PUBLIC_TZ=${TZ}
    restart: always

  redis:
    container_name: immich_redis
    image: redis:6.2
    restart: always

  database:
    container_name: immich_postgres
    image: postgres:14
    env_file:
      - .env
    environment:
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      POSTGRES_USER: ${DB_USERNAME}
      POSTGRES_DB: ${DB_DATABASE_NAME}
      PG_DATA: /var/lib/postgresql/data
    volumes:
      - pgdata:/var/lib/postgresql/data
    restart: always

  immich-proxy:
    container_name: immich_proxy
    image: altran1502/immich-proxy:release
    ports:
      - 2283:8080
    logging:
      driver: none
    depends_on:
      - immich-server
    restart: always

volumes:
  pgdata:

Env File

###################################################################################
# Database
###################################################################################

DB_HOSTNAME=<<redacted>>
DB_USERNAME=<<redacted>>
DB_PASSWORD=<<redacted>>
DB_DATABASE_NAME=<<redacted>>

# Optional Database settings:
# DB_PORT=5432




###################################################################################
# Redis
###################################################################################

REDIS_HOSTNAME=immich_redis

# Optional Redis settings:
# REDIS_PORT=6379
# REDIS_DBINDEX=0
# REDIS_PASSWORD=
# REDIS_SOCKET=

###################################################################################
# Upload File Config
###################################################################################

UPLOAD_LOCATION=<<redacted>>


###################################################################################
# Log message level - [simple|verbose]
###################################################################################

LOG_LEVEL=simple


###################################################################################
# JWT SECRET
###################################################################################

JWT_SECRET=<<redacted>>




###################################################################################
# MAPBOX
####################################################################################

# ENABLE_MAPBOX is either true of false -> if true, you have to provide MAPBOX_KEY
ENABLE_MAPBOX=false
MAPBOX_KEY=


####################################################################################
# WEB - Optional
####################################################################################

# Custom message on the login page, should be written in HTML form.
# For example PUBLIC_LOGIN_PAGE_MESSAGE="This is a demo instance of Immich.<br><br>Email: <i>demo@demo.de</i><br>Password: <i>demo</i>"

PUBLIC_LOGIN_PAGE_MESSAGE=

# For correctly display your local time zone on the web, you can set the time zone here.
# Should work fine by default value, however, in case of incorrect timezone in EXIF, this value
# should be set to the correct timezone.
# Command to get timezone:
# - Linux: curl -s http://ip-api.com/json/ | grep -oP '(?<=timezone":")(.*?)(?=")'

TZ=Pacific/Auckland
Originally created by @nsantini on GitHub (Sep 23, 2022). **Describe the bug** I connected the app to the server and backed up all the photos by enabling auto backup. Then I downloaded my Google Photos and uploaded them to the server via the CLI tool. I'm now seeing lots of "duplicates" both on the mobile app and on the web server. **Task List** *Please complete the task list below. We need this information to help us reproduce the bug or point out problems in your setup. You are not providing enough info may delay our effort to help you.* - [ x ] I have read thoroughly the README setup and installation instructions. - [ x ] I have included my `docker-compose` file. - [ x ] I have included my redacted `.env` file. - [ x ] I have included information on my machine, and environment. **To Reproduce** Steps to reproduce the behavior: 1. Enable auto backup in the phone 2. Upload photos via the CLI tool **Expected behavior** When the phone gets access to the CLI uploaded photos it does not duplicate them by re-uploading them **Screenshots** On the mobile app the duplicates have two different icons, one with a cloud and a tick, the other with a photo icon. ![IMG_3DD2467656B4-1](https://user-images.githubusercontent.com/438741/191852962-8198f408-32f6-4cf7-89f6-072fdb651e6d.jpeg) **System** - Phone OS: iOS: `16.0` - Server Version: `v1.29.5` - Mobile App Version: `1.29.4 build.59` **Docker compose** ``` version: "3.8" services: immich-server: image: altran1502/immich-server:release entrypoint: ["/bin/sh", "./start-server.sh"] volumes: - ${UPLOAD_LOCATION}:/usr/src/app/upload env_file: - .env environment: - NODE_ENV=production depends_on: - redis - database restart: always immich-microservices: image: altran1502/immich-server:release entrypoint: ["/bin/sh", "./start-microservices.sh"] volumes: - ${UPLOAD_LOCATION}:/usr/src/app/upload env_file: - .env environment: - NODE_ENV=production depends_on: - redis - database restart: always immich-machine-learning: image: altran1502/immich-machine-learning:release entrypoint: ["/bin/sh", "./entrypoint.sh"] volumes: - ${UPLOAD_LOCATION}:/usr/src/app/upload env_file: - .env environment: - NODE_ENV=production depends_on: - database restart: always immich-web: image: altran1502/immich-web:release entrypoint: ["/bin/sh", "./entrypoint.sh"] env_file: - .env environment: - PUBLIC_TZ=${TZ} restart: always redis: container_name: immich_redis image: redis:6.2 restart: always database: container_name: immich_postgres image: postgres:14 env_file: - .env environment: POSTGRES_PASSWORD: ${DB_PASSWORD} POSTGRES_USER: ${DB_USERNAME} POSTGRES_DB: ${DB_DATABASE_NAME} PG_DATA: /var/lib/postgresql/data volumes: - pgdata:/var/lib/postgresql/data restart: always immich-proxy: container_name: immich_proxy image: altran1502/immich-proxy:release ports: - 2283:8080 logging: driver: none depends_on: - immich-server restart: always volumes: pgdata: ``` **Env File** ``` ################################################################################### # Database ################################################################################### DB_HOSTNAME=<<redacted>> DB_USERNAME=<<redacted>> DB_PASSWORD=<<redacted>> DB_DATABASE_NAME=<<redacted>> # Optional Database settings: # DB_PORT=5432 ################################################################################### # Redis ################################################################################### REDIS_HOSTNAME=immich_redis # Optional Redis settings: # REDIS_PORT=6379 # REDIS_DBINDEX=0 # REDIS_PASSWORD= # REDIS_SOCKET= ################################################################################### # Upload File Config ################################################################################### UPLOAD_LOCATION=<<redacted>> ################################################################################### # Log message level - [simple|verbose] ################################################################################### LOG_LEVEL=simple ################################################################################### # JWT SECRET ################################################################################### JWT_SECRET=<<redacted>> ################################################################################### # MAPBOX #################################################################################### # ENABLE_MAPBOX is either true of false -> if true, you have to provide MAPBOX_KEY ENABLE_MAPBOX=false MAPBOX_KEY= #################################################################################### # WEB - Optional #################################################################################### # Custom message on the login page, should be written in HTML form. # For example PUBLIC_LOGIN_PAGE_MESSAGE="This is a demo instance of Immich.<br><br>Email: <i>demo@demo.de</i><br>Password: <i>demo</i>" PUBLIC_LOGIN_PAGE_MESSAGE= # For correctly display your local time zone on the web, you can set the time zone here. # Should work fine by default value, however, in case of incorrect timezone in EXIF, this value # should be set to the correct timezone. # Command to get timezone: # - Linux: curl -s http://ip-api.com/json/ | grep -oP '(?<=timezone":")(.*?)(?=")' TZ=Pacific/Auckland ```
OVERLORD commented 2026-02-04 19:40:35 +03:00
Author
Owner
Copy Link

@zackpollard commented on GitHub (Sep 23, 2022):

Hey! So I am currently doing the same and working through a bunch of issues this caused for me. Personally I am currently planning to just accept the inevitable duplicates. The reason this happens will be if you had "High Quality" mode enabled in google photos. Essentially this doesn't save the original file in google photos and instead, it compresses the file. Currently our duplicate detection is only for photos that are exactly the same, however, we are planning on building some tools later into the UI that can detect similar photos, which should help for these kind of cases where the photos are visually the same, but different on disk. I'm going to close this for now as it is expected behaviour, however we do plan to have a solution to this at some point, but it might not be until after our first major release.

@zackpollard commented on GitHub (Sep 23, 2022): Hey! So I am currently doing the same and working through a bunch of issues this caused for me. Personally I am currently planning to just accept the inevitable duplicates. The reason this happens will be if you had "High Quality" mode enabled in google photos. Essentially this doesn't save the original file in google photos and instead, it compresses the file. Currently our duplicate detection is only for photos that are exactly the same, however, we are planning on building some tools later into the UI that can detect similar photos, which should help for these kind of cases where the photos are visually the same, but different on disk. I'm going to close this for now as it is expected behaviour, however we do plan to have a solution to this at some point, but it might not be until after our first major release.
OVERLORD commented 2026-02-04 19:40:47 +03:00
Author
Owner
Copy Link

@nsantini commented on GitHub (Sep 23, 2022):

Thanks @zackpollard for the explanation. Do you have any idea of how to approach a manual dedup process?

Also, about those two different Icons I see on the duplicated photos on the mobile app. What do they mean? one on device and one on server? Would be great to understand that so I know how to deal with them

@nsantini commented on GitHub (Sep 23, 2022): Thanks @zackpollard for the explanation. Do you have any idea of how to approach a manual dedup process? Also, about those two different Icons I see on the duplicated photos on the mobile app. What do they mean? one on device and one on server? Would be great to understand that so I know how to deal with them
OVERLORD commented 2026-02-04 19:41:02 +03:00
Author
Owner
Copy Link

@nsantini commented on GitHub (Sep 23, 2022):

Also, should I try to delete duplicates on the mobile app or on in the web view?

@nsantini commented on GitHub (Sep 23, 2022): Also, should I try to delete duplicates on the mobile app or on in the web view?
OVERLORD commented 2026-02-04 19:41:16 +03:00
Author
Owner
Copy Link

@nsantini commented on GitHub (Sep 23, 2022):

Ok, I figured out. In the mobile app, the ones with the cloud are those in the server, which came from google photos. The ones with the pic icon are on the phone (higher quality). So. by deleting from the phone those in the server, I get rid of the duplicates and keep the higher quality versions

@nsantini commented on GitHub (Sep 23, 2022): Ok, I figured out. In the mobile app, the ones with the cloud are those in the server, which came from google photos. The ones with the pic icon are on the phone (higher quality). So. by deleting from the phone those in the server, I get rid of the duplicates and keep the higher quality versions
OVERLORD commented 2026-02-04 19:41:27 +03:00
Author
Owner
Copy Link

@zackpollard commented on GitHub (Sep 23, 2022):

Ok, I figured out. In the mobile app, the ones with the cloud are those in the server, which came from google photos. The ones with the pic icon are on the phone (higher quality). So. by deleting from the phone those in the server, I get rid of the duplicates and keep the higher quality versions

Yes I believe that is correct. God speed in manually going through all your photos!

@zackpollard commented on GitHub (Sep 23, 2022): > Ok, I figured out. In the mobile app, the ones with the cloud are those in the server, which came from google photos. The ones with the pic icon are on the phone (higher quality). So. by deleting from the phone those in the server, I get rid of the duplicates and keep the higher quality versions Yes I believe that is correct. God speed in manually going through all your photos!
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
accessibility
changelog:enhancement
changelog:security
changelog:skip
changelog:translation
cli
date-time
dependencies
documentation
external-library
format
good first issue
mobile-beta
mobile-beta
mobile-beta
needs-answer
nice to have
pull-request
Mirrored from GitHub Pull Request
 sharing
tech-debt
📱mobile
🖥️web
🗄️server
🧠machine-learning
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
OVERLORD
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: immich-app/immich#327
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?