[BUG] After importing everything through the CLI, the app still tries to upload every file #1123

Closed
opened 2026-02-05 00:33:11 +03:00 by OVERLORD · 1 comment
Owner

Originally created by @gcarrarom on GitHub (Jul 17, 2023).

The bug

I've uploaded all my photos by using the CLI, which worked perfectly fine. Now the issue is that whenever I open the immich app on my phone, it still tries to upload all the images. This results in poor performance of the app and constant usage trying to upload thousands of photos that are already there, only to respond with this error on the server side:

immich/immich-typesense-6bf6bfb8b9-nf8wq[immich-typesense]: I20230717 16:17:25.044348   181 raft_server.h:60] Peer refresh succeeded!
immich/immich-microservices-7b866f9584-lzkv6[immich-microservices]: [Nest] 7  - 07/17/2023, 4:17:25 PM   ERROR [TypesenseRepository] Unable to index documents
immich/immich-postgresql-0[postgresql]: 2023-07-17 16:17:28.242 GMT [13981] ERROR:  duplicate key value violates unique constraint "UQ_userid_checksum"
immich/immich-postgresql-0[postgresql]: 2023-07-17 16:17:28.242 GMT [13981] DETAIL:  Key ("ownerId", checksum)=(cd4afb29-fb65-4511-9827-947542f03fc8, \xe79ba9bbe3568621486219b06b1f5ec9807fb6e9) already exists.
immich/immich-postgresql-0[postgresql]: 2023-07-17 16:17:28.242 GMT [13981] STATEMENT:  INSERT INTO "assets"("id", "deviceAssetId", "ownerId", "deviceId", "type", "originalPath", "resizePath", "webpPath", "thumbhash", "encodedVideoPath", "createdAt", "updatedAt", "fileCreatedAt", "fileModifiedAt", "isFavorite", "isArchived", "isReadOnly", "checksum", "duration", "isVisible", "livePhotoVideoId", "originalFileName", "sidecarPath") VALUES (DEFAULT, $1, $2, $3, $4, $5, $6, $7, $8, $9, DEFAULT, DEFAULT, $10, $11, $12, $13, $14, $15, $16, $17, DEFAULT, $18, $19) RETURNING "id", "webpPath", "encodedVideoPath", "createdAt", "updatedAt", "isFavorite", "isArchived", "isReadOnly", "isVisible"
immich/immich-typesense-6bf6bfb8b9-nf8wq[immich-typesense]: E20230717 16:17:29.044747   165 raft_server.cpp:635] 512 queued writes > healthy write lag of 500
immich/immich-server-7c58fcff6-p9bsc[immich-server]: [Nest] 8  - 07/17/2023, 4:17:30 PM     LOG [CommunicationGateway] Client FpRgXQt-j_9ndPKTAAAB disconnected from Websocket
immich/immich-microservices-7b866f9584-lzkv6[immich-microservices]: [Nest] 7  - 07/17/2023, 4:17:30 PM   ERROR [TypesenseRepository] Unable to index documents
immich/immich-typesense-6bf6bfb8b9-nf8wq[immich-typesense]: I20230717 16:17:35.045292   165 raft_server.cpp:545] Term: 1069, last_index index: 12281, committed_index: 12281, known_applied_index: 12281, applying_index: 0, queued_writes: 512, pending_queue_size: 0, local_sequence: 237094

The OS that Immich Server is running on

Kubernetes - k3s - MicroOS

Version of Immich Server

v1.67.2

Version of Immich Mobile App

1.67.0 build.107

Platform with the issue

  • Server
  • Web
  • Mobile

Your docker-compose.yml content

apiVersion: helm.toolkit.fluxcd.io/v2beta1
kind: HelmRelease
metadata:
  name: immich
  namespace: immich
spec:
  interval: 5m
  chart:
    spec:
      chart: immich
      sourceRef:
        kind: HelmRepository
        name: immich
        namespace: fancywhale
      interval: 1m
  values:

    postgresql:
      enabled: true
    redis:
      enabled: true
    
    typesense:
      enabled: true
      persistence:
        tsdata:
          enabled: true
          existingClaim: typesense-data

    machine-learning:
      enabled: true
      persistence:
        cache:
          enabled: true
          existingClaim: machinelearning-data
    proxy:
      ingress:
        main:
          enabled: true
          ingressClassName: nginx
          annotations:
            # proxy-body-size is set to 0 to remove the body limit on file uploads
            nginx.ingress.kubernetes.io/proxy-body-size: "0"
            cert-manager.io/cluster-issuer: letsencrypt
          hosts:
            - host: $MY_HOSTNAME
              paths:
                - path: "/"
          tls:
            - hosts:
                - $MY_HOSTNAME
              secretName: $MY_HOSTNAME-tls
    image:
      tag: v1.67.2
    immich:
      persistence:
        # Main data store for all photos shared between different components.
        library:
          existingClaim: photos

Your .env content

N/A

Reproduction steps

1. Upload images from the phone using the CLI on a computer.
2. Open the app on the phone and wait for the sync
3. Check that the images are still pending upload.

Additional information

No response

Originally created by @gcarrarom on GitHub (Jul 17, 2023). ### The bug I've uploaded all my photos by using the CLI, which worked perfectly fine. Now the issue is that whenever I open the immich app on my phone, it still tries to upload all the images. This results in poor performance of the app and constant usage trying to upload thousands of photos that are already there, only to respond with this error on the server side: ``` immich/immich-typesense-6bf6bfb8b9-nf8wq[immich-typesense]: I20230717 16:17:25.044348 181 raft_server.h:60] Peer refresh succeeded! immich/immich-microservices-7b866f9584-lzkv6[immich-microservices]: [Nest] 7 - 07/17/2023, 4:17:25 PM ERROR [TypesenseRepository] Unable to index documents immich/immich-postgresql-0[postgresql]: 2023-07-17 16:17:28.242 GMT [13981] ERROR: duplicate key value violates unique constraint "UQ_userid_checksum" immich/immich-postgresql-0[postgresql]: 2023-07-17 16:17:28.242 GMT [13981] DETAIL: Key ("ownerId", checksum)=(cd4afb29-fb65-4511-9827-947542f03fc8, \xe79ba9bbe3568621486219b06b1f5ec9807fb6e9) already exists. immich/immich-postgresql-0[postgresql]: 2023-07-17 16:17:28.242 GMT [13981] STATEMENT: INSERT INTO "assets"("id", "deviceAssetId", "ownerId", "deviceId", "type", "originalPath", "resizePath", "webpPath", "thumbhash", "encodedVideoPath", "createdAt", "updatedAt", "fileCreatedAt", "fileModifiedAt", "isFavorite", "isArchived", "isReadOnly", "checksum", "duration", "isVisible", "livePhotoVideoId", "originalFileName", "sidecarPath") VALUES (DEFAULT, $1, $2, $3, $4, $5, $6, $7, $8, $9, DEFAULT, DEFAULT, $10, $11, $12, $13, $14, $15, $16, $17, DEFAULT, $18, $19) RETURNING "id", "webpPath", "encodedVideoPath", "createdAt", "updatedAt", "isFavorite", "isArchived", "isReadOnly", "isVisible" immich/immich-typesense-6bf6bfb8b9-nf8wq[immich-typesense]: E20230717 16:17:29.044747 165 raft_server.cpp:635] 512 queued writes > healthy write lag of 500 immich/immich-server-7c58fcff6-p9bsc[immich-server]: [Nest] 8 - 07/17/2023, 4:17:30 PM LOG [CommunicationGateway] Client FpRgXQt-j_9ndPKTAAAB disconnected from Websocket immich/immich-microservices-7b866f9584-lzkv6[immich-microservices]: [Nest] 7 - 07/17/2023, 4:17:30 PM ERROR [TypesenseRepository] Unable to index documents immich/immich-typesense-6bf6bfb8b9-nf8wq[immich-typesense]: I20230717 16:17:35.045292 165 raft_server.cpp:545] Term: 1069, last_index index: 12281, committed_index: 12281, known_applied_index: 12281, applying_index: 0, queued_writes: 512, pending_queue_size: 0, local_sequence: 237094 ``` ### The OS that Immich Server is running on Kubernetes - k3s - MicroOS ### Version of Immich Server v1.67.2 ### Version of Immich Mobile App 1.67.0 build.107 ### Platform with the issue - [X] Server - [ ] Web - [X] Mobile ### Your docker-compose.yml content ```YAML apiVersion: helm.toolkit.fluxcd.io/v2beta1 kind: HelmRelease metadata: name: immich namespace: immich spec: interval: 5m chart: spec: chart: immich sourceRef: kind: HelmRepository name: immich namespace: fancywhale interval: 1m values: postgresql: enabled: true redis: enabled: true typesense: enabled: true persistence: tsdata: enabled: true existingClaim: typesense-data machine-learning: enabled: true persistence: cache: enabled: true existingClaim: machinelearning-data proxy: ingress: main: enabled: true ingressClassName: nginx annotations: # proxy-body-size is set to 0 to remove the body limit on file uploads nginx.ingress.kubernetes.io/proxy-body-size: "0" cert-manager.io/cluster-issuer: letsencrypt hosts: - host: $MY_HOSTNAME paths: - path: "/" tls: - hosts: - $MY_HOSTNAME secretName: $MY_HOSTNAME-tls image: tag: v1.67.2 immich: persistence: # Main data store for all photos shared between different components. library: existingClaim: photos ``` ### Your .env content ```Shell N/A ``` ### Reproduction steps ```bash 1. Upload images from the phone using the CLI on a computer. 2. Open the app on the phone and wait for the sync 3. Check that the images are still pending upload. ``` ### Additional information _No response_
Author
Owner

@alextran1502 commented on GitHub (Jul 17, 2023):

We haven't implemented the use of client-side hashing yet on the mobile app, so it will still upload those assets and have the server check for the hash and then reject it. Then those files will be blacklisted and won't be uploaded again. Please follow this for more info

https://github.com/immich-app/immich/issues/2567

@alextran1502 commented on GitHub (Jul 17, 2023): We haven't implemented the use of client-side hashing yet on the mobile app, so it will still upload those assets and have the server check for the hash and then reject it. Then those files will be blacklisted and won't be uploaded again. Please follow this for more info https://github.com/immich-app/immich/issues/2567
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: immich-app/immich#1123