Xiaomi 14 ultra photo cannot process for 1.118.2 version #4593

Closed
opened 2026-02-05 10:43:14 +03:00 by OVERLORD · 20 comments
Owner

Originally created by @cmingkit on GitHub (Oct 19, 2024).

The bug

It can't generate the THUMBNAILS, even set the IMMICH_PROCESS_INVALID_IMAGES=true

Here is the log:
[Nest] 7 - 10/19/2024, 11:06:19 AM ERROR [Microservices:JobService] Unable to run job handler (thumbnailGeneration/generate-thumbnails): Error: Input file has corrupt header: VipsJpeg: ./lib/jpegli/decode_marker.cc:417: Invalid ICC chunk order.
[Nest] 7 - 10/19/2024, 11:06:19 AM ERROR [Microservices:JobService] Error: Input file has corrupt header: VipsJpeg: ./lib/jpegli/decode_marker.cc:417: Invalid ICC chunk order.
at Sharp.toBuffer (/usr/src/app/node_modules/sharp/lib/output.js:163:17)
at MediaRepository.decodeImage (/usr/src/app/dist/repositories/media.repository.js:57:68)
at MediaService.generateImageThumbnails (/usr/src/app/dist/services/media.service.js:153:63)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async MediaService.handleGenerateThumbnails (/usr/src/app/dist/services/media.service.js:101:25)
at async /usr/src/app/dist/services/job.service.js:163:36
at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)

Following is sample image from Xiaomi 14 ultra
testing

The OS that Immich Server is running on

Ubuntu 22.04.5 LTS

Version of Immich Server

v1.118.2

Version of Immich Mobile App

v1.118.2

Platform with the issue

  • Server
  • Web
  • Mobile

Your docker-compose.yml content

version: "3.8"

name: immich

services:
  immich-server:
    container_name: immich_server
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    command: [ "start.sh", "immich" ]
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - /etc/localtime:/etc/localtime:ro
    env_file:
      - .env
    depends_on:
      - redis
      - database
      - typesense
    restart: always

  immich-microservices:
    container_name: immich_microservices
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    # extends:
    #   file: hwaccel.yml
    #   service: hwaccel
    command: [ "start.sh", "microservices" ]
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - /etc/localtime:/etc/localtime:ro
    env_file:
      - .env
    depends_on:
      - redis
      - database
      - typesense
    restart: always

  immich-machine-learning:
    container_name: immich_machine_learning
    image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release}
    volumes:
      - model-cache:/cache
    env_file:
      - .env
    restart: always

  immich-web:
    container_name: immich_web
    image: ghcr.io/immich-app/immich-web:${IMMICH_VERSION:-release}
    env_file:
      - .env
    restart: always

  typesense:
    container_name: immich_typesense
    image: typesense/typesense:0.24.1@sha256:9bcff2b829f12074426ca044b56160ca9d777a0c488303469143dd9f8259d4dd
    environment:
      - TYPESENSE_API_KEY=${TYPESENSE_API_KEY}
      - TYPESENSE_DATA_DIR=/data
      # remove this to get debug messages
      - GLOG_minloglevel=1
    volumes:
      - tsdata:/data
    restart: always

  redis:
    container_name: immich_redis
    image: redis:6.2-alpine@sha256:70a7a5b641117670beae0d80658430853896b5ef269ccf00d1827427e3263fa3
    restart: always

  database:
    container_name: immich_postgres
    image: postgres:14-alpine@sha256:28407a9961e76f2d285dc6991e8e48893503cc3836a4755bbc2d40bcc272a441
    env_file:
      - .env
    environment:
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      POSTGRES_USER: ${DB_USERNAME}
      POSTGRES_DB: ${DB_DATABASE_NAME}
    volumes:
      - pgdata:/var/lib/postgresql/data
    restart: always

  immich-proxy:
    container_name: immich_proxy
    image: ghcr.io/immich-app/immich-proxy:${IMMICH_VERSION:-release}
    ports:
      - 2283:8080
    depends_on:
      - immich-server
      - immich-web
    restart: always

volumes:
  pgdata:
  model-cache:
  tsdata:

Your .env content

# You can find documentation for all the supported env variables at https://immich.app/docs/install/environment-variables

# The location where your uploaded files are stored
UPLOAD_LOCATION=/sharefolder/immich/data/

# The Immich version to use. You can pin this to a specific version like "v1.71.0"
IMMICH_VERSION=release

# Connection secrets for postgres and typesense. You should change these to random passwords
TYPESENSE_API_KEY=some-random-text
DB_PASSWORD=postgres

# The values below this line do not need to be changed
###################################################################################
DB_HOSTNAME=immich_postgres
DB_USERNAME=postgres
DB_DATABASE_NAME=immich

REDIS_HOSTNAME=immich_redis

Reproduction steps

  1. try to synchronize photo from
  2. also try to upload in website

Relevant log output

[Nest] 7  - 10/19/2024, 11:06:19 AM   ERROR [Microservices:JobService] Unable to run job handler (thumbnailGeneration/generate-thumbnails): Error: Input file has corrupt header: VipsJpeg: ./lib/jpegli/decode_marker.cc:417: Invalid ICC chunk order.
[Nest] 7  - 10/19/2024, 11:06:19 AM   ERROR [Microservices:JobService] Error: Input file has corrupt header: VipsJpeg: ./lib/jpegli/decode_marker.cc:417: Invalid ICC chunk order.
    at Sharp.toBuffer (/usr/src/app/node_modules/sharp/lib/output.js:163:17)
    at MediaRepository.decodeImage (/usr/src/app/dist/repositories/media.repository.js:57:68)
    at MediaService.generateImageThumbnails (/usr/src/app/dist/services/media.service.js:153:63)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async MediaService.handleGenerateThumbnails (/usr/src/app/dist/services/media.service.js:101:25)
    at async /usr/src/app/dist/services/job.service.js:163:36
    at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
    at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)

Additional information

No response

Originally created by @cmingkit on GitHub (Oct 19, 2024). ### The bug It can't generate the THUMBNAILS, even set the IMMICH_PROCESS_INVALID_IMAGES=true Here is the log: [Nest] 7 - 10/19/2024, 11:06:19 AM ERROR [Microservices:JobService] Unable to run job handler (thumbnailGeneration/generate-thumbnails): Error: Input file has corrupt header: VipsJpeg: ./lib/jpegli/decode_marker.cc:417: Invalid ICC chunk order. [Nest] 7 - 10/19/2024, 11:06:19 AM ERROR [Microservices:JobService] Error: Input file has corrupt header: VipsJpeg: ./lib/jpegli/decode_marker.cc:417: Invalid ICC chunk order. at Sharp.toBuffer (/usr/src/app/node_modules/sharp/lib/output.js:163:17) at MediaRepository.decodeImage (/usr/src/app/dist/repositories/media.repository.js:57:68) at MediaService.generateImageThumbnails (/usr/src/app/dist/services/media.service.js:153:63) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async MediaService.handleGenerateThumbnails (/usr/src/app/dist/services/media.service.js:101:25) at async /usr/src/app/dist/services/job.service.js:163:36 at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28) at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24) Following is sample image from Xiaomi 14 ultra ![testing](https://github.com/user-attachments/assets/15ec65de-3877-4905-a297-8d398e801501) ### The OS that Immich Server is running on Ubuntu 22.04.5 LTS ### Version of Immich Server v1.118.2 ### Version of Immich Mobile App v1.118.2 ### Platform with the issue - [X] Server - [ ] Web - [ ] Mobile ### Your docker-compose.yml content ```YAML version: "3.8" name: immich services: immich-server: container_name: immich_server image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release} command: [ "start.sh", "immich" ] volumes: - ${UPLOAD_LOCATION}:/usr/src/app/upload - /etc/localtime:/etc/localtime:ro env_file: - .env depends_on: - redis - database - typesense restart: always immich-microservices: container_name: immich_microservices image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release} # extends: # file: hwaccel.yml # service: hwaccel command: [ "start.sh", "microservices" ] volumes: - ${UPLOAD_LOCATION}:/usr/src/app/upload - /etc/localtime:/etc/localtime:ro env_file: - .env depends_on: - redis - database - typesense restart: always immich-machine-learning: container_name: immich_machine_learning image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release} volumes: - model-cache:/cache env_file: - .env restart: always immich-web: container_name: immich_web image: ghcr.io/immich-app/immich-web:${IMMICH_VERSION:-release} env_file: - .env restart: always typesense: container_name: immich_typesense image: typesense/typesense:0.24.1@sha256:9bcff2b829f12074426ca044b56160ca9d777a0c488303469143dd9f8259d4dd environment: - TYPESENSE_API_KEY=${TYPESENSE_API_KEY} - TYPESENSE_DATA_DIR=/data # remove this to get debug messages - GLOG_minloglevel=1 volumes: - tsdata:/data restart: always redis: container_name: immich_redis image: redis:6.2-alpine@sha256:70a7a5b641117670beae0d80658430853896b5ef269ccf00d1827427e3263fa3 restart: always database: container_name: immich_postgres image: postgres:14-alpine@sha256:28407a9961e76f2d285dc6991e8e48893503cc3836a4755bbc2d40bcc272a441 env_file: - .env environment: POSTGRES_PASSWORD: ${DB_PASSWORD} POSTGRES_USER: ${DB_USERNAME} POSTGRES_DB: ${DB_DATABASE_NAME} volumes: - pgdata:/var/lib/postgresql/data restart: always immich-proxy: container_name: immich_proxy image: ghcr.io/immich-app/immich-proxy:${IMMICH_VERSION:-release} ports: - 2283:8080 depends_on: - immich-server - immich-web restart: always volumes: pgdata: model-cache: tsdata: ``` ### Your .env content ```Shell # You can find documentation for all the supported env variables at https://immich.app/docs/install/environment-variables # The location where your uploaded files are stored UPLOAD_LOCATION=/sharefolder/immich/data/ # The Immich version to use. You can pin this to a specific version like "v1.71.0" IMMICH_VERSION=release # Connection secrets for postgres and typesense. You should change these to random passwords TYPESENSE_API_KEY=some-random-text DB_PASSWORD=postgres # The values below this line do not need to be changed ################################################################################### DB_HOSTNAME=immich_postgres DB_USERNAME=postgres DB_DATABASE_NAME=immich REDIS_HOSTNAME=immich_redis ``` ### Reproduction steps 1. try to synchronize photo from 2. also try to upload in website ### Relevant log output ```shell [Nest] 7 - 10/19/2024, 11:06:19 AM ERROR [Microservices:JobService] Unable to run job handler (thumbnailGeneration/generate-thumbnails): Error: Input file has corrupt header: VipsJpeg: ./lib/jpegli/decode_marker.cc:417: Invalid ICC chunk order. [Nest] 7 - 10/19/2024, 11:06:19 AM ERROR [Microservices:JobService] Error: Input file has corrupt header: VipsJpeg: ./lib/jpegli/decode_marker.cc:417: Invalid ICC chunk order. at Sharp.toBuffer (/usr/src/app/node_modules/sharp/lib/output.js:163:17) at MediaRepository.decodeImage (/usr/src/app/dist/repositories/media.repository.js:57:68) at MediaService.generateImageThumbnails (/usr/src/app/dist/services/media.service.js:153:63) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async MediaService.handleGenerateThumbnails (/usr/src/app/dist/services/media.service.js:101:25) at async /usr/src/app/dist/services/job.service.js:163:36 at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28) at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24) ``` ### Additional information _No response_
Author
Owner

@cmingkit commented on GitHub (Oct 19, 2024):

Previous version without this issue

@cmingkit commented on GitHub (Oct 19, 2024): Previous version without this issue
Author
Owner

@alextran1502 commented on GitHub (Oct 19, 2024):

Your docker compose file is out of date. Please update it an retry

@alextran1502 commented on GitHub (Oct 19, 2024): Your docker compose file is out of date. Please update it an retry
Author
Owner

@cmingkit commented on GitHub (Oct 19, 2024):

I copied the wrong one, already use the latest one to update
https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml

and would you mind to use my photo to test at your own environment, thanks.

@cmingkit commented on GitHub (Oct 19, 2024): I copied the wrong one, already use the latest one to update https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml and would you mind to use my photo to test at your own environment, thanks.
Author
Owner

@mertalev commented on GitHub (Oct 19, 2024):

Can you clarify if the error is specific to this image, or if no images from this phone can be processed?

@mertalev commented on GitHub (Oct 19, 2024): Can you clarify if the error is specific to this image, or if no images from this phone can be processed?
Author
Owner

@cmingkit commented on GitHub (Oct 19, 2024):

All photos took by this phone can't be processed. I have Fujifilm photos are work and my friend sent me the whatsapp phone no this problem through this phone to upload.

And this problem start from 1.118.2 version.

@cmingkit commented on GitHub (Oct 19, 2024): All photos took by this phone can't be processed. I have Fujifilm photos are work and my friend sent me the whatsapp phone no this problem through this phone to upload. And this problem start from 1.118.2 version.
Author
Owner

@cmingkit commented on GitHub (Oct 19, 2024):

螢幕擷取畫面 2024-10-20 025251

It still work on Oct 15, 2024

螢幕擷取畫面 2024-10-20 025327

above is upload on Oct 16, 2024 and my Xaioma 14 ultra without any camera app or OTA update in these 2 days.

Following is front page of "Photos" page
螢幕擷取畫面 2024-10-20 025349

@cmingkit commented on GitHub (Oct 19, 2024): ![螢幕擷取畫面 2024-10-20 025251](https://github.com/user-attachments/assets/30d009b8-fae5-45f8-bd9a-c46b84f844be) It still work on Oct 15, 2024 ![螢幕擷取畫面 2024-10-20 025327](https://github.com/user-attachments/assets/8bee5974-a7f5-4d3d-9c2b-133d23f26dba) above is upload on Oct 16, 2024 and my Xaioma 14 ultra without any camera app or OTA update in these 2 days. Following is front page of "Photos" page ![螢幕擷取畫面 2024-10-20 025349](https://github.com/user-attachments/assets/c873da12-e75d-4851-9d58-5c172c420e44)
Author
Owner

@cmingkit commented on GitHub (Oct 19, 2024):

seems like my phone's photo is not compatible with Jpegli
Could I restore my immich to v1.118.1?

Better JPEG compression
Immich now uses Jpegli, a new library leveraging the advancements of JPEG XL to shrink JPEG file size at the same (or higher) quality. This change narrows the gap between JPEG and WebP compression considerably, especially at high quality.

@cmingkit commented on GitHub (Oct 19, 2024): seems like my phone's photo is not compatible with Jpegli Could I restore my immich to v1.118.1? Better JPEG compression Immich now uses [Jpegli](https://opensource.googleblog.com/2024/04/introducing-jpegli-new-jpeg-coding-library.html), a new library leveraging the advancements of JPEG XL to shrink JPEG file size at the same (or higher) quality. This change narrows the gap between JPEG and WebP compression considerably, especially at high quality.
Author
Owner

@mertalev commented on GitHub (Oct 19, 2024):

Could you share a sample file from this phone? I'd like to see if there's anything we can do.

In the meantime, is it possibly for you to take images in HEIC format instead of JPEG? The compression and quality should be better as well.

@mertalev commented on GitHub (Oct 19, 2024): Could you share a sample file from this phone? I'd like to see if there's anything we can do. In the meantime, is it possibly for you to take images in HEIC format instead of JPEG? The compression and quality should be better as well.
Author
Owner

@cmingkit commented on GitHub (Oct 19, 2024):

I tried HEIF (not HEIC), it work and can be workaround.

螢幕擷取畫面 2024-10-20 041855

Following is JPEG files from my phone

https://mkcqb.dynv6.net:666/$/nR3Yo

https://mkcqb.dynv6.net:666/$/wwHPb

https://mkcqb.dynv6.net:666/$/d94Op

https://mkcqb.dynv6.net:666/$/wQiib

Thank you for your quick reply

@cmingkit commented on GitHub (Oct 19, 2024): I tried HEIF (not HEIC), it work and can be workaround. ![螢幕擷取畫面 2024-10-20 041855](https://github.com/user-attachments/assets/6fd6a8a6-b3d8-4bd4-bc26-fdb111ae6a4c) Following is JPEG files from my phone https://mkcqb.dynv6.net:666/$/nR3Yo https://mkcqb.dynv6.net:666/$/wwHPb https://mkcqb.dynv6.net:666/$/d94Op https://mkcqb.dynv6.net:666/$/wQiib Thank you for your quick reply
Author
Owner

@CrushedAsian255 commented on GitHub (Oct 20, 2024):

If the issue stems from Jpegli, it should also be reported to the Jpegli repository.

@CrushedAsian255 commented on GitHub (Oct 20, 2024): If the issue stems from Jpegli, it should also be reported to the Jpegli repository.
Author
Owner

@cmingkit commented on GitHub (Oct 20, 2024):

Agree if the root cause from Jpegli, I am also trying to build the Jpegli in my desktop and run the djpegli tool.
However, not sure is it only the Xiaomi phone issue. There is still possible to keep this ticket open to see the overall impact.

@cmingkit commented on GitHub (Oct 20, 2024): Agree if the root cause from Jpegli, I am also trying to build the Jpegli in my desktop and run the djpegli tool. However, not sure is it only the Xiaomi phone issue. There is still possible to keep this ticket open to see the overall impact.
Author
Owner

@jonnyawsom3 commented on GitHub (Oct 20, 2024):

A lot of phones seem to be creating invalid files since they've been trying to add UltraHDR gainmaps to them as seen here https://github.com/libjxl/libjxl/issues/3882

Could be yet another instance, but more testing is needed

@jonnyawsom3 commented on GitHub (Oct 20, 2024): A lot of phones seem to be creating invalid files since they've been trying to add UltraHDR gainmaps to them as seen here https://github.com/libjxl/libjxl/issues/3882 Could be yet another instance, but more testing is needed
Author
Owner

@cmingkit commented on GitHub (Oct 21, 2024):

OK, Could immich consider has option for user not use libjxl in future version? From user propective, I think the previous version is good at this part for handling the image and I am no concern on the compression and performance.

@cmingkit commented on GitHub (Oct 21, 2024): OK, Could immich consider has option for user not use libjxl in future version? From user propective, I think the previous version is good at this part for handling the image and I am no concern on the compression and performance.
Author
Owner

@mertalev commented on GitHub (Oct 21, 2024):

So for reference, the error is thrown on this line in the jpegli decoder. The equivalent section in libjpeg-turbo is here. Now, I'm not sure if this image would return false for marker_is_icc or trip any of the warnings, but even if it does it will just move on. I think a warning is more appropriate than an error here.

Also, I find it odd that JPEGLI_ERROR makes the image processing fail even when libvips is set to not fail on error.

@mertalev commented on GitHub (Oct 21, 2024): So for reference, the error is thrown on [this](https://github.com/libjxl/libjxl/blob/4df1e9eccdf86b8df4c0c7c08f529263906f9c4f/lib/jpegli/decode_marker.cc#L417) line in the jpegli decoder. The equivalent section in libjpeg-turbo is [here](https://github.com/libjpeg-turbo/libjpeg-turbo/blob/a927b489e22dffa309f3f617005c0cebaf8fc00c/src/jdicc.c#L98). Now, I'm not sure if this image would return false for `marker_is_icc` or trip any of the warnings, but even if it does it will just move on. I think a warning is more appropriate than an error here. Also, I find it odd that `JPEGLI_ERROR` makes the image processing fail even when libvips is set to not fail on error.
Author
Owner

@CrushedAsian255 commented on GitHub (Oct 21, 2024):

I think we keep jpegli and if a failure occurs fall back on libjpeg-turbo.

@CrushedAsian255 commented on GitHub (Oct 21, 2024): I think we keep jpegli and if a failure occurs fall back on libjpeg-turbo.
Author
Owner

@mertalev commented on GitHub (Oct 21, 2024):

No, that isn't really an option. We can't just switch between them like that. I think we should see if this can be addressed upstream in jpegli.

@mertalev commented on GitHub (Oct 21, 2024): No, that isn't really an option. We can't just switch between them like that. I think we should see if this can be addressed upstream in jpegli.
Author
Owner

@qcks commented on GitHub (Oct 27, 2024):

众所周知ios的app从商店更新后用户不能降级版本,android的虽然可以手动处理但是一般用户不会。服务端为了稳定性不应该频繁升级!那么二者各跑各的必将乱成一团。好歹也这么大的code行数了,这点小问题不好解决吗?
EN:
It is well known that ios apps can't be downgraded by the user after updating from the store, android ones can be handled manually but the average user won't. The server side should not be upgraded frequently for the sake of stability! Then both of them will be in a mess if they run their own way. At least it's such a big line of code, can't this small problem be solved?

@qcks commented on GitHub (Oct 27, 2024): 众所周知ios的app从商店更新后用户不能降级版本,android的虽然可以手动处理但是一般用户不会。服务端为了稳定性不应该频繁升级!那么二者各跑各的必将乱成一团。好歹也这么大的code行数了,这点小问题不好解决吗? EN: It is well known that ios apps can't be downgraded by the user after updating from the store, android ones can be handled manually but the average user won't. The server side should not be upgraded frequently for the sake of stability! Then both of them will be in a mess if they run their own way. At least it's such a big line of code, can't this small problem be solved?
Author
Owner

@mertalev commented on GitHub (Oct 31, 2024):

I patched jpegli to make this a warning instead of an error. These images should be successfully processed in the next release. That being said, I still recommend using HEIC instead of JPEG.

@mertalev commented on GitHub (Oct 31, 2024): I patched jpegli to make this a warning instead of an error. These images should be successfully processed in the next release. That being said, I still recommend using HEIC instead of JPEG.
Author
Owner

@mertalev commented on GitHub (Nov 7, 2024):

Should be fixed as of 1.120.0.

@mertalev commented on GitHub (Nov 7, 2024): Should be fixed as of 1.120.0.
Author
Owner

@cmingkit commented on GitHub (Nov 7, 2024):

Great, confirmed the issue is gone in 1.120.0 version, thanks @mertalev!

@cmingkit commented on GitHub (Nov 7, 2024): Great, confirmed the issue is gone in 1.120.0 version, thanks @mertalev!
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: immich-app/immich#4593