immich-app/immich
1
0
Fork 0
mirror of https://github.com/immich-app/immich.git synced 2026-07-25 14:00:45 +03:00
Code Issues 783 Packages Projects Releases 22 Wiki Activity

Sony RAW(.arw) file thumbnail generation failed #3532

New Issue
Closed
opened 2026-02-05 08:51:40 +03:00 by OVERLORD · 7 comments
OVERLORD commented 2026-02-05 08:51:40 +03:00
Owner
Copy Link

Originally created by @kchen0x on GitHub (Jun 15, 2024).

The bug

upload SONY RAW (.arw) file as external library photos but can't not generate thumbnail. Details can be seen at related logs.

image

The OS that Immich Server is running on

Ubuntu 20.04.6 LTS

Version of Immich Server

v1.106.4

Version of Immich Mobile App

v1.106.3

Platform with the issue

  • Server
  • Web
  • Mobile

Your docker-compose.yml content

version: "3.8"

services:
  immich-server:
    container_name: immich_server
    image: ghcr.io/immich-app/immich-server:release
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - /etc/localtime:/etc/localtime:ro
      - /data02/Pictures:/mnt/media/pictures:ro
    env_file:
      - .env
    ports:
      - 2283:3001
    depends_on:
      - redis
      - database
    restart: always

  immich-machine-learning:
    container_name: immich_machine_learning
    image: ghcr.io/immich-app/immich-machine-learning:release
    volumes:
      - model-cache:/cache
    env_file:
      - .env
    restart: always

  redis:
    container_name: immich_redis
    image: redis:6.2
    healthcheck:
      test: redis-cli ping || exit 1
    restart: always
  database:
    container_name: immich_postgres
    image: tensorchord/pgvecto-rs:pg14-v0.2.0
    env_file:
      - .env
    environment:
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      POSTGRES_USER: ${DB_USERNAME}
      POSTGRES_DB: ${DB_DATABASE_NAME}
      POSTGRES_INITDB_ARGS: '--data-checksums'
      PG_DATA: /var/lib/postgresql/data
    volumes:
      - pgdata:/var/lib/postgresql/data
    healthcheck:
      test: pg_isready --dbname='${DB_DATABASE_NAME}' || exit 1; Chksum="$$(psql --dbname='${DB_DATABASE_NAME}' --username='${DB_USERNAME}' --tuples-only --no-align --command='SELECT COALESCE(SUM(checksum_failures), 0) FROM pg_stat_database')"; echo "checksum failure count is $$Chksum"; [ "$$Chksum" = '0' ] || exit 1
      interval: 5m
      start_interval: 30s
      start_period: 5m
    command: ["postgres", "-c" ,"shared_preload_libraries=vectors.so", "-c", 'search_path="$$user", public, vectors', "-c", "logging_collector=on", "-c", "max_wal_size=2GB", "-c", "shared_buffers=512MB", "-c", "wal_compression=on"]
    restart: always

volumes:
  pgdata:
  model-cache:

Your .env content

# You can find documentation for all the supported env variables at https://immich.app/docs/install/environment-variables

# The location where your uploaded files are stored
UPLOAD_LOCATION=/data02/immich

# The Immich version to use. You can pin this to a specific version like "v1.71.0"
IMMICH_VERSION=release

# Connection secret for postgres. You should change it to a random password
DB_PASSWORD=postgres

# The values below this line do not need to be changed
###################################################################################
DB_HOSTNAME=immich_postgres
DB_USERNAME=postgres
DB_DATABASE_NAME=immich

REDIS_HOSTNAME=immich_redis

Reproduction steps

1. upload .arw file to `picture` as external library
2. Scan New Library Files
3. GENERATE THUMBNAILS of MISSING thumbnail

Relevant log output

[Nest] 7  - 06/15/2024, 12:51:52 PM   ERROR [Microservices:JobService] Unable to run job handler (thumbnailGeneration/generate-preview): Error: Input file has corrupt header: magickload: Magick: Unsupported file format or not RAW file `/mnt/media/pictures/DSC00887.ARW' @ error/dng.c/ReadDNGImage/555 (null)
[Nest] 7  - 06/15/2024, 12:51:52 PM   ERROR [Microservices:JobService] Error: Input file has corrupt header: magickload: Magick: Unsupported file format or not RAW file `/mnt/media/pictures/DSC00887.ARW' @ error/dng.c/ReadDNGImage/555 (null)
    at Sharp.toFile (/usr/src/app/node_modules/sharp/lib/output.js:89:19)
    at MediaRepository.generateThumbnail (/usr/src/app/dist/repositories/media.repository.js:69:14)
    at MediaService.generateThumbnail (/usr/src/app/dist/services/media.service.js:158:48)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async MediaService.handleGeneratePreview (/usr/src/app/dist/services/media.service.js:135:29)
    at async /usr/src/app/dist/services/job.service.js:148:36
    at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
    at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)
[Nest] 7  - 06/15/2024, 12:51:52 PM   ERROR [Microservices:JobService] Object:
{
  "id": "94033a9f-371c-4adb-8e6c-b99c6648866c"
}

Additional information

No response

Originally created by @kchen0x on GitHub (Jun 15, 2024). ### The bug upload SONY RAW (.arw) file as external library photos but can't not generate thumbnail. Details can be seen at related logs. <img width="1137" alt="image" src="https://github.com/immich-app/immich/assets/9739009/625e2ab3-c30e-4966-955b-d31c3fc16831"> ### The OS that Immich Server is running on Ubuntu 20.04.6 LTS ### Version of Immich Server v1.106.4 ### Version of Immich Mobile App v1.106.3 ### Platform with the issue - [X] Server - [ ] Web - [ ] Mobile ### Your docker-compose.yml content ```YAML version: "3.8" services: immich-server: container_name: immich_server image: ghcr.io/immich-app/immich-server:release volumes: - ${UPLOAD_LOCATION}:/usr/src/app/upload - /etc/localtime:/etc/localtime:ro - /data02/Pictures:/mnt/media/pictures:ro env_file: - .env ports: - 2283:3001 depends_on: - redis - database restart: always immich-machine-learning: container_name: immich_machine_learning image: ghcr.io/immich-app/immich-machine-learning:release volumes: - model-cache:/cache env_file: - .env restart: always redis: container_name: immich_redis image: redis:6.2 healthcheck: test: redis-cli ping || exit 1 restart: always database: container_name: immich_postgres image: tensorchord/pgvecto-rs:pg14-v0.2.0 env_file: - .env environment: POSTGRES_PASSWORD: ${DB_PASSWORD} POSTGRES_USER: ${DB_USERNAME} POSTGRES_DB: ${DB_DATABASE_NAME} POSTGRES_INITDB_ARGS: '--data-checksums' PG_DATA: /var/lib/postgresql/data volumes: - pgdata:/var/lib/postgresql/data healthcheck: test: pg_isready --dbname='${DB_DATABASE_NAME}' || exit 1; Chksum="$$(psql --dbname='${DB_DATABASE_NAME}' --username='${DB_USERNAME}' --tuples-only --no-align --command='SELECT COALESCE(SUM(checksum_failures), 0) FROM pg_stat_database')"; echo "checksum failure count is $$Chksum"; [ "$$Chksum" = '0' ] || exit 1 interval: 5m start_interval: 30s start_period: 5m command: ["postgres", "-c" ,"shared_preload_libraries=vectors.so", "-c", 'search_path="$$user", public, vectors', "-c", "logging_collector=on", "-c", "max_wal_size=2GB", "-c", "shared_buffers=512MB", "-c", "wal_compression=on"] restart: always volumes: pgdata: model-cache: ``` ### Your .env content ```Shell # You can find documentation for all the supported env variables at https://immich.app/docs/install/environment-variables # The location where your uploaded files are stored UPLOAD_LOCATION=/data02/immich # The Immich version to use. You can pin this to a specific version like "v1.71.0" IMMICH_VERSION=release # Connection secret for postgres. You should change it to a random password DB_PASSWORD=postgres # The values below this line do not need to be changed ################################################################################### DB_HOSTNAME=immich_postgres DB_USERNAME=postgres DB_DATABASE_NAME=immich REDIS_HOSTNAME=immich_redis ``` ### Reproduction steps ```bash 1. upload .arw file to `picture` as external library 2. Scan New Library Files 3. GENERATE THUMBNAILS of MISSING thumbnail ``` ### Relevant log output ```shell [Nest] 7 - 06/15/2024, 12:51:52 PM ERROR [Microservices:JobService] Unable to run job handler (thumbnailGeneration/generate-preview): Error: Input file has corrupt header: magickload: Magick: Unsupported file format or not RAW file `/mnt/media/pictures/DSC00887.ARW' @ error/dng.c/ReadDNGImage/555 (null) [Nest] 7 - 06/15/2024, 12:51:52 PM ERROR [Microservices:JobService] Error: Input file has corrupt header: magickload: Magick: Unsupported file format or not RAW file `/mnt/media/pictures/DSC00887.ARW' @ error/dng.c/ReadDNGImage/555 (null) at Sharp.toFile (/usr/src/app/node_modules/sharp/lib/output.js:89:19) at MediaRepository.generateThumbnail (/usr/src/app/dist/repositories/media.repository.js:69:14) at MediaService.generateThumbnail (/usr/src/app/dist/services/media.service.js:158:48) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async MediaService.handleGeneratePreview (/usr/src/app/dist/services/media.service.js:135:29) at async /usr/src/app/dist/services/job.service.js:148:36 at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28) at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24) [Nest] 7 - 06/15/2024, 12:51:52 PM ERROR [Microservices:JobService] Object: { "id": "94033a9f-371c-4adb-8e6c-b99c6648866c" } ``` ### Additional information _No response_
OVERLORD commented 2026-02-05 08:51:47 +03:00
Author
Owner
Copy Link

@Rubeast commented on GitHub (Jun 16, 2024):

I have a similar problem since the last update. But it seems to be random. I can take 2 pictures. One is uploaded with thumbnail generation without problem. The other gives a broken thumbnail and this log:

[Nest] 6  - 06/16/2024, 7:38:33 PM   ERROR [Microservices:JobService] Unable to run job handler (thumbnailGeneration/generate-preview): Error: Input file contains unsupported image format
[Nest] 6  - 06/16/2024, 7:38:33 PM   ERROR [Microservices:JobService] Error: Input file contains unsupported image format
    at Sharp.toFile (/usr/src/app/node_modules/sharp/lib/output.js:89:19)
    at MediaRepository.generateThumbnail (/usr/src/app/dist/repositories/media.repository.js:69:14)
    at MediaService.generateThumbnail (/usr/src/app/dist/services/media.service.js:158:48)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async MediaService.handleGeneratePreview (/usr/src/app/dist/services/media.service.js:135:29)
    at async /usr/src/app/dist/services/job.service.js:148:36
    at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
    at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)
[Nest] 6  - 06/16/2024, 7:38:33 PM   ERROR [Microservices:JobService] Object:
{
  "id": "4e65ef49-c5e3-4e03-b9a5-a3cea96c9b29",
  "source": "upload"
@Rubeast commented on GitHub (Jun 16, 2024): I have a similar problem since the last update. But it seems to be random. I can take 2 pictures. One is uploaded with thumbnail generation without problem. The other gives a broken thumbnail and this log: ``` [Nest] 6 - 06/16/2024, 7:38:33 PM ERROR [Microservices:JobService] Unable to run job handler (thumbnailGeneration/generate-preview): Error: Input file contains unsupported image format [Nest] 6 - 06/16/2024, 7:38:33 PM ERROR [Microservices:JobService] Error: Input file contains unsupported image format at Sharp.toFile (/usr/src/app/node_modules/sharp/lib/output.js:89:19) at MediaRepository.generateThumbnail (/usr/src/app/dist/repositories/media.repository.js:69:14) at MediaService.generateThumbnail (/usr/src/app/dist/services/media.service.js:158:48) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async MediaService.handleGeneratePreview (/usr/src/app/dist/services/media.service.js:135:29) at async /usr/src/app/dist/services/job.service.js:148:36 at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28) at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24) [Nest] 6 - 06/16/2024, 7:38:33 PM ERROR [Microservices:JobService] Object: { "id": "4e65ef49-c5e3-4e03-b9a5-a3cea96c9b29", "source": "upload" ```
OVERLORD commented 2026-02-05 08:51:50 +03:00
Author
Owner
Copy Link

@mertalev commented on GitHub (Jun 16, 2024):

Can you share a sample image so I can reproduce the issue?

@mertalev commented on GitHub (Jun 16, 2024): Can you share a sample image so I can reproduce the issue?
OVERLORD commented 2026-02-05 08:51:57 +03:00
Author
Owner
Copy Link

@mertalev commented on GitHub (Jun 16, 2024):

Possibly related issue in LibRaw. We should really update to their latest.

@mertalev commented on GitHub (Jun 16, 2024): Possibly related [issue](https://github.com/LibRaw/LibRaw/issues/653) in LibRaw. We should really update to their latest.
OVERLORD commented 2026-02-05 08:52:00 +03:00
Author
Owner
Copy Link

@kchen0x commented on GitHub (Jun 17, 2024):

Can you share a sample image so I can reproduce the issue?

Sure: DSC00965.zip

This photo can be previewed by macOS well:
CleanShot 2024-06-17 at 14 19 50@2x

@kchen0x commented on GitHub (Jun 17, 2024): > Can you share a sample image so I can reproduce the issue? Sure: [DSC00965.zip](https://github.com/user-attachments/files/15862586/DSC00965.zip) This photo can be previewed by macOS well: ![CleanShot 2024-06-17 at 14 19 50@2x](https://github.com/immich-app/immich/assets/9739009/897ffbf8-f250-4ab6-b050-bc54e89e03c5)
OVERLORD commented 2026-02-05 08:52:04 +03:00
Author
Owner
Copy Link

@hugohabicht01 commented on GitHub (Jun 17, 2024):

i've noticed something somewhat similar, raw files (.dng in my case, shot on google pixel) don't have a thumbnail (as shown in the images above), until i click on it. After that they have thumbnails and it shows in the server logs that webp versions were created
(Successfully generated WEBP image thumbnail for asset 923d0475-5ed6-41db-9561-fd1f5cc0a4f3)
So the issue might not be that thumbnails don't get created at all, but just that it takes some time or a specific request

@hugohabicht01 commented on GitHub (Jun 17, 2024): i've noticed something somewhat similar, raw files (.dng in my case, shot on google pixel) don't have a thumbnail (as shown in the images above), until i click on it. After that they have thumbnails and it shows in the server logs that webp versions were created (`Successfully generated WEBP image thumbnail for asset 923d0475-5ed6-41db-9561-fd1f5cc0a4f3`) So the issue might not be that thumbnails don't get created at all, but just that it takes some time or a specific request
OVERLORD commented 2026-02-05 08:52:05 +03:00
Author
Owner
Copy Link

@mertalev commented on GitHub (Jun 19, 2024):

Fixed via #10457

@mertalev commented on GitHub (Jun 19, 2024): Fixed via #10457
OVERLORD commented 2026-02-05 08:52:08 +03:00
Author
Owner
Copy Link

@devtechk commented on GitHub (Oct 25, 2024):

Hello, since 1.118.2 I'm faceing the same issue (not thumbnail for some image file) but with the new iphone 16 pro max photo format. I'm using .dng file pro raw and this is the message. I think that immich is not fully compatbile with newest and professional photo format, I think that the target user use only basic format and not too much for professional one.

[Nest] 7 - 10/16/2024, 9:50:35 PM ERROR [Microservices:JobService] Unable to run job handler (thumbnailGeneration/generate-thumbnails): Error: Input file has corrupt header: magickload: Magick: Unsupported file format or not RAW file upload/library/admin/2024/2024-10-16/IMG_8904.DNG' @ error/dng.c/ReadDNGImage/504 (null)
[Nest] 7 - 10/16/2024, 9:50:35 PM ERROR [Microservices:JobService] Error: Input file has corrupt header: magickload: Magick: Unsupported file format or not RAW file upload/library/admin/2024/2024-10-16/IMG_8904.DNG' @ error/dng.c/ReadDNGImage/504 (null)
at Sharp.toBuffer (/usr/src/app/node_modules/sharp/lib/output.js:163:17)
at MediaRepository.decodeImage (/usr/src/app/dist/repositories/media.repository.js:57:68)
at MediaService.generateImageThumbnails (/usr/src/app/dist/services/media.service.js:153:63)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async MediaService.handleGenerateThumbnails (/usr/src/app/dist/services/media.service.js:101:25)
at async /usr/src/app/dist/services/job.service.js:163:36
at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)
[Nest] 7 - 10/16/2024, 9:50:35 PM ERROR [Microservices:JobService] Object:
{
"id": "b3b7d525-6e7d-46fd-8b21-9e03f02ad63b",
"source": "upload"
}

Screenshot 2024-10-24 153209

@devtechk commented on GitHub (Oct 25, 2024): Hello, since 1.118.2 I'm faceing the same issue (not thumbnail for some image file) but with the new iphone 16 pro max photo format. I'm using .dng file pro raw and this is the message. I think that immich is not fully compatbile with newest and professional photo format, I think that the target user use only basic format and not too much for professional one. [Nest] 7 - 10/16/2024, 9:50:35 PM ERROR [Microservices:JobService] Unable to run job handler (thumbnailGeneration/generate-thumbnails): Error: Input file has corrupt header: magickload: Magick: Unsupported file format or not RAW file upload/library/admin/2024/2024-10-16/IMG_8904.DNG' @ error/dng.c/ReadDNGImage/504 (null) [Nest] 7 - 10/16/2024, 9:50:35 PM ERROR [Microservices:JobService] Error: Input file has corrupt header: magickload: Magick: Unsupported file format or not RAW file upload/library/admin/2024/2024-10-16/IMG_8904.DNG' @ error/dng.c/ReadDNGImage/504 (null) at Sharp.toBuffer (/usr/src/app/node_modules/sharp/lib/output.js:163:17) at MediaRepository.decodeImage (/usr/src/app/dist/repositories/media.repository.js:57:68) at MediaService.generateImageThumbnails (/usr/src/app/dist/services/media.service.js:153:63) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async MediaService.handleGenerateThumbnails (/usr/src/app/dist/services/media.service.js:101:25) at async /usr/src/app/dist/services/job.service.js:163:36 at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28) at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24) [Nest] 7 - 10/16/2024, 9:50:35 PM ERROR [Microservices:JobService] Object: { "id": "b3b7d525-6e7d-46fd-8b21-9e03f02ad63b", "source": "upload" } ![Screenshot 2024-10-24 153209](https://github.com/user-attachments/assets/d238f03c-595d-464a-9b0a-dcf9c587cb4b)
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
accessibility
changelog:enhancement
changelog:security
changelog:skip
changelog:translation
cli
date-time
dependencies
documentation
external-library
format
good first issue
mobile-beta
mobile-beta
mobile-beta
needs-answer
nice to have
pull-request
Mirrored from GitHub Pull Request
 sharing
tech-debt
📱mobile
🖥️web
🗄️server
🧠machine-learning
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
OVERLORD
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: immich-app/immich#3532
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?