immich-app/immich
1
0
Fork 0
mirror of https://github.com/immich-app/immich.git synced 2026-07-25 14:00:45 +03:00
Code Issues 783 Packages Projects Releases 22 Wiki Activity

External library scanning (Scan New Library Files) aborts upon Access Denied #4346

New Issue
Open
opened 2026-02-05 10:11:39 +03:00 by OVERLORD · 11 comments
OVERLORD commented 2026-02-05 10:11:39 +03:00
Owner
Copy Link

Originally created by @psla on GitHub (Sep 16, 2024).

Originally assigned to: @etnoy on GitHub.

The bug

  1. Restrict access to some folders in the external library
  2. Go to "external libraries"
  3. Call "Scan New Library Files"
  4. Observe error
[Nest] 7  - 09/15/2024, 10:27:38 PM   ERROR [Microservices:JobService] Unable to run job handler (library/library-refresh): Error: EACCES: permission denied, scandir '/mnt/media/Fotografie/102SSCAM'
[Nest] 7  - 09/15/2024, 10:27:38 PM   ERROR [Microservices:JobService] Error: EACCES: permission denied, scandir '/mnt/media/Fotografie/102SSCAM'
[Nest] 7  - 09/15/2024, 10:27:38 PM   ERROR [Microservices:JobService] Object:
{
  "id": "27af95c1-57fc-4885-9b55-f78a9bc36276",
  "refreshModifiedFiles": false,
  "refreshAllFiles": false
}

Expected behavior: scanning of the library should continue

I understand that generally you probably want to fail upon errors, but this seems a valid use case (to me). Some photographs/content may be more sensitive, and may not be shared with others (including immich). It would be nice if Immich gracesfully fell back access denied errors.

The OS that Immich Server is running on

Debian Linux #redacted# 6.1.0-23-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.99-1 (2024-07-15) x86_64 GNU/Linux

Version of Immich Server

1.115.0

Version of Immich Mobile App

n/a

Platform with the issue

  • Server
  • Web
  • Mobile

Your docker-compose.yml content

version: "3.3"

services:
  immich-server:
    container_name: immich_server
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - /mnt/nas/public/wspolne/Fotografie:/mnt/media/Fotografie:ro
    env_file:
      - .env
    ports:
      - 2283:3001
    depends_on:
      - redis
      - database
    restart: always

  immich-machine-learning:
    container_name: immich_machine_learning
    image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release}
    volumes:
      - model-cache:/cache
    env_file:
      - .env
    restart: always

  redis:
    container_name: immich_redis
    image: redis:6.2-alpine@sha256:70a7a5b641117670beae0d80658430853896b5ef269ccf00d1827427e3263fa3
    restart: always

  database:
    container_name: immich_postgres
    image: tensorchord/pgvecto-rs:pg14-v0.2.0@sha256:90724186f0a3517cf6914295b5ab410db9ce23190a2d9d0b9dd6463e3fa298f0
    env_file:
      - .env
    environment:
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      POSTGRES_USER: ${DB_USERNAME}
      POSTGRES_DB: ${DB_DATABASE_NAME}
      PG_DATA: /var/lib/postgresql/data
    volumes:
      - pgdata:/var/lib/postgresql/data
    restart: always
  backup:
    container_name: immich_db_dumper
    image: prodrigestivill/postgres-backup-local
    env_file:
      - .env
    environment:
      POSTGRES_HOST: database
      POSTGRES_DB: ${DB_DATABASE_NAME}
      POSTGRES_USER: ${DB_USERNAME}
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      SCHEDULE: "@daily"
      BACKUP_NUM_KEEP: 2
      BACKUP_DIR: /db_dumps
    volumes:
      - /mnt/newhome/unprivileged-docker/immich/db_dumps:/db_dumps
    depends_on:
      - database

volumes:
  pgdata:
  model-cache:

Your .env content

DB_HOSTNAME=#redacted#
DB_USERNAME=#redacted#
DB_PASSWORD=#redacted#
DB_DATABASE_NAME=immich



REDIS_HOSTNAME=immich_redis






UPLOAD_LOCATION=/mnt/nas/srv/immich/pictures


TYPESENSE_API_KEY=#




PUBLIC_LOGIN_PAGE_MESSAGE=


IMMICH_WEB_URL=http://immich-web:3000
IMMICH_SERVER_URL=http://immich-server:3001
IMMICH_MACHINE_LEARNING_URL=http://immich-machine-learning:3003




IMMICH_API_URL_EXTERNAL=#redacted#

Reproduction steps

  1. Restrict access to some folders in the external library
  2. Go to "external libraries"
  3. Call "Scan New Library Files"
  4. Observe error

Relevant log output

[Nest] 7  - 09/15/2024, 10:27:38 PM   ERROR [Microservices:JobService] Unable to run job handler (library/library-refresh): Error: EACCES: permission denied, scandir '/mnt/media/Fotografie/102SSCAM'
[Nest] 7  - 09/15/2024, 10:27:38 PM   ERROR [Microservices:JobService] Error: EACCES: permission denied, scandir '/mnt/media/Fotografie/102SSCAM'
[Nest] 7  - 09/15/2024, 10:27:38 PM   ERROR [Microservices:JobService] Object:
{
  "id": "27af95c1-57fc-4885-9b55-f78a9bc36276",
  "refreshModifiedFiles": false,
  "refreshAllFiles": false
}



### Additional information

_No response_
Originally created by @psla on GitHub (Sep 16, 2024). Originally assigned to: @etnoy on GitHub. ### The bug 0) Restrict access to some folders in the external library 1) Go to "external libraries" 2) Call "Scan New Library Files" 3) Observe error ``` [Nest] 7 - 09/15/2024, 10:27:38 PM ERROR [Microservices:JobService] Unable to run job handler (library/library-refresh): Error: EACCES: permission denied, scandir '/mnt/media/Fotografie/102SSCAM' [Nest] 7 - 09/15/2024, 10:27:38 PM ERROR [Microservices:JobService] Error: EACCES: permission denied, scandir '/mnt/media/Fotografie/102SSCAM' [Nest] 7 - 09/15/2024, 10:27:38 PM ERROR [Microservices:JobService] Object: { "id": "27af95c1-57fc-4885-9b55-f78a9bc36276", "refreshModifiedFiles": false, "refreshAllFiles": false } ``` Expected behavior: scanning of the library should continue I understand that generally you probably want to fail upon errors, but this seems a valid use case (to me). Some photographs/content may be more sensitive, and may not be shared with others (including immich). It would be nice if Immich gracesfully fell back access denied errors. ### The OS that Immich Server is running on Debian Linux #redacted# 6.1.0-23-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.99-1 (2024-07-15) x86_64 GNU/Linux ### Version of Immich Server 1.115.0 ### Version of Immich Mobile App n/a ### Platform with the issue - [ ] Server - [ ] Web - [ ] Mobile ### Your docker-compose.yml content ```YAML version: "3.3" services: immich-server: container_name: immich_server image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release} volumes: - ${UPLOAD_LOCATION}:/usr/src/app/upload - /mnt/nas/public/wspolne/Fotografie:/mnt/media/Fotografie:ro env_file: - .env ports: - 2283:3001 depends_on: - redis - database restart: always immich-machine-learning: container_name: immich_machine_learning image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release} volumes: - model-cache:/cache env_file: - .env restart: always redis: container_name: immich_redis image: redis:6.2-alpine@sha256:70a7a5b641117670beae0d80658430853896b5ef269ccf00d1827427e3263fa3 restart: always database: container_name: immich_postgres image: tensorchord/pgvecto-rs:pg14-v0.2.0@sha256:90724186f0a3517cf6914295b5ab410db9ce23190a2d9d0b9dd6463e3fa298f0 env_file: - .env environment: POSTGRES_PASSWORD: ${DB_PASSWORD} POSTGRES_USER: ${DB_USERNAME} POSTGRES_DB: ${DB_DATABASE_NAME} PG_DATA: /var/lib/postgresql/data volumes: - pgdata:/var/lib/postgresql/data restart: always backup: container_name: immich_db_dumper image: prodrigestivill/postgres-backup-local env_file: - .env environment: POSTGRES_HOST: database POSTGRES_DB: ${DB_DATABASE_NAME} POSTGRES_USER: ${DB_USERNAME} POSTGRES_PASSWORD: ${DB_PASSWORD} SCHEDULE: "@daily" BACKUP_NUM_KEEP: 2 BACKUP_DIR: /db_dumps volumes: - /mnt/newhome/unprivileged-docker/immich/db_dumps:/db_dumps depends_on: - database volumes: pgdata: model-cache: ``` ### Your .env content ```Shell DB_HOSTNAME=#redacted# DB_USERNAME=#redacted# DB_PASSWORD=#redacted# DB_DATABASE_NAME=immich REDIS_HOSTNAME=immich_redis UPLOAD_LOCATION=/mnt/nas/srv/immich/pictures TYPESENSE_API_KEY=# PUBLIC_LOGIN_PAGE_MESSAGE= IMMICH_WEB_URL=http://immich-web:3000 IMMICH_SERVER_URL=http://immich-server:3001 IMMICH_MACHINE_LEARNING_URL=http://immich-machine-learning:3003 IMMICH_API_URL_EXTERNAL=#redacted# ``` ### Reproduction steps 0) Restrict access to some folders in the external library 1) Go to "external libraries" 2) Call "Scan New Library Files" 3) Observe error ### Relevant log output ```shell [Nest] 7 - 09/15/2024, 10:27:38 PM ERROR [Microservices:JobService] Unable to run job handler (library/library-refresh): Error: EACCES: permission denied, scandir '/mnt/media/Fotografie/102SSCAM' [Nest] 7 - 09/15/2024, 10:27:38 PM ERROR [Microservices:JobService] Error: EACCES: permission denied, scandir '/mnt/media/Fotografie/102SSCAM' [Nest] 7 - 09/15/2024, 10:27:38 PM ERROR [Microservices:JobService] Object: { "id": "27af95c1-57fc-4885-9b55-f78a9bc36276", "refreshModifiedFiles": false, "refreshAllFiles": false } ``` ``` ### Additional information _No response_
OVERLORD added the 🗄️serverexternal-library labels 2026-02-05 10:11:39 +03:00
OVERLORD commented 2026-02-05 10:11:45 +03:00
Author
Owner
Copy Link

@alexantao commented on GitHub (Dec 16, 2024):

I don't know if I have the same issue.
I've just installed immich (v1.122.3) on a small box I have and mounted a NAS to a local directory as RO, since I want to make shure nothing will me modified on this disk.

Configured my external library, but no photos are found.
Saw on the logs the same access denied errors. But it's odd, because it does has acess, just that it's RO.

[Nest] 7 - 12/16/2024, 11:39:10 AM LOG [Microservices:LibraryService] Refreshing library ed72481e-f76d-437e-acb2-bb535f199c72 for new assets [Nest] 7 - 12/16/2024, 11:39:10 AM LOG [Microservices:LibraryService] Scanning library ed72481e-f76d-437e-acb2-bb535f199c72 for removed assets [Nest] 7 - 12/16/2024, 11:39:10 AM ERROR [Microservices:JobService] Unable to run job handler (library/library-queue-sync-files): Error: EACCES: permission denied, scandir '/FOTOS/2004/01 - Jan' [Nest] 7 - 12/16/2024, 11:39:10 AM ERROR [Microservices:JobService] Error: EACCES: permission denied, scandir '/FOTOS/2004/01 - Jan' [Nest] 7 - 12/16/2024, 11:39:10 AM ERROR [Microservices:JobService] Object:
And I tested entering bash from docker, accessed the library can also cat a JPG file inside it. Access was OK.
Can it be the same issue ?

@alexantao commented on GitHub (Dec 16, 2024): I don't know if I have the same issue. I've just installed immich (v1.122.3) on a small box I have and mounted a NAS to a local directory as RO, since I want to make shure nothing will me modified on this disk. Configured my external library, but no photos are found. Saw on the logs the same access denied errors. But it's odd, because it does has acess, just that it's RO. `[Nest] 7 - 12/16/2024, 11:39:10 AM LOG [Microservices:LibraryService] Refreshing library ed72481e-f76d-437e-acb2-bb535f199c72 for new assets [Nest] 7 - 12/16/2024, 11:39:10 AM LOG [Microservices:LibraryService] Scanning library ed72481e-f76d-437e-acb2-bb535f199c72 for removed assets [Nest] 7 - 12/16/2024, 11:39:10 AM ERROR [Microservices:JobService] Unable to run job handler (library/library-queue-sync-files): Error: EACCES: permission denied, scandir '/FOTOS/2004/01 - Jan' [Nest] 7 - 12/16/2024, 11:39:10 AM ERROR [Microservices:JobService] Error: EACCES: permission denied, scandir '/FOTOS/2004/01 - Jan' [Nest] 7 - 12/16/2024, 11:39:10 AM ERROR [Microservices:JobService] Object: ` And I tested entering bash from docker, accessed the library can also cat a JPG file inside it. Access was OK. Can it be the same issue ?
OVERLORD commented 2026-02-05 10:11:48 +03:00
Author
Owner
Copy Link

@bo0tzz commented on GitHub (Feb 2, 2025):

Per this comment from @zackpollard, do we want to close this issue?

@bo0tzz commented on GitHub (Feb 2, 2025): Per [this comment](https://github.com/immich-app/immich/pull/13306#issuecomment-2473087839) from @zackpollard, do we want to close this issue?
OVERLORD commented 2026-02-05 10:11:50 +03:00
Author
Owner
Copy Link

@etnoy commented on GitHub (Feb 2, 2025):

I'd say keep this open until we can get fast-glob to fix this one

@etnoy commented on GitHub (Feb 2, 2025): I'd say keep this open until we can get fast-glob to fix this one
OVERLORD commented 2026-02-05 10:11:54 +03:00
Author
Owner
Copy Link

@biappi commented on GitHub (Sep 21, 2025):

After reading this comment i'm not sure if the following falls in the same umbrella as this issue, sorry if otherwise. But after working around EACCESS, i had the same issue with ENOTDIR as follows:

[Nest] 7  - 09/21/2025, 1:20:07 PM   ERROR [Microservices:{"id":"83d57784-995d-4206-bb67-3ea95cb5e5e3"}] Unable to run job handler (LibrarySyncFilesQueueAll): Error: ENOTDIR: not a directory, scandir '/ext-lib/Backups/Thala - 0/.wine/dosdevices/z:/dev/fd/54'
@biappi commented on GitHub (Sep 21, 2025): After reading [this comment](https://github.com/mrmlnc/fast-glob/pull/485#issuecomment-2639918116) i'm not sure if the following falls in the same umbrella as this issue, sorry if otherwise. But after working around `EACCESS`, i had the same issue with `ENOTDIR` as follows: [Nest] 7 - 09/21/2025, 1:20:07 PM ERROR [Microservices:{"id":"83d57784-995d-4206-bb67-3ea95cb5e5e3"}] Unable to run job handler (LibrarySyncFilesQueueAll): Error: ENOTDIR: not a directory, scandir '/ext-lib/Backups/Thala - 0/.wine/dosdevices/z:/dev/fd/54'
OVERLORD commented 2026-02-05 10:11:55 +03:00
Author
Owner
Copy Link

@conchyliculture commented on GitHub (Sep 30, 2025):

I would also like the scanning to skip over directory/files with access errors and go to the next instead of stopping at the first error

immich_server            | [Nest] 7  - 09/30/2025, 7:22:22 PM   ERROR [Microservices:{"id":"633694ff-43de-4ab8-811b-1a8a21bf4a95"}] Unable to run job handler (LibrarySyncFilesQueueAll): Error: EACCES: permission denied, scandir '/mnt/photos/20020815_old'
immich_server            | Error: EACCES: permission denied, scandir '/mnt/photos/20020815_old'
@conchyliculture commented on GitHub (Sep 30, 2025): I would also like the scanning to skip over directory/files with access errors and go to the next instead of stopping at the first error ``` immich_server | [Nest] 7 - 09/30/2025, 7:22:22 PM ERROR [Microservices:{"id":"633694ff-43de-4ab8-811b-1a8a21bf4a95"}] Unable to run job handler (LibrarySyncFilesQueueAll): Error: EACCES: permission denied, scandir '/mnt/photos/20020815_old' immich_server | Error: EACCES: permission denied, scandir '/mnt/photos/20020815_old' ```
OVERLORD commented 2026-02-05 10:11:57 +03:00
Author
Owner
Copy Link

@etnoy commented on GitHub (Sep 30, 2025):

Unfortunately, this is a bug with fast-glob that we rely on to do file scans. We have an open PR with the upstream code to allow custom error handlers, but so far nothing.

https://github.com/mrmlnc/fast-glob/pull/485

Until that is fixed, any error that occurs during scan will cause the scan to fail.

@etnoy commented on GitHub (Sep 30, 2025): Unfortunately, this is a bug with fast-glob that we rely on to do file scans. We have an open PR with the upstream code to allow custom error handlers, but so far nothing. https://github.com/mrmlnc/fast-glob/pull/485 Until that is fixed, any error that occurs during scan will cause the scan to fail.
OVERLORD commented 2026-02-05 10:11:59 +03:00
Author
Owner
Copy Link

@benmccann commented on GitHub (Nov 6, 2025):

One option would be to use fdir since it supports ignoring of errors: https://github.com/thecodrr/fdir/blob/master/documentation.md#witherrors

fdir supports early termination, but not streaming yet. There's an open PR for streaming support. @etnoy is streaming support a blocker in terms of performance? Maybe it'd be better to switch to fdir for stability and then upgrade it for streaming performance once that's merged and released?

@benmccann commented on GitHub (Nov 6, 2025): One option would be to use `fdir` since it supports ignoring of errors: https://github.com/thecodrr/fdir/blob/master/documentation.md#witherrors `fdir` supports early termination, but not streaming yet. There's an open PR for streaming support. @etnoy is streaming support a blocker in terms of performance? Maybe it'd be better to switch to `fdir` for stability and then upgrade it for streaming performance once that's merged and released?
OVERLORD commented 2026-02-05 10:12:02 +03:00
Author
Owner
Copy Link

@etnoy commented on GitHub (Nov 7, 2025):

One option would be to use fdir since it supports ignoring of errors: https://github.com/thecodrr/fdir/blob/master/documentation.md#witherrors

fdir supports early termination, but not streaming yet. There's an open PR for streaming support. @etnoy is streaming support a blocker in terms of performance? Maybe it'd be better to switch to fdir for stability and then upgrade it for streaming performance once that's merged and released?

I think the easier route here is to fork fast-glob and add the three-ish lines to it to give it a better error handler.

Yes, streaming is critical. Without it, there's no way to scan really large libraries (over 1M assets), and performance will be terrible. I've been able to optimize performance by streaming in large chunks. See https://github.com/immich-app/immich/pull/14456

@etnoy commented on GitHub (Nov 7, 2025): > One option would be to use `fdir` since it supports ignoring of errors: https://github.com/thecodrr/fdir/blob/master/documentation.md#witherrors > > `fdir` supports early termination, but not streaming yet. There's an open PR for streaming support. [@etnoy](https://github.com/etnoy) is streaming support a blocker in terms of performance? Maybe it'd be better to switch to `fdir` for stability and then upgrade it for streaming performance once that's merged and released? I think the easier route here is to fork fast-glob and add the three-ish lines to it to give it a better error handler. Yes, streaming is critical. Without it, there's no way to scan really large libraries (over 1M assets), and performance will be terrible. I've been able to optimize performance by streaming in large chunks. See https://github.com/immich-app/immich/pull/14456
OVERLORD commented 2026-02-05 10:12:05 +03:00
Author
Owner
Copy Link

@benmccann commented on GitHub (Nov 7, 2025):

I'll keep pushing for streaming in tinyglobby then. Most of the JS ecosystem has moved off of fast-glob now with its downloads coming mainly from legacy projects. It's a big supply chain risk because it has so many dependencies and moving to tinyglobby would really help with that and it's also faster as well

@benmccann commented on GitHub (Nov 7, 2025): I'll keep pushing for streaming in tinyglobby then. Most of the JS ecosystem has moved off of fast-glob now with its downloads coming mainly from legacy projects. It's a big supply chain risk because it has so many dependencies and moving to tinyglobby would really help with that and it's also faster as well
OVERLORD commented 2026-02-05 10:12:09 +03:00
Author
Owner
Copy Link

@valentino-saitz-optris commented on GitHub (Jan 16, 2026):

Fast-glob hasn't had any activity for a whole year at this point. Relying on a library with a sole maintainer that won't react to pull requests and hasn't had nearly any activity on Github this past year seems like a bad idea. Meanwhile tinyglobby is actively maintained.

@valentino-saitz-optris commented on GitHub (Jan 16, 2026): Fast-glob hasn't had any activity for a whole year at this point. Relying on a library with a sole maintainer that won't react to pull requests and hasn't had nearly any activity on Github this past year seems like a bad idea. Meanwhile tinyglobby is actively maintained.
OVERLORD commented 2026-02-05 10:12:10 +03:00
Author
Owner
Copy Link

@etnoy commented on GitHub (Jan 16, 2026):

Fast-glob hasn't had any activity for a whole year at this point. Relying on a library with a sole maintainer that won't react to pull requests and hasn't had nearly any activity on Github this past year seems like a bad idea. Meanwhile tinyglobby is actively maintained.

Agreed, but we are relying on features in fast-glob that I haven't found anywhere else. Last I checked, tinyglobby can't be used since it lacks features.

@etnoy commented on GitHub (Jan 16, 2026): > Fast-glob hasn't had any activity for a whole year at this point. Relying on a library with a sole maintainer that won't react to pull requests and hasn't had nearly any activity on Github this past year seems like a bad idea. Meanwhile tinyglobby is actively maintained. Agreed, but we are relying on features in fast-glob that I haven't found anywhere else. Last I checked, tinyglobby can't be used since it lacks features.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
accessibility
changelog:enhancement
changelog:security
changelog:skip
changelog:translation
cli
date-time
dependencies
documentation
external-library
format
good first issue
mobile-beta
mobile-beta
mobile-beta
needs-answer
nice to have
pull-request
Mirrored from GitHub Pull Request
 sharing
tech-debt
📱mobile
🖥️web
🗄️server
🧠machine-learning
No Label external-library 🗄️server
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
OVERLORD
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: immich-app/immich#4346
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?