[BUG] Running as Non-Root User - Reverse Geocoding Fails #875

Closed
opened 2026-02-04 23:14:47 +03:00 by OVERLORD · 13 comments
Owner

Originally created by @jeremywillans on GitHub (May 23, 2023).

The bug

Attempting to follow the FAQ Instructions results in the following error in the log -

immich_microservices       | [Nest] 1  - 05/23/2023, 9:03:59 AM   ERROR [MetadataExtractionProcessor] Unable to initialize reverse geocoding: Error: EACCES: permission denied, rmdir '/usr/src/app/.reverse-geocoding-dump/'
immich_microservices       | Error: EACCES: permission denied, rmdir '/usr/src/app/.reverse-geocoding-dump/'

Tried with both a mounted folder with 777 and a docker volume.

The OS that Immich Server is running on

CentOS Linux release 7.9.2009

Version of Immich Server

1.57

Version of Immich Mobile App

1.57

Platform with the issue

  • Server
  • Web
  • Mobile

Your docker-compose.yml content

version: "3.8"

services:
  immich-server:
    user: 1024:100
    container_name: immich_server
    image: altran1502/immich-server:release
    entrypoint: [ "/bin/sh", "./start-server.sh" ]
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
    env_file:
      - .env
    environment:
      - NODE_ENV=production
    depends_on:
      - redis
      - database
    restart: always

  immich-microservices:
    user: 1024:100
    container_name: immich_microservices
    image: altran1502/immich-server:release
    entrypoint: [ "/bin/sh", "./start-microservices.sh" ]
    volumes:
      - geocoding:/usr/src/app/.reverse-geocoding-dump
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
    env_file:
      - .env
    environment:
      - NODE_ENV=production
    depends_on:
      - redis
      - database
      - typesense
    restart: always

  immich-machine-learning:
    user: 1024:100
    container_name: immich_machine_learning
    image: altran1502/immich-machine-learning:release
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - model-cache:/cache
    env_file:
      - .env
    environment:
      - NODE_ENV=production
    restart: always

  immich-web:
    container_name: immich_web
    image: altran1502/immich-web:release
    entrypoint: [ "/bin/sh", "./entrypoint.sh" ]
    env_file:
      - .env
    restart: always

  redis:
    container_name: immich_redis
    image: redis:6.2
    restart: always
    labels:
      - wud.watch=false

  database:
    user: 1024:100
    container_name: immich_postgres
    image: postgres:14
    env_file:
      - .env
    environment:
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      POSTGRES_USER: ${DB_USERNAME}
      POSTGRES_DB: ${DB_DATABASE_NAME}
      PG_DATA: /var/lib/postgresql/data
    volumes:
      - ${PG_LOCATION}:/var/lib/postgresql/data
    restart: always
    labels:
      - wud.watch=false

  immich-proxy:
    container_name: immich_proxy
    image: altran1502/immich-proxy:release
    environment:
      # Make sure these values get passed through from the env file
      - IMMICH_SERVER_URL
      - IMMICH_WEB_URL
    ports:
      - 2283:8080
    logging:
      driver: none
    depends_on:
      - immich-server
    restart: always

  typesense:
    container_name: immich_typesense
    image: typesense/typesense:0.24.0
    environment:
      - TYPESENSE_API_KEY=${TYPESENSE_API_KEY}
      - TYPESENSE_DATA_DIR=/data
    logging:
      driver: none
    volumes:
      - tsdata:/data
    labels:
      - wud.watch=false
    restart: always

volumes:
  model-cache:
  tsdata:
  geocoding:

Your .env content

###################################################################################
# Database
###################################################################################

DB_HOSTNAME=immich_postgres
DB_USERNAME=postgres
DB_PASSWORD=.....
DB_DATABASE_NAME=immich

# Optional Database settings:
# DB_PORT=5432

###################################################################################
# Redis
###################################################################################

REDIS_HOSTNAME=immich_redis

# Optional Redis settings:
# REDIS_PORT=6379
# REDIS_DBINDEX=0
# REDIS_PASSWORD=
# REDIS_SOCKET=

###################################################################################
# Upload File Location
#
# This is the location where uploaded files are stored.
###################################################################################

UPLOAD_LOCATION=/mnt/containers/config/immich/photos
PG_LOCATION=/mnt/containers/config/immich/db

###################################################################################
# Typesense
###################################################################################

TYPESENSE_API_KEY=.....

###################################################################################
# Reverse Geocoding
#
# Reverse geocoding is done locally which has a small impact on memory usage
# This memory usage can be altered by changing the REVERSE_GEOCODING_PRECISION variable
# This ranges from 0-3 with 3 being the most precise
# 3 - Cities > 500 population: ~200MB RAM
# 2 - Cities > 1000 population: ~150MB RAM
# 1 - Cities > 5000 population: ~80MB RAM
# 0 - Cities > 15000 population: ~40MB RAM
####################################################################################

# DISABLE_REVERSE_GEOCODING=false
# REVERSE_GEOCODING_PRECISION=3

####################################################################################
# WEB - Optional
#
# Custom message on the login page, should be written in HTML form.
# For example:
# PUBLIC_LOGIN_PAGE_MESSAGE="This is a demo instance of Immich.<br><br>Email: <i>demo@demo.de</i><br>Password: <i>demo</i>"
####################################################################################

PUBLIC_LOGIN_PAGE_MESSAGE=

####################################################################################
# Alternative Service Addresses - Optional
#
# This is an advanced feature for users who may be running their immich services on different hosts.
# It will not change which address or port that services bind to within their containers, but it will change where other services look for their peers.
# Note: immich-microservices is bound to 3002, but no references are made
####################################################################################

IMMICH_WEB_URL=http://immich-web:3000
IMMICH_SERVER_URL=http://immich-server:3001
IMMICH_MACHINE_LEARNING_URL=http://immich-machine-learning:3003

####################################################################################
# Alternative API's External Address - Optional
#
# This is an advanced feature used to control the public server endpoint returned to clients during Well-known discovery.
# You should only use this if you want mobile apps to access the immich API over a custom URL. Do not include trailing slash.
# NOTE: At this time, the web app will not be affected by this setting and will continue to use the relative path: /api
# Examples: http://localhost:3001, http://immich-api.example.com, etc
####################################################################################

#IMMICH_API_URL_EXTERNAL=http://localhost:3001

Reproduction steps

1. Update Docker Compose
2. Start Services
3. Error shown in Docker logs

Additional information

No response

Originally created by @jeremywillans on GitHub (May 23, 2023). ### The bug Attempting to follow the [FAQ Instructions](https://documentation.immich.app/docs/FAQ#how-can-i-run-immich-as-a-non-root-user) results in the following error in the log - ``` immich_microservices | [Nest] 1 - 05/23/2023, 9:03:59 AM ERROR [MetadataExtractionProcessor] Unable to initialize reverse geocoding: Error: EACCES: permission denied, rmdir '/usr/src/app/.reverse-geocoding-dump/' immich_microservices | Error: EACCES: permission denied, rmdir '/usr/src/app/.reverse-geocoding-dump/' ``` Tried with both a mounted folder with 777 and a docker volume. ### The OS that Immich Server is running on CentOS Linux release 7.9.2009 ### Version of Immich Server 1.57 ### Version of Immich Mobile App 1.57 ### Platform with the issue - [X] Server - [ ] Web - [ ] Mobile ### Your docker-compose.yml content ```YAML version: "3.8" services: immich-server: user: 1024:100 container_name: immich_server image: altran1502/immich-server:release entrypoint: [ "/bin/sh", "./start-server.sh" ] volumes: - ${UPLOAD_LOCATION}:/usr/src/app/upload env_file: - .env environment: - NODE_ENV=production depends_on: - redis - database restart: always immich-microservices: user: 1024:100 container_name: immich_microservices image: altran1502/immich-server:release entrypoint: [ "/bin/sh", "./start-microservices.sh" ] volumes: - geocoding:/usr/src/app/.reverse-geocoding-dump - ${UPLOAD_LOCATION}:/usr/src/app/upload env_file: - .env environment: - NODE_ENV=production depends_on: - redis - database - typesense restart: always immich-machine-learning: user: 1024:100 container_name: immich_machine_learning image: altran1502/immich-machine-learning:release volumes: - ${UPLOAD_LOCATION}:/usr/src/app/upload - model-cache:/cache env_file: - .env environment: - NODE_ENV=production restart: always immich-web: container_name: immich_web image: altran1502/immich-web:release entrypoint: [ "/bin/sh", "./entrypoint.sh" ] env_file: - .env restart: always redis: container_name: immich_redis image: redis:6.2 restart: always labels: - wud.watch=false database: user: 1024:100 container_name: immich_postgres image: postgres:14 env_file: - .env environment: POSTGRES_PASSWORD: ${DB_PASSWORD} POSTGRES_USER: ${DB_USERNAME} POSTGRES_DB: ${DB_DATABASE_NAME} PG_DATA: /var/lib/postgresql/data volumes: - ${PG_LOCATION}:/var/lib/postgresql/data restart: always labels: - wud.watch=false immich-proxy: container_name: immich_proxy image: altran1502/immich-proxy:release environment: # Make sure these values get passed through from the env file - IMMICH_SERVER_URL - IMMICH_WEB_URL ports: - 2283:8080 logging: driver: none depends_on: - immich-server restart: always typesense: container_name: immich_typesense image: typesense/typesense:0.24.0 environment: - TYPESENSE_API_KEY=${TYPESENSE_API_KEY} - TYPESENSE_DATA_DIR=/data logging: driver: none volumes: - tsdata:/data labels: - wud.watch=false restart: always volumes: model-cache: tsdata: geocoding: ``` ### Your .env content ```Shell ################################################################################### # Database ################################################################################### DB_HOSTNAME=immich_postgres DB_USERNAME=postgres DB_PASSWORD=..... DB_DATABASE_NAME=immich # Optional Database settings: # DB_PORT=5432 ################################################################################### # Redis ################################################################################### REDIS_HOSTNAME=immich_redis # Optional Redis settings: # REDIS_PORT=6379 # REDIS_DBINDEX=0 # REDIS_PASSWORD= # REDIS_SOCKET= ################################################################################### # Upload File Location # # This is the location where uploaded files are stored. ################################################################################### UPLOAD_LOCATION=/mnt/containers/config/immich/photos PG_LOCATION=/mnt/containers/config/immich/db ################################################################################### # Typesense ################################################################################### TYPESENSE_API_KEY=..... ################################################################################### # Reverse Geocoding # # Reverse geocoding is done locally which has a small impact on memory usage # This memory usage can be altered by changing the REVERSE_GEOCODING_PRECISION variable # This ranges from 0-3 with 3 being the most precise # 3 - Cities > 500 population: ~200MB RAM # 2 - Cities > 1000 population: ~150MB RAM # 1 - Cities > 5000 population: ~80MB RAM # 0 - Cities > 15000 population: ~40MB RAM #################################################################################### # DISABLE_REVERSE_GEOCODING=false # REVERSE_GEOCODING_PRECISION=3 #################################################################################### # WEB - Optional # # Custom message on the login page, should be written in HTML form. # For example: # PUBLIC_LOGIN_PAGE_MESSAGE="This is a demo instance of Immich.<br><br>Email: <i>demo@demo.de</i><br>Password: <i>demo</i>" #################################################################################### PUBLIC_LOGIN_PAGE_MESSAGE= #################################################################################### # Alternative Service Addresses - Optional # # This is an advanced feature for users who may be running their immich services on different hosts. # It will not change which address or port that services bind to within their containers, but it will change where other services look for their peers. # Note: immich-microservices is bound to 3002, but no references are made #################################################################################### IMMICH_WEB_URL=http://immich-web:3000 IMMICH_SERVER_URL=http://immich-server:3001 IMMICH_MACHINE_LEARNING_URL=http://immich-machine-learning:3003 #################################################################################### # Alternative API's External Address - Optional # # This is an advanced feature used to control the public server endpoint returned to clients during Well-known discovery. # You should only use this if you want mobile apps to access the immich API over a custom URL. Do not include trailing slash. # NOTE: At this time, the web app will not be affected by this setting and will continue to use the relative path: /api # Examples: http://localhost:3001, http://immich-api.example.com, etc #################################################################################### #IMMICH_API_URL_EXTERNAL=http://localhost:3001 ``` ### Reproduction steps ```bash 1. Update Docker Compose 2. Start Services 3. Error shown in Docker logs ``` ### Additional information _No response_
Author
Owner

@uhthomas commented on GitHub (May 24, 2023):

I don't think this is an immich bug, this can be resolved by mounting an empty volume to /usr/src/app/.reverse-geocoding-dump.

See for example, in Kubernetes as non-root.

0549cc9fbd/k8s/unwind/immich/immich_microservice/deployment_list.cue (L29-L32)

0549cc9fbd/k8s/unwind/immich/immich_microservice/deployment_list.cue (L86-L89)

@uhthomas commented on GitHub (May 24, 2023): I don't think this is an immich bug, this can be resolved by mounting an empty volume to `/usr/src/app/.reverse-geocoding-dump`. See for example, in Kubernetes as non-root. https://github.com/uhthomas/automata/blob/0549cc9fbdf656e969b6ba963a34cc6e94402bd7/k8s/unwind/immich/immich_microservice/deployment_list.cue#L29-L32 https://github.com/uhthomas/automata/blob/0549cc9fbdf656e969b6ba963a34cc6e94402bd7/k8s/unwind/immich/immich_microservice/deployment_list.cue#L86-L89
Author
Owner

@uhthomas commented on GitHub (May 24, 2023):

Though, possibly relates to https://github.com/immich-app/immich/discussions/2537. It would be nice to document which paths immich uses rather than trial and error.

@uhthomas commented on GitHub (May 24, 2023): Though, possibly relates to https://github.com/immich-app/immich/discussions/2537. It would be nice to document which paths immich uses rather than trial and error.
Author
Owner

@jeremywillans commented on GitHub (May 24, 2023):

@uhthomas, I did mount an empty volume to this location. Tried both a local, remote and docker volumes and got the same error for each.

Per the logs, it appears as it's trying to delete the entire folder during startup?

@jeremywillans commented on GitHub (May 24, 2023): @uhthomas, I did mount an empty volume to this location. Tried both a local, remote and docker volumes and got the same error for each. Per the logs, it appears as it's trying to delete the entire folder during startup?
Author
Owner

@jrasm91 commented on GitHub (May 24, 2023):

@uhthomas, I did mount an empty volume to this location. Tried both a local, remote and docker volumes and got the same error for each.

Per the logs, it appears as it's trying to delete the entire folder during startup?

It used to crash if there was a corrupt CSV file. Now, it tries to detect that and delete the directory so it's redownload.

@jrasm91 commented on GitHub (May 24, 2023): > @uhthomas, I did mount an empty volume to this location. Tried both a local, remote and docker volumes and got the same error for each. > > Per the logs, it appears as it's trying to delete the entire folder during startup? It used to crash if there was a corrupt CSV file. Now, it tries to detect that and delete the directory so it's redownload.
Author
Owner

@jeremywillans commented on GitHub (May 24, 2023):

given the directory is mounted externally though, is that now not actually possible?

@jeremywillans commented on GitHub (May 24, 2023): given the directory is mounted externally though, is that now not actually possible?
Author
Owner

@uhthomas commented on GitHub (May 24, 2023):

Makes sense, apologies for my misunderstanding. I also see this.

[Nest] 1  - 05/23/2023, 9:54:12 PM   ERROR [MetadataExtractionProcessor] Unable to initialize reverse geocoding: Error: EROFS: read-only file system, rmdir '/usr/src/app/.reverse-geocoding-dump/'
Error: EROFS: read-only file system, rmdir '/usr/src/app/.reverse-geocoding-dump/'
@uhthomas commented on GitHub (May 24, 2023): Makes sense, apologies for my misunderstanding. I also see this. ``` [Nest] 1 - 05/23/2023, 9:54:12 PM ERROR [MetadataExtractionProcessor] Unable to initialize reverse geocoding: Error: EROFS: read-only file system, rmdir '/usr/src/app/.reverse-geocoding-dump/' Error: EROFS: read-only file system, rmdir '/usr/src/app/.reverse-geocoding-dump/' ```
Author
Owner

@jrasm91 commented on GitHub (May 24, 2023):

Yes, I'm wondering if we need to change it to remove the sub folders/files not the parent directory itself.

@jrasm91 commented on GitHub (May 24, 2023): Yes, I'm wondering if we need to change it to remove the sub folders/files not the parent directory itself.
Author
Owner

@jrasm91 commented on GitHub (May 24, 2023):

You can try manually deleting the contents and it should "just work". But you'll still have the issue of occasional corrupt CSV download at which time you'll have to manually remove the contents again. The goal was to try to auto-fix it, but the current solution conflicts for non root setups as well as whenever the folder is a volume mount.

@jrasm91 commented on GitHub (May 24, 2023): You can try manually deleting the contents and it should "just work". But you'll still have the issue of occasional corrupt CSV download at which time you'll have to manually remove the contents again. The goal was to try to auto-fix it, but the current solution conflicts for non root setups as well as whenever the folder is a volume mount.
Author
Owner

@jeremywillans commented on GitHub (May 24, 2023):

in my test cases, the directory is empty on first attempt, and its still failing unfortunately

@jeremywillans commented on GitHub (May 24, 2023): in my test cases, the directory is empty on first attempt, and its still failing unfortunately
Author
Owner

@uhthomas commented on GitHub (May 24, 2023):

You can try manually deleting the contents and it should "just work". But you'll still have the issue of occasional corrupt CSV download at which time you'll have to manually remove the contents again. The goal was to try to auto-fix it, but the current solution conflicts for non root setups as well as whenever the folder is a volume mount.

Given these are empty, non-persistent volumes then immich should have no reason to clean it up.

@uhthomas commented on GitHub (May 24, 2023): > You can try manually deleting the contents and it should "just work". But you'll still have the issue of occasional corrupt CSV download at which time you'll have to manually remove the contents again. The goal was to try to auto-fix it, but the current solution conflicts for non root setups as well as whenever the folder is a volume mount. Given these are empty, non-persistent volumes then immich should have no reason to clean it up.
Author
Owner

@uhthomas commented on GitHub (May 24, 2023):

Maybe use /tmp/immich-reverse-geocoding-dump instead? That would probably be a better place for it anyway?

@uhthomas commented on GitHub (May 24, 2023): Maybe use `/tmp/immich-reverse-geocoding-dump` instead? That would probably be a better place for it anyway?
Author
Owner

@jeremywillans commented on GitHub (May 24, 2023):

Based on L61 and L31 it looks to try clean it up each startup regardless on current status.

so yeah in theory either moving the directory path or only deleting contents should resolve this issue moving forward. im happy to write up a PR for it, just which preference ? 😄

@jeremywillans commented on GitHub (May 24, 2023): Based on [L61](https://github.com/immich-app/immich/blob/c2145cbe1128d5ba36c50737d821fb240c072614/server/apps/microservices/src/processors/metadata-extraction.processor.ts#L61) and [L31](https://github.com/immich-app/immich/blob/c2145cbe1128d5ba36c50737d821fb240c072614/server/libs/infra/src/repositories/geocoding.repository.ts#L31) it looks to try clean it up each startup regardless on current status. so yeah in theory either moving the directory path or only deleting contents should resolve this issue moving forward. im happy to write up a PR for it, just which preference ? 😄
Author
Owner

@jrasm91 commented on GitHub (May 24, 2023):

Oh shoooooot lol, that's definitely a bug. It's only supposed to delete on an unhandled csv exception here

@jrasm91 commented on GitHub (May 24, 2023): Oh shoooooot lol, that's definitely a bug. It's only supposed to delete on an unhandled csv exception [here](https://github.com/immich-app/immich/blob/c2145cbe1128d5ba36c50737d821fb240c072614/server/apps/microservices/src/main.ts#L22-L30)
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: immich-app/immich#875