Microservices metrics not responding #7312

Closed
opened 2026-02-05 12:56:15 +03:00 by OVERLORD · 1 comment
Owner

Originally created by @irishpadres on GitHub (Sep 22, 2025).

I have searched the existing issues, both open and closed, to make sure this is not a duplicate report.

  • Yes

The bug

I have Prometheus metrics enabled using IMMICH_TELEMETRY_INCLUDE=all. The server metrics are working on port 8081 just fine, but the microservices metrics on 8082 are not responding. I verified the port is open and listening but when I try to access it locally via curl it just stays open and never responds.

The OS that Immich Server is running on

Debian 13 (docker)

Version of Immich Server

v1.142.1

Version of Immich Mobile App

v1.142.1

Platform with the issue

  • Server
  • Web
  • Mobile

Device make and model

No response

Your docker-compose.yml content

#
# WARNING: To install Immich, follow our guide: https://immich.app/docs/install/docker-compose
#
# Make sure to use the docker-compose.yml of the current release:
#
# https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml
#
# The compose file on main may not be compatible with the latest release.

name: immich

services:
  immich-server:
    container_name: immich_server
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    extends:
      file: hwaccel.transcoding.yml
      service: nvenc # set to one of [nvenc, quicksync, rkmpp, vaapi, vaapi-wsl] for accelerated transcoding
    volumes:
      # Do not edit the next line. If you want to change the media storage location on your system, edit the value of UPLOAD_LOCATION in the .env file
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - /etc/localtime:/etc/localtime:ro
      - /media/photo:/media
    env_file:
      - .env
    ports:
      - '2283:2283'
      - '28081:8081'
      - '28082:8082'
    depends_on:
      - redis
      - database
    restart: always
    healthcheck:
      disable: false
    labels:
      - "com.centurylinklabs.watchtower.monitor-only=true"

  immich-machine-learning:
    container_name: immich_machine_learning
    # For hardware acceleration, add one of -[armnn, cuda, rocm, openvino, rknn] to the image tag.
    # Example tag: ${IMMICH_VERSION:-release}-cuda
    image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release}-cuda
    extends: # uncomment this section for hardware acceleration - see https://immich.app/docs/features/ml-hardware-acceleration
      file: hwaccel.ml.yml
      service: cuda # set to one of [armnn, cuda, rocm, openvino, openvino-wsl, rknn] for accelerated inference - use the `-wsl` version for WSL2 where applicable
    volumes:
      - model-cache:/cache
    env_file:
      - .env
    restart: always
    healthcheck:
      disable: false
    labels:
      - "com.centurylinklabs.watchtower.monitor-only=true"

  redis:
    container_name: immich_redis
    image: docker.io/valkey/valkey:8-bookworm@sha256:fea8b3e67b15729d4bb70589eb03367bab9ad1ee89c876f54327fc7c6e618571
    healthcheck:
      test: redis-cli ping || exit 1
    restart: always
    labels:
      - "com.centurylinklabs.watchtower.enable=false"

  database:
    container_name: immich_postgres
    image: ghcr.io/immich-app/postgres:14-vectorchord0.3.0-pgvectors0.2.0@sha256:c44be5f2871c59362966d71eab4268170eb6f5653c0e6170184e72b38ffdf107
    environment:
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      POSTGRES_USER: ${DB_USERNAME}
      POSTGRES_DB: ${DB_DATABASE_NAME}
      POSTGRES_INITDB_ARGS: '--data-checksums'
      # Uncomment the DB_STORAGE_TYPE: 'HDD' var if your database isn't stored on SSDs
      # DB_STORAGE_TYPE: 'HDD'
    volumes:
      # Do not edit the next line. If you want to change the database storage location on your system, edit the value of DB_DATA_LOCATION in the .env file
      - ${DB_DATA_LOCATION}:/var/lib/postgresql/data
    shm_size: 128mb
    restart: always
    labels:
      - "com.centurylinklabs.watchtower.enable=false"

volumes:
  model-cache:
  pgdata:

Your .env content

# You can find documentation for all the supported env variables at https://immich.app/docs/install/environment-variables

# The location where your uploaded files are stored
UPLOAD_LOCATION=./library

# The location where your database files are stored. Network shares are not supported for the database
DB_DATA_LOCATION=pgdata

# To set a timezone, uncomment the next line and change Etc/UTC to a TZ identifier from this list: https://en.wikipedia.org/wiki/List_of_tz_database_time_zones#List
TZ=America/Los_Angeles

# The Immich version to use. You can pin this to a specific version like "v1.71.0"
IMMICH_VERSION=release

# Connection secret for postgres. You should change it to a random password
# Please use only the characters `A-Za-z0-9`, without special characters or spaces
DB_PASSWORD=************

# The values below this line do not need to be changed
###################################################################################
DB_USERNAME=postgres
DB_DATABASE_NAME=immich

IMMICH_TELEMETRY_INCLUDE=all

Reproduction steps

  1. Start container
  2. docker compose exec -it immich-server /bin/bash
  3. curl http://localhost:8082/metrics
    ...

Relevant log output


Additional information

No response

Originally created by @irishpadres on GitHub (Sep 22, 2025). ### I have searched the existing issues, both open and closed, to make sure this is not a duplicate report. - [x] Yes ### The bug I have Prometheus metrics enabled using IMMICH_TELEMETRY_INCLUDE=all. The server metrics are working on port 8081 just fine, but the microservices metrics on 8082 are not responding. I verified the port is open and listening but when I try to access it locally via curl it just stays open and never responds. ### The OS that Immich Server is running on Debian 13 (docker) ### Version of Immich Server v1.142.1 ### Version of Immich Mobile App v1.142.1 ### Platform with the issue - [x] Server - [ ] Web - [ ] Mobile ### Device make and model _No response_ ### Your docker-compose.yml content ```YAML # # WARNING: To install Immich, follow our guide: https://immich.app/docs/install/docker-compose # # Make sure to use the docker-compose.yml of the current release: # # https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml # # The compose file on main may not be compatible with the latest release. name: immich services: immich-server: container_name: immich_server image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release} extends: file: hwaccel.transcoding.yml service: nvenc # set to one of [nvenc, quicksync, rkmpp, vaapi, vaapi-wsl] for accelerated transcoding volumes: # Do not edit the next line. If you want to change the media storage location on your system, edit the value of UPLOAD_LOCATION in the .env file - ${UPLOAD_LOCATION}:/usr/src/app/upload - /etc/localtime:/etc/localtime:ro - /media/photo:/media env_file: - .env ports: - '2283:2283' - '28081:8081' - '28082:8082' depends_on: - redis - database restart: always healthcheck: disable: false labels: - "com.centurylinklabs.watchtower.monitor-only=true" immich-machine-learning: container_name: immich_machine_learning # For hardware acceleration, add one of -[armnn, cuda, rocm, openvino, rknn] to the image tag. # Example tag: ${IMMICH_VERSION:-release}-cuda image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release}-cuda extends: # uncomment this section for hardware acceleration - see https://immich.app/docs/features/ml-hardware-acceleration file: hwaccel.ml.yml service: cuda # set to one of [armnn, cuda, rocm, openvino, openvino-wsl, rknn] for accelerated inference - use the `-wsl` version for WSL2 where applicable volumes: - model-cache:/cache env_file: - .env restart: always healthcheck: disable: false labels: - "com.centurylinklabs.watchtower.monitor-only=true" redis: container_name: immich_redis image: docker.io/valkey/valkey:8-bookworm@sha256:fea8b3e67b15729d4bb70589eb03367bab9ad1ee89c876f54327fc7c6e618571 healthcheck: test: redis-cli ping || exit 1 restart: always labels: - "com.centurylinklabs.watchtower.enable=false" database: container_name: immich_postgres image: ghcr.io/immich-app/postgres:14-vectorchord0.3.0-pgvectors0.2.0@sha256:c44be5f2871c59362966d71eab4268170eb6f5653c0e6170184e72b38ffdf107 environment: POSTGRES_PASSWORD: ${DB_PASSWORD} POSTGRES_USER: ${DB_USERNAME} POSTGRES_DB: ${DB_DATABASE_NAME} POSTGRES_INITDB_ARGS: '--data-checksums' # Uncomment the DB_STORAGE_TYPE: 'HDD' var if your database isn't stored on SSDs # DB_STORAGE_TYPE: 'HDD' volumes: # Do not edit the next line. If you want to change the database storage location on your system, edit the value of DB_DATA_LOCATION in the .env file - ${DB_DATA_LOCATION}:/var/lib/postgresql/data shm_size: 128mb restart: always labels: - "com.centurylinklabs.watchtower.enable=false" volumes: model-cache: pgdata: ``` ### Your .env content ```Shell # You can find documentation for all the supported env variables at https://immich.app/docs/install/environment-variables # The location where your uploaded files are stored UPLOAD_LOCATION=./library # The location where your database files are stored. Network shares are not supported for the database DB_DATA_LOCATION=pgdata # To set a timezone, uncomment the next line and change Etc/UTC to a TZ identifier from this list: https://en.wikipedia.org/wiki/List_of_tz_database_time_zones#List TZ=America/Los_Angeles # The Immich version to use. You can pin this to a specific version like "v1.71.0" IMMICH_VERSION=release # Connection secret for postgres. You should change it to a random password # Please use only the characters `A-Za-z0-9`, without special characters or spaces DB_PASSWORD=************ # The values below this line do not need to be changed ################################################################################### DB_USERNAME=postgres DB_DATABASE_NAME=immich IMMICH_TELEMETRY_INCLUDE=all ``` ### Reproduction steps 1. Start container 2. docker compose exec -it immich-server /bin/bash 3. curl http://localhost:8082/metrics ... ### Relevant log output ```shell ``` ### Additional information _No response_
Author
Owner

@skatsubo commented on GitHub (Sep 25, 2025):

Thanks for the info provided.
I did some testing and wasn't able to reproduce it so far. Tried with v1.142.1 and v1.143.0 on x64 and arm64.
Both curl calls to 8081 and 8082 from inside the container return data promptly - curl http://localhost:8082/metrics.
So I'm going to convert this to discussion.

Have you tried to reproduce it on a fresh Immich using stock compose+.env?

@skatsubo commented on GitHub (Sep 25, 2025): Thanks for the info provided. I did some testing and wasn't able to reproduce it so far. Tried with v1.142.1 and v1.143.0 on x64 and arm64. Both curl calls to 8081 and 8082 from inside the container return data promptly - `curl http://localhost:8082/metrics`. So I'm going to convert this to discussion. Have you tried to reproduce it on a fresh Immich using stock compose+.env?
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: immich-app/immich#7312