immich-app/immich
1
0
Fork 0
mirror of https://github.com/immich-app/immich.git synced 2026-07-25 14:00:45 +03:00
Code Issues 783 Packages Projects Releases 22 Wiki Activity

upload doesn't work while using an SSL Client Certificate #4101

New Issue
Closed
opened 2026-02-05 09:43:48 +03:00 by OVERLORD · 19 comments
OVERLORD commented 2026-02-05 09:43:48 +03:00
Owner
Copy Link

Originally created by @sebpapillon on GitHub (Aug 17, 2024).

The bug

upload doesn't work when I'm using an SSL Client Certificate, works well to connect to the server, download, display pictures but will not upload anything. I'm using haproxy and the error message is quite clear :

fd[0x21] OpenSSL error[0xa0000c7] tls_process_client_certificate: peer did not return a certificate

The OS that Immich Server is running on

Debian 12

Version of Immich Server

v1.112.1

Version of Immich Mobile App

v1.112.1

Platform with the issue

  • Server
  • Web
  • Mobile

Your docker-compose.yml content

#
# WARNING: Make sure to use the docker-compose.yml of the current release:
#
# https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml
#
# The compose file on main may not be compatible with the latest release.
#

name: immich

services:
  immich-server:
    container_name: immich_server
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    # extends:
    #   file: hwaccel.transcoding.yml
    #   service: cpu # set to one of [nvenc, quicksync, rkmpp, vaapi, vaapi-wsl] for accelerated transcoding
    volumes:
      # Do not edit the next line. If you want to change the media storage location on your system, edit the value of UPLOAD_LOCATION in the .env file
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - /etc/localtime:/etc/localtime:ro
    env_file:
      - .env
    ports:
      - 2283:3001
    depends_on:
      - redis
      - database
    restart: always

  immich-machine-learning:
    container_name: immich_machine_learning
    # For hardware acceleration, add one of -[armnn, cuda, openvino] to the image tag.
    # Example tag: ${IMMICH_VERSION:-release}-cuda
    image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release}
    # extends: # uncomment this section for hardware acceleration - see https://immich.app/docs/features/ml-hardware-acceleration
    #   file: hwaccel.ml.yml
    #   service: cpu # set to one of [armnn, cuda, openvino, openvino-wsl] for accelerated inference - use the `-wsl` version for WSL2 where applicable
    volumes:
      - model-cache:/cache
    env_file:
      - .env
    restart: always

  redis:
    container_name: immich_redis
    image: docker.io/redis:6.2-alpine@sha256:e3b17ba9479deec4b7d1eeec1548a253acc5374d68d3b27937fcfe4df8d18c7e
    healthcheck:
      test: redis-cli ping || exit 1
    restart: always

  database:
    container_name: immich_postgres
    image: docker.io/tensorchord/pgvecto-rs:pg14-v0.2.0@sha256:90724186f0a3517cf6914295b5ab410db9ce23190a2d9d0b9dd6463e3fa298f0
    environment:
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      POSTGRES_USER: ${DB_USERNAME}
      POSTGRES_DB: ${DB_DATABASE_NAME}
      POSTGRES_INITDB_ARGS: '--data-checksums'
    volumes:
      # Do not edit the next line. If you want to change the database storage location on your system, edit the value of DB_DATA_LOCATION in the .env file
      - ${DB_DATA_LOCATION}:/var/lib/postgresql/data
    healthcheck:
      test: pg_isready --dbname='${DB_DATABASE_NAME}' --username='${DB_USERNAME}' || exit 1; Chksum="$$(psql --dbname='${DB_DATABASE_NAME}' --username='${DB_USERNAME}' --tuples-only --no-align --command='SELECT COALESCE(SUM(checksum_failures), 0) FROM pg_stat_database')"; echo "checksum failure count is $$Chksum"; [ "$$Chksum" = '0' ] || exit 1
      interval: 5m
      start_interval: 30s
      start_period: 5m
    command: ["postgres", "-c" ,"shared_preload_libraries=vectors.so", "-c", 'search_path="$$user", public, vectors', "-c", "logging_collector=on", "-c", "max_wal_size=2GB", "-c", "shared_buffers=512MB", "-c", "wal_compression=on"]
    restart: always

volumes:
  model-cache:

Your .env content

# You can find documentation for all the supported env variables at https://immich.app/docs/install/environment-variables

# The location where your uploaded files are stored
UPLOAD_LOCATION=./library
# The location where your database files are stored
DB_DATA_LOCATION=./postgres

# To set a timezone, uncomment the next line and change Etc/UTC to a TZ identifier from this list: https://en.wikipedia.org/wiki/List_of_tz_database_time_zones#List
# TZ=Etc/UTC

# The Immich version to use. You can pin this to a specific version like "v1.71.0"
IMMICH_VERSION=release

# Connection secret for postgres. You should change it to a random password
DB_PASSWORD=postgres

# The values below this line do not need to be changed
###################################################################################
DB_USERNAME=postgres
DB_DATABASE_NAME=immich

Reproduction steps

1.open the mobile app
2.long press on an image marked as not uploaded 
3.select upload
...

Relevant log output

from haproxy in debug mode
fd[0x21] OpenSSL error[0xa0000c7] tls_process_client_certificate: peer did not return a certificate

Additional information

No response

Originally created by @sebpapillon on GitHub (Aug 17, 2024). ### The bug upload doesn't work when I'm using an SSL Client Certificate, works well to connect to the server, download, display pictures but will not upload anything. I'm using haproxy and the error message is quite clear : fd[0x21] OpenSSL error[0xa0000c7] tls_process_client_certificate: peer did not return a certificate ### The OS that Immich Server is running on Debian 12 ### Version of Immich Server v1.112.1 ### Version of Immich Mobile App v1.112.1 ### Platform with the issue - [ ] Server - [ ] Web - [X] Mobile ### Your docker-compose.yml content ```YAML # # WARNING: Make sure to use the docker-compose.yml of the current release: # # https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml # # The compose file on main may not be compatible with the latest release. # name: immich services: immich-server: container_name: immich_server image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release} # extends: # file: hwaccel.transcoding.yml # service: cpu # set to one of [nvenc, quicksync, rkmpp, vaapi, vaapi-wsl] for accelerated transcoding volumes: # Do not edit the next line. If you want to change the media storage location on your system, edit the value of UPLOAD_LOCATION in the .env file - ${UPLOAD_LOCATION}:/usr/src/app/upload - /etc/localtime:/etc/localtime:ro env_file: - .env ports: - 2283:3001 depends_on: - redis - database restart: always immich-machine-learning: container_name: immich_machine_learning # For hardware acceleration, add one of -[armnn, cuda, openvino] to the image tag. # Example tag: ${IMMICH_VERSION:-release}-cuda image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release} # extends: # uncomment this section for hardware acceleration - see https://immich.app/docs/features/ml-hardware-acceleration # file: hwaccel.ml.yml # service: cpu # set to one of [armnn, cuda, openvino, openvino-wsl] for accelerated inference - use the `-wsl` version for WSL2 where applicable volumes: - model-cache:/cache env_file: - .env restart: always redis: container_name: immich_redis image: docker.io/redis:6.2-alpine@sha256:e3b17ba9479deec4b7d1eeec1548a253acc5374d68d3b27937fcfe4df8d18c7e healthcheck: test: redis-cli ping || exit 1 restart: always database: container_name: immich_postgres image: docker.io/tensorchord/pgvecto-rs:pg14-v0.2.0@sha256:90724186f0a3517cf6914295b5ab410db9ce23190a2d9d0b9dd6463e3fa298f0 environment: POSTGRES_PASSWORD: ${DB_PASSWORD} POSTGRES_USER: ${DB_USERNAME} POSTGRES_DB: ${DB_DATABASE_NAME} POSTGRES_INITDB_ARGS: '--data-checksums' volumes: # Do not edit the next line. If you want to change the database storage location on your system, edit the value of DB_DATA_LOCATION in the .env file - ${DB_DATA_LOCATION}:/var/lib/postgresql/data healthcheck: test: pg_isready --dbname='${DB_DATABASE_NAME}' --username='${DB_USERNAME}' || exit 1; Chksum="$$(psql --dbname='${DB_DATABASE_NAME}' --username='${DB_USERNAME}' --tuples-only --no-align --command='SELECT COALESCE(SUM(checksum_failures), 0) FROM pg_stat_database')"; echo "checksum failure count is $$Chksum"; [ "$$Chksum" = '0' ] || exit 1 interval: 5m start_interval: 30s start_period: 5m command: ["postgres", "-c" ,"shared_preload_libraries=vectors.so", "-c", 'search_path="$$user", public, vectors', "-c", "logging_collector=on", "-c", "max_wal_size=2GB", "-c", "shared_buffers=512MB", "-c", "wal_compression=on"] restart: always volumes: model-cache: ``` ### Your .env content ```Shell # You can find documentation for all the supported env variables at https://immich.app/docs/install/environment-variables # The location where your uploaded files are stored UPLOAD_LOCATION=./library # The location where your database files are stored DB_DATA_LOCATION=./postgres # To set a timezone, uncomment the next line and change Etc/UTC to a TZ identifier from this list: https://en.wikipedia.org/wiki/List_of_tz_database_time_zones#List # TZ=Etc/UTC # The Immich version to use. You can pin this to a specific version like "v1.71.0" IMMICH_VERSION=release # Connection secret for postgres. You should change it to a random password DB_PASSWORD=postgres # The values below this line do not need to be changed ################################################################################### DB_USERNAME=postgres DB_DATABASE_NAME=immich ``` ### Reproduction steps ```bash 1.open the mobile app 2.long press on an image marked as not uploaded 3.select upload ... ``` ### Relevant log output ```shell from haproxy in debug mode fd[0x21] OpenSSL error[0xa0000c7] tls_process_client_certificate: peer did not return a certificate ``` ### Additional information _No response_
OVERLORD commented 2026-02-05 09:43:52 +03:00
Author
Owner
Copy Link

@tgutzler commented on GitHub (Aug 19, 2024):

I was having upload issues after putting nginx in front of my immich server.
The root cause was different - I had forgotten to increase the default body size, which caused nginx to throw an error.

@tgutzler commented on GitHub (Aug 19, 2024): I was having upload issues after putting nginx in front of my immich server. The root cause was different - I had forgotten to increase the default body size, which caused nginx to throw an error.
OVERLORD commented 2026-02-05 09:43:53 +03:00
Author
Owner
Copy Link

@PottiMc commented on GitHub (Aug 22, 2024):

As my issue was unfortunately moved to discussions I can tell you I have the exact same issue but don't use a self-hosted reverse proxy. Instead I use a Cloudflare Tunnel. I can't find any logs there but my issue is exactly the same. Can do everything in instance, only mobile uploading is not working.
When I switched the app to the local IP, everything is working fine.

@PottiMc commented on GitHub (Aug 22, 2024): As my issue was unfortunately moved to discussions I can tell you I have the exact same issue but don't use a self-hosted reverse proxy. Instead I use a Cloudflare Tunnel. I can't find any logs there but my issue is exactly the same. Can do everything in instance, only mobile uploading is not working. When I switched the app to the local IP, everything is working fine.
OVERLORD commented 2026-02-05 09:43:56 +03:00
Author
Owner
Copy Link

@RogerHartmann commented on GitHub (Aug 26, 2024):

I have a similar issue with nginx and iOS. Features beside of uploading seem to work fine.
What I am wondering is that the iOS client is also giving a hint about iCloud when I am connected via reverse proxy and trying to do an upload (and the upload fails). When I am directly connected to immich, I do not see that hint and the upload is working.
No idea if that is somehow related...

Update: when trying to gather some information for debugging, I got the update working (Update2: but not reliably).
After first login, it failed again. Find the iCloud download screenshot
iCloud_Download_stuck

Then, when I wanted to re-trigger the upload and record some nginx logs, I killed the iOS app. After restarting the iOS app, the immich upload was working.

I wonder how iCloud download is related to image upload. The photo was just taken locally and never uploaded to iCloud.

Update2: it is not really stable. Now, after some more minutes, upload is getting stuck again

@RogerHartmann commented on GitHub (Aug 26, 2024): I have a similar issue with nginx and iOS. Features beside of uploading seem to work fine. What I am wondering is that the iOS client is also giving a hint about iCloud when I am connected via reverse proxy and trying to do an upload (and the upload fails). When I am directly connected to immich, I do not see that hint and the upload is working. No idea if that is somehow related... Update: when trying to gather some information for debugging, I got the update working (Update2: but not reliably). After first login, it failed again. Find the iCloud download screenshot ![iCloud_Download_stuck](https://github.com/user-attachments/assets/24a76d9e-460e-4038-8245-f4ed8625a367) Then, when I wanted to re-trigger the upload and record some nginx logs, I killed the iOS app. After restarting the iOS app, the immich upload was working. I wonder how iCloud download is related to image upload. The photo was just taken locally and never uploaded to iCloud. Update2: it is not really stable. Now, after some more minutes, upload is getting stuck again
OVERLORD commented 2026-02-05 09:43:57 +03:00
Author
Owner
Copy Link

@sebpapillon commented on GitHub (Aug 27, 2024):

the problem seems to have solved itself for whatever reason unknown to me.

@sebpapillon commented on GitHub (Aug 27, 2024): the problem seems to have solved itself for whatever reason unknown to me.
OVERLORD commented 2026-02-05 09:43:58 +03:00
Author
Owner
Copy Link

@neothematrix commented on GitHub (Aug 27, 2024):

thanks @bo0tzz for linking my issue #12067 here, however, in my case, the "foreground" backup is working properly, only the background backup is not, and the reason seems to be that background backup process doesn't present the SSL Client certificate to the server.

@neothematrix commented on GitHub (Aug 27, 2024): thanks @bo0tzz for linking my issue #12067 here, however, in my case, the "foreground" backup is working properly, only the background backup is not, and the reason seems to be that background backup process doesn't present the SSL Client certificate to the server.
OVERLORD commented 2026-02-05 09:43:59 +03:00
Author
Owner
Copy Link

@neothematrix commented on GitHub (Aug 27, 2024):

to complement my comment: if I disable the mTLS requirement on cloudflare (that of course, I'd like to keep), background backup works flawlessly.

@neothematrix commented on GitHub (Aug 27, 2024): to complement my comment: if I disable the mTLS requirement on cloudflare (that of course, I'd like to keep), background backup works flawlessly.
OVERLORD commented 2026-02-05 09:44:00 +03:00
Author
Owner
Copy Link

@PottiMc commented on GitHub (Aug 27, 2024):

the problem seems to have solved itself for whatever reason unknown to me.

Not for me unfortunetaly.

@PottiMc commented on GitHub (Aug 27, 2024): > the problem seems to have solved itself for whatever reason unknown to me. Not for me unfortunetaly.
OVERLORD commented 2026-02-05 09:44:02 +03:00
Author
Owner
Copy Link

@mmomjian commented on GitHub (Aug 27, 2024):

thanks @bo0tzz for linking my issue #12067 here, however, in my case, the "foreground" backup is working properly, only the background backup is not, and the reason seems to be that background backup process doesn't present the SSL Client certificate to the server.

These problems are similar enough to be tracked in one issue.

@mmomjian commented on GitHub (Aug 27, 2024): > thanks @bo0tzz for linking my issue #12067 here, however, in my case, the "foreground" backup is working properly, only the background backup is not, and the reason seems to be that background backup process doesn't present the SSL Client certificate to the server. These problems are similar enough to be tracked in one issue.
OVERLORD commented 2026-02-05 09:44:03 +03:00
Author
Owner
Copy Link

@neothematrix commented on GitHub (Aug 27, 2024):

the problem seems to have solved itself for whatever reason unknown to me.

Not for me unfortunetaly.

@PottiMc are you also using mTLS on cloudflare?

@neothematrix commented on GitHub (Aug 27, 2024): > > the problem seems to have solved itself for whatever reason unknown to me. > > Not for me unfortunetaly. @PottiMc are you also using mTLS on cloudflare?
OVERLORD commented 2026-02-05 09:44:04 +03:00
Author
Owner
Copy Link

@PottiMc commented on GitHub (Aug 27, 2024):

No, I am not. Only a cloudflare tunnel to my instance. Everything worked fine before updating to 1.112. I still don't know why my issue was moved to discussions, but you can find all my comments here: #11873

@PottiMc commented on GitHub (Aug 27, 2024): No, I am not. Only a cloudflare tunnel to my instance. Everything worked fine before updating to 1.112. I still don't know why my issue was moved to discussions, but you can find all my comments here: #11873
OVERLORD commented 2026-02-05 09:44:05 +03:00
Author
Owner
Copy Link

@neothematrix commented on GitHub (Aug 29, 2024):

maybe @yjiang-c as owner of the mTLS PR implementation can have a look here?

@neothematrix commented on GitHub (Aug 29, 2024): maybe @yjiang-c as owner of the [mTLS PR implementation](10860) can have a look here?
OVERLORD commented 2026-02-05 09:44:06 +03:00
Author
Owner
Copy Link

@yjiang-c commented on GitHub (Aug 30, 2024):

I cannot reproduce the issue with my iPhone and Ngnix reverse proxy server. I checked the source code and cannot find any different between upload (POST) and download (GET), both use the same http client stack.

@yjiang-c commented on GitHub (Aug 30, 2024): I cannot reproduce the issue with my iPhone and Ngnix reverse proxy server. I checked the source code and cannot find any different between upload (POST) and download (GET), both use the same http client stack.
OVERLORD commented 2026-02-05 09:44:07 +03:00
Author
Owner
Copy Link

@PottiMc commented on GitHub (Aug 30, 2024):

As you can see in my discussion #11873 my issue got solved because cloudflare changed something in their Managed Firewall Ruleset. @sebpapillon maybe you are running a firewall aswell?

@PottiMc commented on GitHub (Aug 30, 2024): As you can see in my discussion #11873 my issue got solved because cloudflare changed something in their Managed Firewall Ruleset. @sebpapillon maybe you are running a firewall aswell?
OVERLORD commented 2026-02-05 09:44:10 +03:00
Author
Owner
Copy Link

@neothematrix commented on GitHub (Aug 31, 2024):

I cannot reproduce the issue with my iPhone and Ngnix reverse proxy server. I checked the source code and cannot find any different between upload (POST) and download (GET), both use the same http client stack.

thanks for checking @yjiang-c ! I'm configuring an iPhone to do the same test, just for clarity, did you check that the "automatic background backup" is working properly with mTLS certificate? because in my scenario, the foreground backup and the manual upload is working without any issue, but when the background upload triggers, I see the requests sent do not contain the mTLS certificate.

@neothematrix commented on GitHub (Aug 31, 2024): > I cannot reproduce the issue with my iPhone and Ngnix reverse proxy server. I checked the source code and cannot find any different between upload (POST) and download (GET), both use the same http client stack. thanks for checking @yjiang-c ! I'm configuring an iPhone to do the same test, just for clarity, did you check that the "automatic background backup" is working properly with mTLS certificate? because in my scenario, the foreground backup and the manual upload is working without any issue, but when the background upload triggers, I see the requests sent do not contain the mTLS certificate.
OVERLORD commented 2026-02-05 09:44:11 +03:00
Author
Owner
Copy Link

@yjiang-c commented on GitHub (Sep 1, 2024):

@neothematrix Yeah, I can confirm that background backup with SSL client certificate does not work. In addition, I think that background backup does not work with server with self signed certificate.

Background back runs in different process/zone in which http SSL options are not set properly. I will create MR to fix it.

@yjiang-c commented on GitHub (Sep 1, 2024): @neothematrix Yeah, I can confirm that background backup with SSL client certificate does not work. In addition, I think that background backup does not work with server with self signed certificate. Background back runs in different process/zone in which http SSL options are not set properly. I will create MR to fix it.
OVERLORD commented 2026-02-05 09:44:13 +03:00
Author
Owner
Copy Link

@neothematrix commented on GitHub (Sep 1, 2024):

Thanks a lot @yjiang-c that's really appreciated!

While you're at it, any chance you could look into a similar issue? https://github.com/immich-app/immich/issues/5553#issuecomment-2322936058

However, not sure it's very easy to fix and it's probably platform dependent, it looks like the same SSL options are not passed to external players used to playback videos.

Again, thank you so much and I'll be very happy to test anything that need to be tested!

@neothematrix commented on GitHub (Sep 1, 2024): Thanks a lot @yjiang-c that's really appreciated! While you're at it, any chance you could look into a similar issue? https://github.com/immich-app/immich/issues/5553#issuecomment-2322936058 However, not sure it's very easy to fix and it's probably platform dependent, it looks like the same SSL options are not passed to external players used to playback videos. Again, thank you so much and I'll be very happy to test anything that need to be tested!
OVERLORD commented 2026-02-05 09:44:15 +03:00
Author
Owner
Copy Link

@yjiang-c commented on GitHub (Sep 1, 2024):

@neothematrix #5553 is not easy to be resolved. The video player used in immich mobile is a flutter plugin, which does not support customised HTTPS options. In immich side, we can only fix it by totally replacing the video player and rewrite a new video player from scratch.

@yjiang-c commented on GitHub (Sep 1, 2024): @neothematrix [#5553 ](https://github.com/immich-app/immich/issues/5553#issuecomment-2322936058) is not easy to be resolved. The video player used in immich mobile is a flutter plugin, which does not support customised HTTPS options. In immich side, we can only fix it by totally replacing the video player and rewrite a new video player from scratch.
OVERLORD commented 2026-02-05 09:44:16 +03:00
Author
Owner
Copy Link

@neothematrix commented on GitHub (Sep 1, 2024):

@yjiang-c thanks a lot for the PR, hopefully it will be accepted and merged soon!
thanks a lot for also checking #5553 I had a feeling it was not that easy, perhaps we need @alextran1502 attention there, however, the easy workaround is using webbrowser instead of Immich App to playback the videos.

@neothematrix commented on GitHub (Sep 1, 2024): @yjiang-c thanks a lot for the PR, hopefully it will be accepted and merged soon! thanks a lot for also checking #5553 I had a feeling it was not that easy, perhaps we need @alextran1502 attention there, however, the easy workaround is using webbrowser instead of Immich App to playback the videos.
OVERLORD commented 2026-02-05 09:44:17 +03:00
Author
Owner
Copy Link

@neothematrix commented on GitHub (Sep 5, 2024):

tested with the updated app on Android and now it's working flawlessly with Client Certificate, thanks a lot @yjiang-c and @alextran1502 !
now to fix #5553 :)

@neothematrix commented on GitHub (Sep 5, 2024): tested with the updated app on Android and now it's working flawlessly with Client Certificate, thanks a lot @yjiang-c and @alextran1502 ! now to fix #5553 :)
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
accessibility
changelog:enhancement
changelog:security
changelog:skip
changelog:translation
cli
date-time
dependencies
documentation
external-library
format
good first issue
mobile-beta
mobile-beta
mobile-beta
needs-answer
nice to have
pull-request
Mirrored from GitHub Pull Request
 sharing
tech-debt
📱mobile
🖥️web
🗄️server
🧠machine-learning
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
OVERLORD
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: immich-app/immich#4101
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?