[FEATURE] Support backup photos and videos that were uploaded on iCloud #208

Closed
opened 2026-02-04 18:44:42 +03:00 by OVERLORD · 15 comments
Owner

Originally created by @Sixth2538 on GitHub (Aug 16, 2022).

Describe the bug
When backing up 30,000 photo and video assets from iPhone, uploading is stuck at after 3,000 uploads. Upload status will also show progress of uploading, but number of "Backup" photo does not increase.

Optimise iPhone storage option is selected on this device.

However, newest photos will get uploaded as normal. Once the new photos finish uploading, uploading progress will continue but older photo will not get backed up.

Task List

Please complete the task list below. We need this information to help us reproduce the bug or point out problems in your setup. You are not providing enough info may delay our effort to help you.

  • [Y] I have read thoroughly the README setup and installation instructions.
  • [Y] I have included my docker-compose file.
  • [Y] I have included my redacted .env file.
  • [Y] I have included information on my machine, and environment.

Docker Compose:

version: "3.8"

services:
immich-server:
image: altran1502/immich-server:release
entrypoint: ["/bin/sh", "./start-server.sh"]
volumes:
- ${UPLOAD_LOCATION}:/usr/src/app/upload
env_file:
- .env
environment:
- NODE_ENV=production
depends_on:
- redis
- database
restart: always
networks:
- traefik-public

immich-microservices:
image: altran1502/immich-server:release
entrypoint: ["/bin/sh", "./start-microservices.sh"]
volumes:
- ${UPLOAD_LOCATION}:/usr/src/app/upload
env_file:
- .env
environment:
- NODE_ENV=production
depends_on:
- redis
- database
restart: always
networks:
- traefik-public

immich-machine-learning:
image: altran1502/immich-machine-learning:release
entrypoint: ["/bin/sh", "./entrypoint.sh"]
volumes:
- ${UPLOAD_LOCATION}:/usr/src/app/upload
env_file:
- .env
environment:
- NODE_ENV=production
depends_on:
- database
restart: always
networks:
- traefik-public

immich-web:
image: altran1502/immich-web:release
entrypoint: ["/bin/sh", "./entrypoint.sh"]
env_file:
- .env
restart: always
networks:
- traefik-public

redis:
container_name: immich_redis
image: redis:6.2
restart: always
networks:
- traefik-public

database:
container_name: immich_postgres
image: postgres:14
env_file:
- .env
environment:
POSTGRES_PASSWORD: ${DB_PASSWORD}
POSTGRES_USER: ${DB_USERNAME}
POSTGRES_DB: ${DB_DATABASE_NAME}
PG_DATA: /var/lib/postgresql/data
volumes:
- [local bind volume]:/var/lib/postgresql/data
restart: always
networks:
- traefik-public

immich-proxy:
container_name: immich_proxy
image: altran1502/immich-proxy:release
ports:
- 2283:80
logging:
driver: none
depends_on:
- immich-server
labels:
- traefik.enable=true
- traefik.docker.network=traefik-public
- traefik.http.routers.photoview.rule=Host(photo.xxx.xxx)
- traefik.http.routers.photoview.entrypoints=https
- traefik.http.routers.photoview.tls=true
- traefik.http.routers.photoview.tls.certresolver=le
restart: always
networks:
- traefik-public

networks:
traefik-public:
external: true

Environmental Variables:

DB_HOSTNAME=immich_postgres
DB_USERNAME=xxx
DB_PASSWORD=xxx
DB_DATABASE_NAME=immich
REDIS_HOSTNAME=immich_redis
UPLOAD_LOCATION=[local bind volume]
JWT_SECRET=xxx
ENABLE_MAPBOX=true
MAPBOX_KEY=xxx
VITE_SERVER_ENDPOINT=https://photo.xxx.xxx/api

To Reproduce

  1. Open Immich app on iOS
  2. Click back up
  3. Select "Recent (All)" album
  4. Start backup

Expected behavior
Upload all 30,000 assets to server. Continue uploading older photo as shown on progress bar.

Screenshots
IMG_1150

System

  • Phone OS [iOS, Android]: 15.6
  • Server Version: 1.22.0
  • Mobile App Version: 1.22.0 build 42

Additional context
Add any other context about the problem here.

Originally created by @Sixth2538 on GitHub (Aug 16, 2022). **Describe the bug** When backing up 30,000 photo and video assets from iPhone, uploading is stuck at after 3,000 uploads. Upload status will also show progress of uploading, but number of "Backup" photo does not increase. Optimise iPhone storage option is selected on this device. However, newest photos will get uploaded as normal. Once the new photos finish uploading, uploading progress will continue but older photo will not get backed up. **Task List** *Please complete the task list below. We need this information to help us reproduce the bug or point out problems in your setup. You are not providing enough info may delay our effort to help you.* - [Y] I have read thoroughly the README setup and installation instructions. - [Y] I have included my `docker-compose` file. - [Y] I have included my redacted `.env` file. - [Y] I have included information on my machine, and environment. Docker Compose: version: "3.8" services: immich-server: image: altran1502/immich-server:release entrypoint: ["/bin/sh", "./start-server.sh"] volumes: - ${UPLOAD_LOCATION}:/usr/src/app/upload env_file: - .env environment: - NODE_ENV=production depends_on: - redis - database restart: always networks: - traefik-public immich-microservices: image: altran1502/immich-server:release entrypoint: ["/bin/sh", "./start-microservices.sh"] volumes: - ${UPLOAD_LOCATION}:/usr/src/app/upload env_file: - .env environment: - NODE_ENV=production depends_on: - redis - database restart: always networks: - traefik-public immich-machine-learning: image: altran1502/immich-machine-learning:release entrypoint: ["/bin/sh", "./entrypoint.sh"] volumes: - ${UPLOAD_LOCATION}:/usr/src/app/upload env_file: - .env environment: - NODE_ENV=production depends_on: - database restart: always networks: - traefik-public immich-web: image: altran1502/immich-web:release entrypoint: ["/bin/sh", "./entrypoint.sh"] env_file: - .env restart: always networks: - traefik-public redis: container_name: immich_redis image: redis:6.2 restart: always networks: - traefik-public database: container_name: immich_postgres image: postgres:14 env_file: - .env environment: POSTGRES_PASSWORD: ${DB_PASSWORD} POSTGRES_USER: ${DB_USERNAME} POSTGRES_DB: ${DB_DATABASE_NAME} PG_DATA: /var/lib/postgresql/data volumes: - [local bind volume]:/var/lib/postgresql/data restart: always networks: - traefik-public immich-proxy: container_name: immich_proxy image: altran1502/immich-proxy:release ports: - 2283:80 logging: driver: none depends_on: - immich-server labels: - traefik.enable=true - traefik.docker.network=traefik-public - traefik.http.routers.photoview.rule=Host(`photo.xxx.xxx`) - traefik.http.routers.photoview.entrypoints=https - traefik.http.routers.photoview.tls=true - traefik.http.routers.photoview.tls.certresolver=le restart: always networks: - traefik-public networks: traefik-public: external: true Environmental Variables: DB_HOSTNAME=immich_postgres DB_USERNAME=xxx DB_PASSWORD=xxx DB_DATABASE_NAME=immich REDIS_HOSTNAME=immich_redis UPLOAD_LOCATION=[local bind volume] JWT_SECRET=xxx ENABLE_MAPBOX=true MAPBOX_KEY=xxx VITE_SERVER_ENDPOINT=https://photo.xxx.xxx/api **To Reproduce** 1. Open Immich app on iOS 2. Click back up 3. Select "Recent (All)" album 4. Start backup **Expected behavior** Upload all 30,000 assets to server. Continue uploading older photo as shown on progress bar. **Screenshots** ![IMG_1150](https://user-images.githubusercontent.com/111109779/184821905-8566f11c-f8b6-46a3-b5bc-724fe3a28ecc.png) **System** - Phone OS [iOS, Android]: `15.6` - Server Version: `1.22.0` - Mobile App Version: `1.22.0 build 42` **Additional context** Add any other context about the problem here.
Author
Owner

@MalteKiefer commented on GitHub (Aug 16, 2022):

I think you must disable the Optimize system.
The images must be on the device

@MalteKiefer commented on GitHub (Aug 16, 2022): I think you must disable the Optimize system. The images must be on the device
Author
Owner

@20pins commented on GitHub (Aug 16, 2022):

I think you must disable the Optimize system.

The images must be on the device

I wonder how many other iOS issues are a result of this. I also have Optimize enabled. I currently have 3570 of 13k items uploaded.

@20pins commented on GitHub (Aug 16, 2022): > I think you must disable the Optimize system. > > The images must be on the device I wonder how many other iOS issues are a result of this. I also have Optimize enabled. I currently have 3570 of 13k items uploaded.
Author
Owner

@alextran1502 commented on GitHub (Aug 16, 2022):

If the assets are on iCloud, they won't be uploaded at this stage. I will look into this issue when I implemented Live Photo feature

@alextran1502 commented on GitHub (Aug 16, 2022): If the assets are on iCloud, they won't be uploaded at this stage. I will look into this issue when I implemented Live Photo feature
Author
Owner

@Sixth2538 commented on GitHub (Aug 16, 2022):

Thank you for the info and rapid development of the app.

Seems like choosing optimizing storage is the root of the problem. Looking forward to see if this can be sorted out. Seems like it is by design to tie people down with the system.

I have tried various method to download my entire library and there really is no easy way to do it. If this feature can be implemented, it could be a game changer for many iOS users.

@Sixth2538 commented on GitHub (Aug 16, 2022): Thank you for the info and rapid development of the app. Seems like choosing optimizing storage is the root of the problem. Looking forward to see if this can be sorted out. Seems like it is by design to tie people down with the system. I have tried various method to download my entire library and there really is no easy way to do it. If this feature can be implemented, it could be a game changer for many iOS users.
Author
Owner

@esmondmissen commented on GitHub (Aug 18, 2022):

Oddly enough I have optimise storage on and the backup is working fine. I've noticed it's a lot slower, but still works for me. I am on iOS 16 however.

@esmondmissen commented on GitHub (Aug 18, 2022): Oddly enough I have optimise storage on and the backup is working fine. I've noticed it's a lot slower, but still works for me. I am on iOS 16 however.
Author
Owner

@MalteKiefer commented on GitHub (Aug 18, 2022):

How many photos do you have?

@MalteKiefer commented on GitHub (Aug 18, 2022): How many photos do you have?
Author
Owner

@esmondmissen commented on GitHub (Aug 18, 2022):

About 17k, I definitely don’t have enough storage on my device to have them all locally.

@esmondmissen commented on GitHub (Aug 18, 2022): About 17k, I definitely don’t have enough storage on my device to have them all locally.
Author
Owner

@2600box commented on GitHub (Aug 21, 2022):

I am on 15.1 and no photos from my Photos.app show up in Immich. I have the icloud optimised storage turned on.

Interesting if iOS16 is in fact the determining factor...

@2600box commented on GitHub (Aug 21, 2022): I am on 15.1 and no photos from my Photos.app show up in Immich. I have the icloud optimised storage turned on. Interesting if iOS16 is in fact the determining factor...
Author
Owner

@Sixth2538 commented on GitHub (Sep 13, 2022):

Happy to report back that iOS 16 backup function is working and backing up all photo and video assets slowly. Even with optimised storage turned on.

@Sixth2538 commented on GitHub (Sep 13, 2022): Happy to report back that iOS 16 backup function is working and backing up all photo and video assets slowly. Even with optimised storage turned on.
Author
Owner

@alextran1502 commented on GitHub (Sep 13, 2022):

@Sixth2538 WOOO! This is good news! I guess we can reduce additional code thanks to iOS 16 :)

@alextran1502 commented on GitHub (Sep 13, 2022): @Sixth2538 WOOO! This is good news! I guess we can reduce additional code thanks to iOS 16 :)
Author
Owner

@gorootde commented on GitHub (Nov 12, 2022):

Any updates on this one? I am on 16.1.1 and still no photos show up at all (optimize storage enabled)

@gorootde commented on GitHub (Nov 12, 2022): Any updates on this one? I am on 16.1.1 and still no photos show up at all (optimize storage enabled)
Author
Owner

@jwinut commented on GitHub (Nov 14, 2022):

I just started using immich. great product!

I also encounter this problem from iCloud Shared Album. I am subscribing to the albums and don't have those images on my iPhone (images not taken by me and I disable iCloud Photos on my device/ Shared Album enable).

I notice that the upload stuck at 100% on the same file name but when I Cancel the backup and start again, immich app backup 3-5 images before stuck at 100% again.

I kept smashing Cancel and Start Backup until my Shared Album is backed up (~100 images + some video).

my device: iPhone X
iOS 15.7.1

@jwinut commented on GitHub (Nov 14, 2022): I just started using immich. great product! I also encounter this problem from iCloud Shared Album. I am subscribing to the albums and don't have those images on my iPhone (images not taken by me and I disable iCloud Photos on my device/ Shared Album enable). I notice that the upload stuck at 100% on the same file name but when I Cancel the backup and start again, immich app backup 3-5 images before stuck at 100% again. I kept smashing Cancel and Start Backup until my Shared Album is backed up (~100 images + some video). my device: iPhone X iOS 15.7.1
Author
Owner

@alextran1502 commented on GitHub (Nov 14, 2022):

@jwinut There are some differences in the internal APIs of iOS in term of accessing the file system between version 15 and 16 I believe. Would it be possible for your phone to update to iOS 16?

@alextran1502 commented on GitHub (Nov 14, 2022): @jwinut There are some differences in the internal APIs of iOS in term of accessing the file system between version 15 and 16 I believe. Would it be possible for your phone to update to iOS 16?
Author
Owner

@jwinut commented on GitHub (Nov 14, 2022):

@alextran1502 it is possible. Would it help you with your development? if yes then I will.

@jwinut commented on GitHub (Nov 14, 2022): @alextran1502 it is possible. Would it help you with your development? if yes then I will.
Author
Owner

@jwinut commented on GitHub (Nov 15, 2022):

Updated to iOS 16.1.1. same behavior. need to cancel and start backup multiple times to complete remaining backup.

@jwinut commented on GitHub (Nov 15, 2022): Updated to iOS 16.1.1. same behavior. need to cancel and start backup multiple times to complete remaining backup.
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: immich-app/immich#208