immich-app/immich
1
0
Fork 0
mirror of https://github.com/immich-app/immich.git synced 2026-02-05 00:30:57 +03:00
Code Issues 11 Packages Projects Releases 10 Wiki Activity

[BUG] all services logging errors in v1.84 and can not get web or mobile to load #1549

New Issue
Closed
opened 2026-02-05 02:20:00 +03:00 by OVERLORD · 0 comments
OVERLORD commented 2026-02-05 02:20:00 +03:00
Owner
Copy Link

Originally created by @ghost on GitHub (Nov 2, 2023).

The bug

getting lots of errors after upgrading to 1.84

worked fine to start with, but updated the iOS app on the phone and now just getting server 500 errors in the web and mobile app.. most errors seem to come from typesets

         Name                        Command               State                  Ports                
-------------------------------------------------------------------------------------------------------
immich_machine_learning   tini -- ./start.sh               Up                                          
immich_microservices      tini -- /bin/sh start.sh m ...   Up      3001/tcp                            
immich_postgres           docker-entrypoint.sh postgres    Up      5432/tcp                            
immich_proxy              /docker-entrypoint.sh ngin ...   Up      0.0.0.0:80->8080/tcp,:::80->8080/tcp
immich_redis              docker-entrypoint.sh redis ...   Up      6379/tcp                            
immich_server             tini -- /bin/sh start.sh i ...   Up      3001/tcp                            
immich_typesense          /opt/typesense-server            Up      8108/tcp                            
immich_web                tini -- /bin/sh entrypoint.sh    Up      3000/tcp                            
root@immich:~/immich-app# docker ps
CONTAINER ID   IMAGE                                                COMMAND                  CREATED         STATUS          PORTS                                   NAMES
f16fdfa39499   ghcr.io/immich-app/immich-proxy:release              "/docker-entrypoint.…"   8 minutes ago   Up 41 seconds   0.0.0.0:80->8080/tcp, :::80->8080/tcp   immich_proxy
183368e2782e   ghcr.io/immich-app/immich-server:release             "tini -- /bin/sh sta…"   8 minutes ago   Up 42 seconds   3001/tcp                                immich_server
2b9ad892b652   ghcr.io/immich-app/immich-server:release             "tini -- /bin/sh sta…"   8 minutes ago   Up 42 seconds   3001/tcp                                immich_microservices
f291c0c12a3a   redis:6.2-alpine                                     "docker-entrypoint.s…"   8 minutes ago   Up 42 seconds   6379/tcp                                immich_redis
090b09d021ff   ghcr.io/immich-app/immich-machine-learning:release   "tini -- ./start.sh"     8 minutes ago   Up 42 seconds                                           immich_machine_learning
0eae301a664c   typesense/typesense:0.24.1                           "/opt/typesense-serv…"   8 minutes ago   Up 42 seconds   8108/tcp                                immich_typesense
50138c9bd5e4   ghcr.io/immich-app/immich-web:release                "tini -- /bin/sh ent…"   8 minutes ago   Up 42 seconds   3000/tcp                                immich_web
1076f28b948e   postgres:14-alpine                                   "docker-entrypoint.s…"   8 minutes ago   Up 42 seconds   5432/tcp                                immich_postgres

after some time

         Name                        Command                State                    Ports                
----------------------------------------------------------------------------------------------------------
immich_machine_learning   tini -- ./start.sh               Up                                             
immich_microservices      tini -- /bin/sh start.sh m ...   Exit 134                                       
immich_postgres           docker-entrypoint.sh postgres    Exit 1                                         
immich_proxy              /docker-entrypoint.sh ngin ...   Up         0.0.0.0:80->8080/tcp,:::80->8080/tcp
immich_redis              docker-entrypoint.sh redis ...   Up         6379/tcp                            
immich_server             tini -- /bin/sh start.sh i ...   Exit 1                                         
immich_typesense          /opt/typesense-server            Up         8108/tcp                            
immich_web                tini -- /bin/sh entrypoint.sh    Up         3000/tcp                            
root@immich:~/immich-app# docker ps
CONTAINER ID   IMAGE                                                COMMAND                  CREATED          STATUS                         PORTS                                   NAMES
f16fdfa39499   ghcr.io/immich-app/immich-proxy:release              "/docker-entrypoint.…"   20 minutes ago   Up 12 minutes                  0.0.0.0:80->8080/tcp, :::80->8080/tcp   immich_proxy
183368e2782e   ghcr.io/immich-app/immich-server:release             "tini -- /bin/sh sta…"   20 minutes ago   Up 9 minutes                   3001/tcp                                immich_server
2b9ad892b652   ghcr.io/immich-app/immich-server:release             "tini -- /bin/sh sta…"   20 minutes ago   Up 12 minutes                  3001/tcp                                immich_microservices
f291c0c12a3a   redis:6.2-alpine                                     "docker-entrypoint.s…"   20 minutes ago   Up 12 minutes                  6379/tcp                                immich_redis
090b09d021ff   ghcr.io/immich-app/immich-machine-learning:release   "tini -- ./start.sh"     20 minutes ago   Up 12 minutes                                                          immich_machine_learning
0eae301a664c   typesense/typesense:0.24.1                           "/opt/typesense-serv…"   20 minutes ago   Up 12 minutes                  8108/tcp                                immich_typesense
50138c9bd5e4   ghcr.io/immich-app/immich-web:release                "tini -- /bin/sh ent…"   20 minutes ago   Up 12 minutes                  3000/tcp                                immich_web
1076f28b948e   postgres:14-alpine                                   "docker-entrypoint.s…"   20 minutes ago   Restarting (1) 7 minutes ago                                           immich_postgres

The OS that Immich Server is running on

ubuntu 22

Version of Immich Server

1.84

Version of Immich Mobile App

latest today

Platform with the issue

  • Server
  • Web
  • Mobile

Your docker-compose.yml content

default

Your .env content

default

Reproduction steps

1.
2.
3.
...

Additional information

immich_typesense           | E20231102 09:13:12.495561   425 collection.cpp:364] Write to disk failed. Will restore old document
immich_typesense           | E20231102 09:13:12.495658   425 collection.cpp:364] Write to disk failed. Will restore old document
immich_typesense           | E20231102 09:13:12.495747   425 collection.cpp:364] Write to disk failed. Will restore old document
immich_typesense           | E20231102 09:13:12.495829   425 collection.cpp:364] Write to disk failed. Will restore old document
immich_typesense           | E20231102 09:13:12.495914   425 collection.cpp:364] Write to disk failed. Will restore old document
immich_typesense           | E20231102 09:13:12.495994   425 collection.cpp:364] Write to disk failed. Will restore old document
immich_typesense           | E20231102 09:13:12.496078   425 collection.cpp:364] Write to disk failed. Will restore old document
immich_typesense           | E20231102 09:13:12.496163   425 collection.cpp:364] Write to disk failed. Will restore old document
immich_typesense           | E20231102 09:13:12.496245   425 collection.cpp:364] Write to disk failed. Will restore old document
immich_typesense           | E20231102 09:13:12.496345   425 collection.cpp:364] Write to disk failed. Will restore old document
immich_typesense           | E20231102 09:13:12.496434   425 collection.cpp:364] Write to disk failed. Will restore old document
immich_typesense           | E20231102 09:13:12.496521   425 collection.cpp:364] Write to disk failed. Will restore old document
immich_typesense           | E20231102 09:13:12.496608   425 collection.cpp:364] Write to disk failed. Will restore old document
immich_typesense           | E20231102 09:13:12.496691   425 collection.cpp:364] Write to disk failed. Will restore old document
immich_typesense           | E20231102 09:13:12.496780   425 collection.cpp:364] Write to disk failed. Will restore old document
immich_typesense           | E20231102 09:13:12.496865   425 collection.cpp:364] Write to disk failed. Will restore old document
immich_typesense           | E20231102 09:13:12.496949   425 collection.cpp:364] Write to disk failed. Will restore old document
immich_typesense           | E20231102 09:13:12.497061   425 collection.cpp:364] Write to disk failed. Will restore old document
immich_typesense           | E20231102 09:13:12.497184   425 collection.cpp:364] Write to disk failed. Will restore old document
immich_typesense           | E20231102 09:13:12.497293   425 collection.cpp:364] Write to disk failed. Will restore old document
immich_typesense           | E20231102 09:13:12.497406   425 collection.cpp:364] Write to disk failed. Will restore old document
immich_typesense           | E20231102 09:13:12.497514   425 collection.cpp:364] Write to disk failed. Will restore old document
immich_typesense           | E20231102 09:13:12.497628   425 collection.cpp:364] Write to disk failed. Will restore old document
immich_typesense           | E20231102 09:13:12.497740   425 collection.cpp:364] Write to disk failed. Will restore old document
immich_server              | Request #1698916334394: Request to Node 0 failed due to "undefined Request failed with HTTP code 503 | Server said: Not Ready or Lagging"
immich_server              | Request #1698916334394: Sleeping for 4s and then retrying request...
immich_typesense           | E20231102 09:13:14.599668   358 raft_server.cpp:624] 1988 queued writes > healthy read lag of 1000
immich_typesense           | E20231102 09:13:14.599679   358 raft_server.cpp:636] 1988 queued writes > healthy write lag of 500
immich_microservices       | Request #1698916347097: Request to Node 0 failed due to "undefined Request failed with HTTP code 503 | Server said: Not Ready or Lagging"
immich_microservices       | Request #1698916347097: Sleeping for 4s and then retrying request...
immich_typesense           | W20231102 09:13:15.233565   425 index.cpp:5406] Error while removing field `embedding` from document, message: Label not found
immich_typesense           | W20231102 09:13:16.940913   425 index.cpp:5406] Error while removing field `embedding` from document, message: Label not found
immich_server              | /usr/src/app/node_modules/typesense/lib/Typesense/Errors/TypesenseError.js:23
immich_server              |         var _this = _super.call(this, message) || this;
immich_server              |                            ^
immich_server              | 
immich_server              | ServerError: Request failed with HTTP code 503 | Server said: Not Ready or Lagging
immich_server              |     at ServerError.TypesenseError [as constructor] (/usr/src/app/node_modules/typesense/lib/Typesense/Errors/TypesenseError.js:23:28)
immich_server              |     at new ServerError (/usr/src/app/node_modules/typesense/lib/Typesense/Errors/ServerError.js:25:42)
immich_server              |     at ApiCall.customErrorForResponse (/usr/src/app/node_modules/typesense/lib/Typesense/ApiCall.js:376:21)
immich_server              |     at /usr/src/app/node_modules/typesense/lib/Typesense/ApiCall.js:225:58
immich_server              |     at step (/usr/src/app/node_modules/typesense/lib/Typesense/ApiCall.js:33:23)
immich_server              |     at Object.next (/usr/src/app/node_modules/typesense/lib/Typesense/ApiCall.js:14:53)
immich_server              |     at step (/usr/src/app/node_modules/typesense/lib/Typesense/ApiCall.js:18:139)
immich_server              |     at Object.next (/usr/src/app/node_modules/typesense/lib/Typesense/ApiCall.js:14:53)
immich_server              |     at fulfilled (/usr/src/app/node_modules/typesense/lib/Typesense/ApiCall.js:5:58)
immich_server              |     at process.processTicksAndRejections (node:internal/process/task_queues:95:5) {
immich_server              |   httpStatus: 503
immich_server              | }
immich_server              | 
immich_server              | Node.js v20.8.1
immich_typesense           | W20231102 09:13:18.530943   425 index.cpp:5406] Error while removing field `embedding` from document, message: Label not found
immich_typesense           | W20231102 09:13:18.838958   425 index.cpp:5406] Error while removing field `embedding` from document, message: Label not found
immich_typesense           | W20231102 09:13:19.053009   425 index.cpp:5406] Error while removing field `embedding` from document, message: Label not found
immich_microservices       | Request #1698916347097: Request to Node 0 failed due to "undefined Request failed with HTTP code 503 | Server said: Not Ready or Lagging"
immich_microservices       | Request #1698916347097: Sleeping for 4s and then retrying request...
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [NestFactory] Starting Nest application...
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [InstanceLoader] TypeOrmModule dependencies initialized +32ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [InstanceLoader] BullModule dependencies initialized +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [InstanceLoader] ConfigHostModule dependencies initialized +1ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [InstanceLoader] DiscoveryModule dependencies initialized +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [InstanceLoader] ScheduleModule dependencies initialized +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [InstanceLoader] ConfigModule dependencies initialized +4ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [InstanceLoader] BullModule dependencies initialized +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [InstanceLoader] ScheduleModule dependencies initialized +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [InstanceLoader] BullModule dependencies initialized +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [InstanceLoader] TypeOrmCoreModule dependencies initialized +157ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [InstanceLoader] TypeOrmModule dependencies initialized +1ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [InstanceLoader] TypeOrmModule dependencies initialized +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [InstanceLoader] InfraModule dependencies initialized +2ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [InstanceLoader] AppModule dependencies initialized +5ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [InstanceLoader] DomainModule dependencies initialized +13ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RoutesResolver] ActivityController {/activity}: +106ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/activity, GET} route +3ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/activity/statistics, GET} route +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/activity, POST} route +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/activity/:id, DELETE} route +1ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RoutesResolver] AssetController {/asset}: +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/asset/map-marker, GET} route +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/asset/memory-lane, GET} route +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/asset/random, GET} route +1ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/asset/download/info, POST} route +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/asset/download/archive, POST} route +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/asset/download/:id, POST} route +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/asset/statistics, GET} route +1ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/asset/time-buckets, GET} route +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/asset/time-bucket, GET} route +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/asset/jobs, POST} route +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/asset, PUT} route +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/asset, DELETE} route +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/asset/restore, POST} route +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/asset/trash/empty, POST} route +1ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/asset/trash/restore, POST} route +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/asset/stack/parent, PUT} route +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/asset/:id, PUT} route +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RoutesResolver] AssetController {/asset}: +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/asset/upload, POST} route +1ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/asset/import, POST} route +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/asset/file/:id, GET} route +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/asset/thumbnail/:id, GET} route +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/asset/curated-objects, GET} route +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/asset/curated-locations, GET} route +1ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/asset/search-terms, GET} route +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/asset/search, POST} route +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/asset, GET} route +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/asset/:deviceId, GET} route +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/asset/assetById/:id, GET} route +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/asset/exist, POST} route +2ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/asset/bulk-upload-check, POST} route +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RoutesResolver] AppController {/}: +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/refresh-config, POST} route +1ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RoutesResolver] AlbumController {/album}: +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/album/count, GET} route +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/album, GET} route +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/album, POST} route +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/album/:id, GET} route +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/album/:id, PATCH} route +1ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/album/:id, DELETE} route +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/album/:id/assets, PUT} route +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/album/:id/assets, DELETE} route +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/album/:id/users, PUT} route +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/album/:id/user/:userId, DELETE} route +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RoutesResolver] APIKeyController {/api-key}: +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/api-key, POST} route +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/api-key, GET} route +1ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/api-key/:id, GET} route +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/api-key/:id, PUT} route +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/api-key/:id, DELETE} route +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RoutesResolver] AuditController {/audit}: +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/audit/deletes, GET} route +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/audit/file-report, GET} route +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/audit/file-report/checksum, POST} route +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/audit/file-report/fix, POST} route +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RoutesResolver] AuthController {/auth}: +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/auth/login, POST} route +1ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/auth/admin-sign-up, POST} route +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/auth/devices, GET} route +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/auth/devices, DELETE} route +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/auth/devices/:id, DELETE} route +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/auth/validateToken, POST} route +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/auth/change-password, POST} route +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/auth/logout, POST} route +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RoutesResolver] JobController {/jobs}: +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/jobs, GET} route +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/jobs/:id, PUT} route +1ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RoutesResolver] LibraryController {/library}: +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/library, GET} route +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/library, POST} route +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/library/:id, PUT} route +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/library/:id, GET} route +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/library/:id, DELETE} route +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/library/:id/statistics, GET} route +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/library/:id/scan, POST} route +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/library/:id/removeOffline, POST} route +1ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RoutesResolver] OAuthController {/oauth}: +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/oauth/mobile-redirect, GET} route +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/oauth/config, POST} route +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/oauth/authorize, POST} route +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/oauth/callback, POST} route +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/oauth/link, POST} route +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/oauth/unlink, POST} route +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RoutesResolver] PartnerController {/partner}: +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/partner, GET} route +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/partner/:id, POST} route +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/partner/:id, DELETE} route +1ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RoutesResolver] SearchController {/search}: +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/search/person, GET} route +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/search, GET} route +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/search/explore, GET} route +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RoutesResolver] ServerInfoController {/server-info}: +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/server-info, GET} route +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/server-info/ping, GET} route +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/server-info/version, GET} route +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/server-info/features, GET} route +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/server-info/theme, GET} route +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/server-info/config, GET} route +1ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/server-info/stats, GET} route +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/server-info/media-types, GET} route +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RoutesResolver] SharedLinkController {/shared-link}: +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/shared-link, GET} route +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/shared-link/me, GET} route +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/shared-link/:id, GET} route +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/shared-link, POST} route +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/shared-link/:id, PATCH} route +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/shared-link/:id, DELETE} route +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/shared-link/:id/assets, PUT} route +1ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/shared-link/:id/assets, DELETE} route +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RoutesResolver] SystemConfigController {/system-config}: +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/system-config, GET} route +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/system-config/defaults, GET} route +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/system-config, PUT} route +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/system-config/storage-template-options, GET} route +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RoutesResolver] TagController {/tag}: +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/tag, POST} route +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/tag, GET} route +1ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/tag/:id, GET} route +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/tag/:id, PATCH} route +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/tag/:id, DELETE} route +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/tag/:id/assets, GET} route +0ms
immich_server              | [Nest] 7  - 11/02/2023, 7:13:20 PM     LOG [RouterExplorer] Mapped {/tag/:id/assets, PUT} route +0ms
immich_typesense           | E20231102 09:17:27.830929   425 collection.cpp:340] Update to disk failed. Will restore old document
immich_typesense           | E20231102 09:17:27.833989   425 collection.cpp:340] Update to disk failed. Will restore old document
immich_typesense           | E20231102 09:17:27.845978   425 collection.cpp:340] Update to disk failed. Will restore old document
immich_typesense           | E20231102 09:17:27.852667   425 collection.cpp:340] Update to disk failed. Will restore old document
immich_typesense           | E20231102 09:17:27.856185   425 collection.cpp:340] Update to disk failed. Will restore old document
immich_typesense           | E20231102 09:17:27.864369   425 collection.cpp:340] Update to disk failed. Will restore old document
immich_microservices       | [Nest] 7  - 11/02/2023, 7:17:29 PM     LOG [NestFactory] Starting Nest application...
immich_microservices       | [Nest] 7  - 11/02/2023, 7:17:29 PM     LOG [InstanceLoader] TypeOrmModule dependencies initialized +34ms
immich_microservices       | [Nest] 7  - 11/02/2023, 7:17:29 PM     LOG [InstanceLoader] BullModule dependencies initialized +0ms
immich_microservices       | [Nest] 7  - 11/02/2023, 7:17:29 PM     LOG [InstanceLoader] ConfigHostModule dependencies initialized +0ms
immich_microservices       | [Nest] 7  - 11/02/2023, 7:17:29 PM     LOG [InstanceLoader] DiscoveryModule dependencies initialized +0ms
immich_microservices       | [Nest] 7  - 11/02/2023, 7:17:29 PM     LOG [InstanceLoader] ScheduleModule dependencies initialized +0ms
immich_microservices       | [Nest] 7  - 11/02/2023, 7:17:29 PM     LOG [InstanceLoader] ConfigModule dependencies initialized +5ms
immich_microservices       | [Nest] 7  - 11/02/2023, 7:17:29 PM     LOG [InstanceLoader] BullModule dependencies initialized +0ms
immich_microservices       | [Nest] 7  - 11/02/2023, 7:17:29 PM     LOG [InstanceLoader] BullModule dependencies initialized +0ms
immich_microservices       | /usr/src/app/node_modules/redis-parser/lib/parser.js:179
immich_microservices       |     return new ReplyError(string)
immich_microservices       |            ^
immich_microservices       | 
immich_microservices       | ReplyError: MISCONF Redis is configured to save RDB snapshots, but it is currently not able to persist on disk. Commands that may modify the data set are disabled, because this instance is configured to report errors during writes if RDB snapshotting fails (stop-writes-on-bgsave-error option). Please check the Redis logs for details about the RDB error.
immich_microservices       |     at parseError (/usr/src/app/node_modules/redis-parser/lib/parser.js:179:12)
immich_microservices       |     at parseType (/usr/src/app/node_modules/redis-parser/lib/parser.js:302:14) {
immich_microservices       |   command: {
immich_microservices       |     name: 'hset',
immich_microservices       |     args: [
immich_microservices       |       'immich_bull:thumbnailGeneration:meta',
immich_microservices       |       'opts.maxLenEvents',
immich_microservices       |       '10000'
immich_microservices       |     ]
immich_microservices       |   }
immich_microservices       | }
immich_microservices       | 
immich_microservices       | Node.js v20.8.1
immich_microservices exited with code 1
immich_redis               | 1:M 02 Nov 2023 09:17:30.015 * 100 changes in 300 seconds. Saving...
immich_redis               | 1:M 02 Nov 2023 09:17:30.015 * Background saving started by pid 18
immich_redis               | 18:C 02 Nov 2023 09:17:30.019 # Write error saving DB on disk: No space left on device
immich_redis               | 1:M 02 Nov 2023 09:17:30.116 # Background saving error
immich_server              | Request #1698916602115: Request to Node 0 failed due to "undefined Request failed with HTTP code 503 | Server said: Not Ready or Lagging"
immich_server              | Request #1698916602115: Sleeping for 4s and then retrying request...
immich_typesense           | W20231102 09:17:31.800505   425 index.cpp:5406] Error while removing field `embedding` from document, message: Label not found
immich_microservices       | [Nest] 7  - 11/02/2023, 7:17:31 PM     LOG [NestFactory] Starting Nest application...
immich_microservices       | [Nest] 7  - 11/02/2023, 7:17:32 PM     LOG [InstanceLoader] TypeOrmModule dependencies initialized +35ms
immich_microservices       | [Nest] 7  - 11/02/2023, 7:17:32 PM     LOG [InstanceLoader] BullModule dependencies initialized +0ms
immich_microservices       | [Nest] 7  - 11/02/2023, 7:17:32 PM     LOG [InstanceLoader] ConfigHostModule dependencies initialized +1ms
immich_microservices       | [Nest] 7  - 11/02/2023, 7:17:32 PM     LOG [InstanceLoader] DiscoveryModule dependencies initialized +0ms
immich_microservices       | [Nest] 7  - 11/02/2023, 7:17:32 PM     LOG [InstanceLoader] ScheduleModule dependencies initialized +0ms
immich_microservices       | [Nest] 7  - 11/02/2023, 7:17:32 PM     LOG [InstanceLoader] ConfigModule dependencies initialized +5ms
immich_microservices       | [Nest] 7  - 11/02/2023, 7:17:32 PM     LOG [InstanceLoader] BullModule dependencies initialized +0ms
immich_microservices       | [Nest] 7  - 11/02/2023, 7:17:32 PM     LOG [InstanceLoader] BullModule dependencies initialized +0ms
immich_microservices       | /usr/src/app/node_modules/redis-parser/lib/parser.js:179
immich_microservices       |     return new ReplyError(string)
immich_microservices       |            ^
immich_microservices       | 
immich_microservices       | ReplyError: MISCONF Redis is configured to save RDB snapshots, but it is currently not able to persist on disk. Commands that may modify the data set are disabled, because this instance is configured to report errors during writes if RDB snapshotting fails (stop-writes-on-bgsave-error option). Please check the Redis logs for details about the RDB error.
immich_microservices       |     at parseError (/usr/src/app/node_modules/redis-parser/lib/parser.js:179:12)
immich_microservices       |     at parseType (/usr/src/app/node_modules/redis-parser/lib/parser.js:302:14) {
immich_microservices       |   command: {
immich_microservices       |     name: 'hset',
immich_microservices       |     args: [
immich_microservices       |       'immich_bull:thumbnailGeneration:meta',
immich_microservices       |       'opts.maxLenEvents',
immich_microservices       |       '10000'
immich_microservices       |     ]
immich_microservices       |   }
immich_microservices       | }
immich_microservices       | 
immich_microservices       | Node.js v20.8.1
immich_microservices exited with code 1
immich_typesense           | W20231102 09:17:32.621006   425 index.cpp:5406] Error while removing field `embedding` from document, message: Label not found
immich_server              | Request #1698916602115: Request to Node 0 failed due to "undefined Request failed with HTTP code 503 | Server said: Not Ready or Lagging"
immich_server              | Request #1698916602115: Sleeping for 4s and then retrying request...
immich_microservices       | [Nest] 7  - 11/02/2023, 7:17:34 PM     LOG [NestFactory] Starting Nest application...
immich_microservices       | [Nest] 7  - 11/02/2023, 7:17:34 PM     LOG [InstanceLoader] TypeOrmModule dependencies initialized +34ms
immich_microservices       | [Nest] 7  - 11/02/2023, 7:17:34 PM     LOG [InstanceLoader] BullModule dependencies initialized +0ms
immich_microservices       | [Nest] 7  - 11/02/2023, 7:17:34 PM     LOG [InstanceLoader] ConfigHostModule dependencies initialized +1ms
immich_microservices       | [Nest] 7  - 11/02/2023, 7:17:34 PM     LOG [InstanceLoader] DiscoveryModule dependencies initialized +0ms
immich_microservices       | [Nest] 7  - 11/02/2023, 7:17:34 PM     LOG [InstanceLoader] ScheduleModule dependencies initialized +0ms
immich_microservices       | [Nest] 7  - 11/02/2023, 7:17:34 PM     LOG [InstanceLoader] ConfigModule dependencies initialized +5ms
immich_microservices       | [Nest] 7  - 11/02/2023, 7:17:34 PM     LOG [InstanceLoader] BullModule dependencies initialized +0ms
immich_microservices       | [Nest] 7  - 11/02/2023, 7:17:34 PM     LOG [InstanceLoader] BullModule dependencies initialized +0ms
immich_microservices       | /usr/src/app/node_modules/redis-parser/lib/parser.js:179
immich_microservices       |     return new ReplyError(string)
immich_microservices       |            ^
immich_microservices       | 
immich_microservices       | ReplyError: MISCONF Redis is configured to save RDB snapshots, but it is currently not able to persist on disk. Commands that may modify the data set are disabled, because this instance is configured to report errors during writes if RDB snapshotting fails (stop-writes-on-bgsave-error option). Please check the Redis logs for details about the RDB error.
immich_microservices       |     at parseError (/usr/src/app/node_modules/redis-parser/lib/parser.js:179:12)
immich_microservices       |     at parseType (/usr/src/app/node_modules/redis-parser/lib/parser.js:302:14) {
immich_microservices       |   command: {
immich_microservices       |     name: 'hset',
immich_microservices       |     args: [
immich_microservices       |       'immich_bull:thumbnailGeneration:meta',
immich_microservices       |       'opts.maxLenEvents',
immich_microservices       |       '10000'
immich_microservices       |     ]
immich_microservices       |   }
immich_microservices       | }
immich_microservices       | 
immich_microservices       | Node.js v20.8.1
immich_microservices exited with code 1
immich_typesense           | E20231102 09:17:35.622918   358 raft_server.cpp:624] 1328 queued writes > healthy read lag of 1000
immich_typesense           | E20231102 09:17:35.622938   358 raft_server.cpp:636] 1328 queued writes > healthy write lag of 500
immich_redis               | 1:M 02 Nov 2023 09:17:36.029 * 100 changes in 300 seconds. Saving...
immich_redis               | 1:M 02 Nov 2023 09:17:36.030 * Background saving started by pid 19
immich_redis               | 19:C 02 Nov 2023 09:17:36.030 # Write error saving DB on disk: No space left on device
immich_redis               | 1:M 02 Nov 2023 09:17:36.130 # Background saving error
immich_typesense           | E20231102 09:17:36.212462   425 collection.cpp:364] Write to disk failed. Will restore old document
immich_typesense           | E20231102 09:17:36.212574   425 collection.cpp:364] Write to disk failed. Will restore old document
immich_typesense           | E20231102 09:17:36.212651   425 collection.cpp:364] Write to disk failed. Will restore old document
immich_typesense           | E20231102 09:17:36.212726   425 collection.cpp:364] Write to disk failed. Will restore old document
immich_typesense           | E20231102 09:17:36.212800   425 collection.cpp:364] Write to disk failed. Will restore old document
immich_typesense           | E20231102 09:17:36.212888   425 collection.cpp:364] Write to disk failed. Will restore old document
immich_typesense           | E20231102 09:17:36.212983   425 collection.cpp:364] Write to disk failed. Will restore old document
immich_typesense           | E20231102 09:17:36.213068   425 collection.cpp:364] Write to disk failed. Will restore old document
immich_typesense           | E20231102 09:17:36.213156   425 collection.cpp:364] Write to disk failed. Will restore old document
immich_typesense           | E20231102 09:17:36.213254   425 collection.cpp:364] Write to disk failed. Will restore old document
immich_typesense           | E20231102 09:17:36.213352   425 collection.cpp:364] Write to disk failed. Will restore old document
immich_typesense           | E20231102 09:17:36.213446   425 collection.cpp:364] Write to disk failed. Will restore old document
immich_typesense           | E20231102 09:17:36.213533   425 collection.cpp:364] Write to disk failed. Will restore old document
immich_typesense           | E20231102 09:17:36.213629   425 collection.cpp:364] Write to disk failed. Will restore old document
immich_typesense           | E20231102 09:17:36.213722   425 collection.cpp:364] Write to disk failed. Will restore old document
immich_typesense           | E20231102 09:17:36.213806   425 collection.cpp:364] Write to disk failed. Will restore old document
immich_typesense           | E20231102 09:17:36.213900   425 collection.cpp:364] Write to disk failed. Will restore old document
immich_typesense           | E20231102 09:17:36.213986   425 collection.cpp:364] Write to disk failed. Will restore old document
immich_typesense           | E20231102 09:17:36.214071   425 collection.cpp:364] Write to disk failed. Will restore old document
immich_typesense           | E20231102 09:17:36.214159   425 collection.cpp:364] Write to disk failed. Will restore old document
immich_typesense           | E20231102 09:17:36.214268   425 collection.cpp:364] Write to disk failed. Will restore old document
immich_typesense           | E20231102 09:17:36.214377   425 collection.cpp:364] Write to disk failed. Will restore old document
immich_typesense           | E20231102 09:17:36.214483   425 collection.cpp:364] Write to disk failed. Will restore old document
immich_typesense           | E20231102 09:17:36.214592   425 collection.cpp:364] Write to disk failed. Will restore old document
immich_typesense           | E20231102 09:17:36.214679   425 collection.cpp:364] Write to disk failed. Will restore old document
immich_typesense           | E20231102 09:17:36.214782   425 collection.cpp:364] Write to disk failed. Will restore old document
immich_typesense           | E20231102 09:17:36.214882   425 collection.cpp:364] Write to disk failed. Will restore old document
immich_typesense           | E20231102 09:17:36.215003   425 collection.cpp:364] Write to disk failed. Will restore old document
immich_typesense           | E20231102 09:17:36.215093   425 collection.cpp:364] Write to disk failed. Will restore old document
immich_typesense           | E20231102 09:17:36.215206   425 collection.cpp:364] Write to disk failed. Will restore old document
Originally created by @ghost on GitHub (Nov 2, 2023). ### The bug getting lots of errors after upgrading to 1.84 worked fine to start with, but updated the iOS app on the phone and now just getting server 500 errors in the web and mobile app.. most errors seem to come from typesets ``` Name Command State Ports ------------------------------------------------------------------------------------------------------- immich_machine_learning tini -- ./start.sh Up immich_microservices tini -- /bin/sh start.sh m ... Up 3001/tcp immich_postgres docker-entrypoint.sh postgres Up 5432/tcp immich_proxy /docker-entrypoint.sh ngin ... Up 0.0.0.0:80->8080/tcp,:::80->8080/tcp immich_redis docker-entrypoint.sh redis ... Up 6379/tcp immich_server tini -- /bin/sh start.sh i ... Up 3001/tcp immich_typesense /opt/typesense-server Up 8108/tcp immich_web tini -- /bin/sh entrypoint.sh Up 3000/tcp root@immich:~/immich-app# docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES f16fdfa39499 ghcr.io/immich-app/immich-proxy:release "/docker-entrypoint.…" 8 minutes ago Up 41 seconds 0.0.0.0:80->8080/tcp, :::80->8080/tcp immich_proxy 183368e2782e ghcr.io/immich-app/immich-server:release "tini -- /bin/sh sta…" 8 minutes ago Up 42 seconds 3001/tcp immich_server 2b9ad892b652 ghcr.io/immich-app/immich-server:release "tini -- /bin/sh sta…" 8 minutes ago Up 42 seconds 3001/tcp immich_microservices f291c0c12a3a redis:6.2-alpine "docker-entrypoint.s…" 8 minutes ago Up 42 seconds 6379/tcp immich_redis 090b09d021ff ghcr.io/immich-app/immich-machine-learning:release "tini -- ./start.sh" 8 minutes ago Up 42 seconds immich_machine_learning 0eae301a664c typesense/typesense:0.24.1 "/opt/typesense-serv…" 8 minutes ago Up 42 seconds 8108/tcp immich_typesense 50138c9bd5e4 ghcr.io/immich-app/immich-web:release "tini -- /bin/sh ent…" 8 minutes ago Up 42 seconds 3000/tcp immich_web 1076f28b948e postgres:14-alpine "docker-entrypoint.s…" 8 minutes ago Up 42 seconds 5432/tcp immich_postgres ``` after some time ``` Name Command State Ports ---------------------------------------------------------------------------------------------------------- immich_machine_learning tini -- ./start.sh Up immich_microservices tini -- /bin/sh start.sh m ... Exit 134 immich_postgres docker-entrypoint.sh postgres Exit 1 immich_proxy /docker-entrypoint.sh ngin ... Up 0.0.0.0:80->8080/tcp,:::80->8080/tcp immich_redis docker-entrypoint.sh redis ... Up 6379/tcp immich_server tini -- /bin/sh start.sh i ... Exit 1 immich_typesense /opt/typesense-server Up 8108/tcp immich_web tini -- /bin/sh entrypoint.sh Up 3000/tcp root@immich:~/immich-app# docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES f16fdfa39499 ghcr.io/immich-app/immich-proxy:release "/docker-entrypoint.…" 20 minutes ago Up 12 minutes 0.0.0.0:80->8080/tcp, :::80->8080/tcp immich_proxy 183368e2782e ghcr.io/immich-app/immich-server:release "tini -- /bin/sh sta…" 20 minutes ago Up 9 minutes 3001/tcp immich_server 2b9ad892b652 ghcr.io/immich-app/immich-server:release "tini -- /bin/sh sta…" 20 minutes ago Up 12 minutes 3001/tcp immich_microservices f291c0c12a3a redis:6.2-alpine "docker-entrypoint.s…" 20 minutes ago Up 12 minutes 6379/tcp immich_redis 090b09d021ff ghcr.io/immich-app/immich-machine-learning:release "tini -- ./start.sh" 20 minutes ago Up 12 minutes immich_machine_learning 0eae301a664c typesense/typesense:0.24.1 "/opt/typesense-serv…" 20 minutes ago Up 12 minutes 8108/tcp immich_typesense 50138c9bd5e4 ghcr.io/immich-app/immich-web:release "tini -- /bin/sh ent…" 20 minutes ago Up 12 minutes 3000/tcp immich_web 1076f28b948e postgres:14-alpine "docker-entrypoint.s…" 20 minutes ago Restarting (1) 7 minutes ago immich_postgres ``` ### The OS that Immich Server is running on ubuntu 22 ### Version of Immich Server 1.84 ### Version of Immich Mobile App latest today ### Platform with the issue - [X] Server - [X] Web - [X] Mobile ### Your docker-compose.yml content ```YAML default ``` ### Your .env content ```Shell default ``` ### Reproduction steps ```bash 1. 2. 3. ... ``` ### Additional information ``` immich_typesense | E20231102 09:13:12.495561 425 collection.cpp:364] Write to disk failed. Will restore old document immich_typesense | E20231102 09:13:12.495658 425 collection.cpp:364] Write to disk failed. Will restore old document immich_typesense | E20231102 09:13:12.495747 425 collection.cpp:364] Write to disk failed. Will restore old document immich_typesense | E20231102 09:13:12.495829 425 collection.cpp:364] Write to disk failed. Will restore old document immich_typesense | E20231102 09:13:12.495914 425 collection.cpp:364] Write to disk failed. Will restore old document immich_typesense | E20231102 09:13:12.495994 425 collection.cpp:364] Write to disk failed. Will restore old document immich_typesense | E20231102 09:13:12.496078 425 collection.cpp:364] Write to disk failed. Will restore old document immich_typesense | E20231102 09:13:12.496163 425 collection.cpp:364] Write to disk failed. Will restore old document immich_typesense | E20231102 09:13:12.496245 425 collection.cpp:364] Write to disk failed. Will restore old document immich_typesense | E20231102 09:13:12.496345 425 collection.cpp:364] Write to disk failed. Will restore old document immich_typesense | E20231102 09:13:12.496434 425 collection.cpp:364] Write to disk failed. Will restore old document immich_typesense | E20231102 09:13:12.496521 425 collection.cpp:364] Write to disk failed. Will restore old document immich_typesense | E20231102 09:13:12.496608 425 collection.cpp:364] Write to disk failed. Will restore old document immich_typesense | E20231102 09:13:12.496691 425 collection.cpp:364] Write to disk failed. Will restore old document immich_typesense | E20231102 09:13:12.496780 425 collection.cpp:364] Write to disk failed. Will restore old document immich_typesense | E20231102 09:13:12.496865 425 collection.cpp:364] Write to disk failed. Will restore old document immich_typesense | E20231102 09:13:12.496949 425 collection.cpp:364] Write to disk failed. Will restore old document immich_typesense | E20231102 09:13:12.497061 425 collection.cpp:364] Write to disk failed. Will restore old document immich_typesense | E20231102 09:13:12.497184 425 collection.cpp:364] Write to disk failed. Will restore old document immich_typesense | E20231102 09:13:12.497293 425 collection.cpp:364] Write to disk failed. Will restore old document immich_typesense | E20231102 09:13:12.497406 425 collection.cpp:364] Write to disk failed. Will restore old document immich_typesense | E20231102 09:13:12.497514 425 collection.cpp:364] Write to disk failed. Will restore old document immich_typesense | E20231102 09:13:12.497628 425 collection.cpp:364] Write to disk failed. Will restore old document immich_typesense | E20231102 09:13:12.497740 425 collection.cpp:364] Write to disk failed. Will restore old document immich_server | Request #1698916334394: Request to Node 0 failed due to "undefined Request failed with HTTP code 503 | Server said: Not Ready or Lagging" immich_server | Request #1698916334394: Sleeping for 4s and then retrying request... immich_typesense | E20231102 09:13:14.599668 358 raft_server.cpp:624] 1988 queued writes > healthy read lag of 1000 immich_typesense | E20231102 09:13:14.599679 358 raft_server.cpp:636] 1988 queued writes > healthy write lag of 500 immich_microservices | Request #1698916347097: Request to Node 0 failed due to "undefined Request failed with HTTP code 503 | Server said: Not Ready or Lagging" immich_microservices | Request #1698916347097: Sleeping for 4s and then retrying request... immich_typesense | W20231102 09:13:15.233565 425 index.cpp:5406] Error while removing field `embedding` from document, message: Label not found immich_typesense | W20231102 09:13:16.940913 425 index.cpp:5406] Error while removing field `embedding` from document, message: Label not found immich_server | /usr/src/app/node_modules/typesense/lib/Typesense/Errors/TypesenseError.js:23 immich_server | var _this = _super.call(this, message) || this; immich_server | ^ immich_server | immich_server | ServerError: Request failed with HTTP code 503 | Server said: Not Ready or Lagging immich_server | at ServerError.TypesenseError [as constructor] (/usr/src/app/node_modules/typesense/lib/Typesense/Errors/TypesenseError.js:23:28) immich_server | at new ServerError (/usr/src/app/node_modules/typesense/lib/Typesense/Errors/ServerError.js:25:42) immich_server | at ApiCall.customErrorForResponse (/usr/src/app/node_modules/typesense/lib/Typesense/ApiCall.js:376:21) immich_server | at /usr/src/app/node_modules/typesense/lib/Typesense/ApiCall.js:225:58 immich_server | at step (/usr/src/app/node_modules/typesense/lib/Typesense/ApiCall.js:33:23) immich_server | at Object.next (/usr/src/app/node_modules/typesense/lib/Typesense/ApiCall.js:14:53) immich_server | at step (/usr/src/app/node_modules/typesense/lib/Typesense/ApiCall.js:18:139) immich_server | at Object.next (/usr/src/app/node_modules/typesense/lib/Typesense/ApiCall.js:14:53) immich_server | at fulfilled (/usr/src/app/node_modules/typesense/lib/Typesense/ApiCall.js:5:58) immich_server | at process.processTicksAndRejections (node:internal/process/task_queues:95:5) { immich_server | httpStatus: 503 immich_server | } immich_server | immich_server | Node.js v20.8.1 immich_typesense | W20231102 09:13:18.530943 425 index.cpp:5406] Error while removing field `embedding` from document, message: Label not found immich_typesense | W20231102 09:13:18.838958 425 index.cpp:5406] Error while removing field `embedding` from document, message: Label not found immich_typesense | W20231102 09:13:19.053009 425 index.cpp:5406] Error while removing field `embedding` from document, message: Label not found immich_microservices | Request #1698916347097: Request to Node 0 failed due to "undefined Request failed with HTTP code 503 | Server said: Not Ready or Lagging" immich_microservices | Request #1698916347097: Sleeping for 4s and then retrying request... immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [NestFactory] Starting Nest application... immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [InstanceLoader] TypeOrmModule dependencies initialized +32ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [InstanceLoader] BullModule dependencies initialized +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [InstanceLoader] ConfigHostModule dependencies initialized +1ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [InstanceLoader] DiscoveryModule dependencies initialized +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [InstanceLoader] ScheduleModule dependencies initialized +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [InstanceLoader] ConfigModule dependencies initialized +4ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [InstanceLoader] BullModule dependencies initialized +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [InstanceLoader] ScheduleModule dependencies initialized +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [InstanceLoader] BullModule dependencies initialized +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [InstanceLoader] TypeOrmCoreModule dependencies initialized +157ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [InstanceLoader] TypeOrmModule dependencies initialized +1ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [InstanceLoader] TypeOrmModule dependencies initialized +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [InstanceLoader] InfraModule dependencies initialized +2ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [InstanceLoader] AppModule dependencies initialized +5ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [InstanceLoader] DomainModule dependencies initialized +13ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RoutesResolver] ActivityController {/activity}: +106ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/activity, GET} route +3ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/activity/statistics, GET} route +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/activity, POST} route +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/activity/:id, DELETE} route +1ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RoutesResolver] AssetController {/asset}: +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/asset/map-marker, GET} route +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/asset/memory-lane, GET} route +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/asset/random, GET} route +1ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/asset/download/info, POST} route +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/asset/download/archive, POST} route +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/asset/download/:id, POST} route +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/asset/statistics, GET} route +1ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/asset/time-buckets, GET} route +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/asset/time-bucket, GET} route +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/asset/jobs, POST} route +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/asset, PUT} route +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/asset, DELETE} route +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/asset/restore, POST} route +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/asset/trash/empty, POST} route +1ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/asset/trash/restore, POST} route +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/asset/stack/parent, PUT} route +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/asset/:id, PUT} route +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RoutesResolver] AssetController {/asset}: +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/asset/upload, POST} route +1ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/asset/import, POST} route +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/asset/file/:id, GET} route +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/asset/thumbnail/:id, GET} route +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/asset/curated-objects, GET} route +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/asset/curated-locations, GET} route +1ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/asset/search-terms, GET} route +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/asset/search, POST} route +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/asset, GET} route +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/asset/:deviceId, GET} route +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/asset/assetById/:id, GET} route +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/asset/exist, POST} route +2ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/asset/bulk-upload-check, POST} route +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RoutesResolver] AppController {/}: +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/refresh-config, POST} route +1ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RoutesResolver] AlbumController {/album}: +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/album/count, GET} route +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/album, GET} route +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/album, POST} route +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/album/:id, GET} route +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/album/:id, PATCH} route +1ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/album/:id, DELETE} route +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/album/:id/assets, PUT} route +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/album/:id/assets, DELETE} route +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/album/:id/users, PUT} route +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/album/:id/user/:userId, DELETE} route +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RoutesResolver] APIKeyController {/api-key}: +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/api-key, POST} route +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/api-key, GET} route +1ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/api-key/:id, GET} route +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/api-key/:id, PUT} route +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/api-key/:id, DELETE} route +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RoutesResolver] AuditController {/audit}: +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/audit/deletes, GET} route +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/audit/file-report, GET} route +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/audit/file-report/checksum, POST} route +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/audit/file-report/fix, POST} route +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RoutesResolver] AuthController {/auth}: +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/auth/login, POST} route +1ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/auth/admin-sign-up, POST} route +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/auth/devices, GET} route +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/auth/devices, DELETE} route +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/auth/devices/:id, DELETE} route +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/auth/validateToken, POST} route +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/auth/change-password, POST} route +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/auth/logout, POST} route +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RoutesResolver] JobController {/jobs}: +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/jobs, GET} route +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/jobs/:id, PUT} route +1ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RoutesResolver] LibraryController {/library}: +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/library, GET} route +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/library, POST} route +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/library/:id, PUT} route +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/library/:id, GET} route +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/library/:id, DELETE} route +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/library/:id/statistics, GET} route +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/library/:id/scan, POST} route +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/library/:id/removeOffline, POST} route +1ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RoutesResolver] OAuthController {/oauth}: +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/oauth/mobile-redirect, GET} route +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/oauth/config, POST} route +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/oauth/authorize, POST} route +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/oauth/callback, POST} route +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/oauth/link, POST} route +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/oauth/unlink, POST} route +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RoutesResolver] PartnerController {/partner}: +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/partner, GET} route +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/partner/:id, POST} route +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/partner/:id, DELETE} route +1ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RoutesResolver] SearchController {/search}: +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/search/person, GET} route +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/search, GET} route +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/search/explore, GET} route +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RoutesResolver] ServerInfoController {/server-info}: +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/server-info, GET} route +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/server-info/ping, GET} route +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/server-info/version, GET} route +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/server-info/features, GET} route +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/server-info/theme, GET} route +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/server-info/config, GET} route +1ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/server-info/stats, GET} route +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/server-info/media-types, GET} route +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RoutesResolver] SharedLinkController {/shared-link}: +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/shared-link, GET} route +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/shared-link/me, GET} route +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/shared-link/:id, GET} route +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/shared-link, POST} route +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/shared-link/:id, PATCH} route +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/shared-link/:id, DELETE} route +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/shared-link/:id/assets, PUT} route +1ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/shared-link/:id/assets, DELETE} route +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RoutesResolver] SystemConfigController {/system-config}: +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/system-config, GET} route +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/system-config/defaults, GET} route +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/system-config, PUT} route +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/system-config/storage-template-options, GET} route +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RoutesResolver] TagController {/tag}: +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/tag, POST} route +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/tag, GET} route +1ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/tag/:id, GET} route +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/tag/:id, PATCH} route +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/tag/:id, DELETE} route +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/tag/:id/assets, GET} route +0ms immich_server | [Nest] 7 - 11/02/2023, 7:13:20 PM LOG [RouterExplorer] Mapped {/tag/:id/assets, PUT} route +0ms immich_typesense | E20231102 09:17:27.830929 425 collection.cpp:340] Update to disk failed. Will restore old document immich_typesense | E20231102 09:17:27.833989 425 collection.cpp:340] Update to disk failed. Will restore old document immich_typesense | E20231102 09:17:27.845978 425 collection.cpp:340] Update to disk failed. Will restore old document immich_typesense | E20231102 09:17:27.852667 425 collection.cpp:340] Update to disk failed. Will restore old document immich_typesense | E20231102 09:17:27.856185 425 collection.cpp:340] Update to disk failed. Will restore old document immich_typesense | E20231102 09:17:27.864369 425 collection.cpp:340] Update to disk failed. Will restore old document immich_microservices | [Nest] 7 - 11/02/2023, 7:17:29 PM LOG [NestFactory] Starting Nest application... immich_microservices | [Nest] 7 - 11/02/2023, 7:17:29 PM LOG [InstanceLoader] TypeOrmModule dependencies initialized +34ms immich_microservices | [Nest] 7 - 11/02/2023, 7:17:29 PM LOG [InstanceLoader] BullModule dependencies initialized +0ms immich_microservices | [Nest] 7 - 11/02/2023, 7:17:29 PM LOG [InstanceLoader] ConfigHostModule dependencies initialized +0ms immich_microservices | [Nest] 7 - 11/02/2023, 7:17:29 PM LOG [InstanceLoader] DiscoveryModule dependencies initialized +0ms immich_microservices | [Nest] 7 - 11/02/2023, 7:17:29 PM LOG [InstanceLoader] ScheduleModule dependencies initialized +0ms immich_microservices | [Nest] 7 - 11/02/2023, 7:17:29 PM LOG [InstanceLoader] ConfigModule dependencies initialized +5ms immich_microservices | [Nest] 7 - 11/02/2023, 7:17:29 PM LOG [InstanceLoader] BullModule dependencies initialized +0ms immich_microservices | [Nest] 7 - 11/02/2023, 7:17:29 PM LOG [InstanceLoader] BullModule dependencies initialized +0ms immich_microservices | /usr/src/app/node_modules/redis-parser/lib/parser.js:179 immich_microservices | return new ReplyError(string) immich_microservices | ^ immich_microservices | immich_microservices | ReplyError: MISCONF Redis is configured to save RDB snapshots, but it is currently not able to persist on disk. Commands that may modify the data set are disabled, because this instance is configured to report errors during writes if RDB snapshotting fails (stop-writes-on-bgsave-error option). Please check the Redis logs for details about the RDB error. immich_microservices | at parseError (/usr/src/app/node_modules/redis-parser/lib/parser.js:179:12) immich_microservices | at parseType (/usr/src/app/node_modules/redis-parser/lib/parser.js:302:14) { immich_microservices | command: { immich_microservices | name: 'hset', immich_microservices | args: [ immich_microservices | 'immich_bull:thumbnailGeneration:meta', immich_microservices | 'opts.maxLenEvents', immich_microservices | '10000' immich_microservices | ] immich_microservices | } immich_microservices | } immich_microservices | immich_microservices | Node.js v20.8.1 immich_microservices exited with code 1 immich_redis | 1:M 02 Nov 2023 09:17:30.015 * 100 changes in 300 seconds. Saving... immich_redis | 1:M 02 Nov 2023 09:17:30.015 * Background saving started by pid 18 immich_redis | 18:C 02 Nov 2023 09:17:30.019 # Write error saving DB on disk: No space left on device immich_redis | 1:M 02 Nov 2023 09:17:30.116 # Background saving error immich_server | Request #1698916602115: Request to Node 0 failed due to "undefined Request failed with HTTP code 503 | Server said: Not Ready or Lagging" immich_server | Request #1698916602115: Sleeping for 4s and then retrying request... immich_typesense | W20231102 09:17:31.800505 425 index.cpp:5406] Error while removing field `embedding` from document, message: Label not found immich_microservices | [Nest] 7 - 11/02/2023, 7:17:31 PM LOG [NestFactory] Starting Nest application... immich_microservices | [Nest] 7 - 11/02/2023, 7:17:32 PM LOG [InstanceLoader] TypeOrmModule dependencies initialized +35ms immich_microservices | [Nest] 7 - 11/02/2023, 7:17:32 PM LOG [InstanceLoader] BullModule dependencies initialized +0ms immich_microservices | [Nest] 7 - 11/02/2023, 7:17:32 PM LOG [InstanceLoader] ConfigHostModule dependencies initialized +1ms immich_microservices | [Nest] 7 - 11/02/2023, 7:17:32 PM LOG [InstanceLoader] DiscoveryModule dependencies initialized +0ms immich_microservices | [Nest] 7 - 11/02/2023, 7:17:32 PM LOG [InstanceLoader] ScheduleModule dependencies initialized +0ms immich_microservices | [Nest] 7 - 11/02/2023, 7:17:32 PM LOG [InstanceLoader] ConfigModule dependencies initialized +5ms immich_microservices | [Nest] 7 - 11/02/2023, 7:17:32 PM LOG [InstanceLoader] BullModule dependencies initialized +0ms immich_microservices | [Nest] 7 - 11/02/2023, 7:17:32 PM LOG [InstanceLoader] BullModule dependencies initialized +0ms immich_microservices | /usr/src/app/node_modules/redis-parser/lib/parser.js:179 immich_microservices | return new ReplyError(string) immich_microservices | ^ immich_microservices | immich_microservices | ReplyError: MISCONF Redis is configured to save RDB snapshots, but it is currently not able to persist on disk. Commands that may modify the data set are disabled, because this instance is configured to report errors during writes if RDB snapshotting fails (stop-writes-on-bgsave-error option). Please check the Redis logs for details about the RDB error. immich_microservices | at parseError (/usr/src/app/node_modules/redis-parser/lib/parser.js:179:12) immich_microservices | at parseType (/usr/src/app/node_modules/redis-parser/lib/parser.js:302:14) { immich_microservices | command: { immich_microservices | name: 'hset', immich_microservices | args: [ immich_microservices | 'immich_bull:thumbnailGeneration:meta', immich_microservices | 'opts.maxLenEvents', immich_microservices | '10000' immich_microservices | ] immich_microservices | } immich_microservices | } immich_microservices | immich_microservices | Node.js v20.8.1 immich_microservices exited with code 1 immich_typesense | W20231102 09:17:32.621006 425 index.cpp:5406] Error while removing field `embedding` from document, message: Label not found immich_server | Request #1698916602115: Request to Node 0 failed due to "undefined Request failed with HTTP code 503 | Server said: Not Ready or Lagging" immich_server | Request #1698916602115: Sleeping for 4s and then retrying request... immich_microservices | [Nest] 7 - 11/02/2023, 7:17:34 PM LOG [NestFactory] Starting Nest application... immich_microservices | [Nest] 7 - 11/02/2023, 7:17:34 PM LOG [InstanceLoader] TypeOrmModule dependencies initialized +34ms immich_microservices | [Nest] 7 - 11/02/2023, 7:17:34 PM LOG [InstanceLoader] BullModule dependencies initialized +0ms immich_microservices | [Nest] 7 - 11/02/2023, 7:17:34 PM LOG [InstanceLoader] ConfigHostModule dependencies initialized +1ms immich_microservices | [Nest] 7 - 11/02/2023, 7:17:34 PM LOG [InstanceLoader] DiscoveryModule dependencies initialized +0ms immich_microservices | [Nest] 7 - 11/02/2023, 7:17:34 PM LOG [InstanceLoader] ScheduleModule dependencies initialized +0ms immich_microservices | [Nest] 7 - 11/02/2023, 7:17:34 PM LOG [InstanceLoader] ConfigModule dependencies initialized +5ms immich_microservices | [Nest] 7 - 11/02/2023, 7:17:34 PM LOG [InstanceLoader] BullModule dependencies initialized +0ms immich_microservices | [Nest] 7 - 11/02/2023, 7:17:34 PM LOG [InstanceLoader] BullModule dependencies initialized +0ms immich_microservices | /usr/src/app/node_modules/redis-parser/lib/parser.js:179 immich_microservices | return new ReplyError(string) immich_microservices | ^ immich_microservices | immich_microservices | ReplyError: MISCONF Redis is configured to save RDB snapshots, but it is currently not able to persist on disk. Commands that may modify the data set are disabled, because this instance is configured to report errors during writes if RDB snapshotting fails (stop-writes-on-bgsave-error option). Please check the Redis logs for details about the RDB error. immich_microservices | at parseError (/usr/src/app/node_modules/redis-parser/lib/parser.js:179:12) immich_microservices | at parseType (/usr/src/app/node_modules/redis-parser/lib/parser.js:302:14) { immich_microservices | command: { immich_microservices | name: 'hset', immich_microservices | args: [ immich_microservices | 'immich_bull:thumbnailGeneration:meta', immich_microservices | 'opts.maxLenEvents', immich_microservices | '10000' immich_microservices | ] immich_microservices | } immich_microservices | } immich_microservices | immich_microservices | Node.js v20.8.1 immich_microservices exited with code 1 immich_typesense | E20231102 09:17:35.622918 358 raft_server.cpp:624] 1328 queued writes > healthy read lag of 1000 immich_typesense | E20231102 09:17:35.622938 358 raft_server.cpp:636] 1328 queued writes > healthy write lag of 500 immich_redis | 1:M 02 Nov 2023 09:17:36.029 * 100 changes in 300 seconds. Saving... immich_redis | 1:M 02 Nov 2023 09:17:36.030 * Background saving started by pid 19 immich_redis | 19:C 02 Nov 2023 09:17:36.030 # Write error saving DB on disk: No space left on device immich_redis | 1:M 02 Nov 2023 09:17:36.130 # Background saving error immich_typesense | E20231102 09:17:36.212462 425 collection.cpp:364] Write to disk failed. Will restore old document immich_typesense | E20231102 09:17:36.212574 425 collection.cpp:364] Write to disk failed. Will restore old document immich_typesense | E20231102 09:17:36.212651 425 collection.cpp:364] Write to disk failed. Will restore old document immich_typesense | E20231102 09:17:36.212726 425 collection.cpp:364] Write to disk failed. Will restore old document immich_typesense | E20231102 09:17:36.212800 425 collection.cpp:364] Write to disk failed. Will restore old document immich_typesense | E20231102 09:17:36.212888 425 collection.cpp:364] Write to disk failed. Will restore old document immich_typesense | E20231102 09:17:36.212983 425 collection.cpp:364] Write to disk failed. Will restore old document immich_typesense | E20231102 09:17:36.213068 425 collection.cpp:364] Write to disk failed. Will restore old document immich_typesense | E20231102 09:17:36.213156 425 collection.cpp:364] Write to disk failed. Will restore old document immich_typesense | E20231102 09:17:36.213254 425 collection.cpp:364] Write to disk failed. Will restore old document immich_typesense | E20231102 09:17:36.213352 425 collection.cpp:364] Write to disk failed. Will restore old document immich_typesense | E20231102 09:17:36.213446 425 collection.cpp:364] Write to disk failed. Will restore old document immich_typesense | E20231102 09:17:36.213533 425 collection.cpp:364] Write to disk failed. Will restore old document immich_typesense | E20231102 09:17:36.213629 425 collection.cpp:364] Write to disk failed. Will restore old document immich_typesense | E20231102 09:17:36.213722 425 collection.cpp:364] Write to disk failed. Will restore old document immich_typesense | E20231102 09:17:36.213806 425 collection.cpp:364] Write to disk failed. Will restore old document immich_typesense | E20231102 09:17:36.213900 425 collection.cpp:364] Write to disk failed. Will restore old document immich_typesense | E20231102 09:17:36.213986 425 collection.cpp:364] Write to disk failed. Will restore old document immich_typesense | E20231102 09:17:36.214071 425 collection.cpp:364] Write to disk failed. Will restore old document immich_typesense | E20231102 09:17:36.214159 425 collection.cpp:364] Write to disk failed. Will restore old document immich_typesense | E20231102 09:17:36.214268 425 collection.cpp:364] Write to disk failed. Will restore old document immich_typesense | E20231102 09:17:36.214377 425 collection.cpp:364] Write to disk failed. Will restore old document immich_typesense | E20231102 09:17:36.214483 425 collection.cpp:364] Write to disk failed. Will restore old document immich_typesense | E20231102 09:17:36.214592 425 collection.cpp:364] Write to disk failed. Will restore old document immich_typesense | E20231102 09:17:36.214679 425 collection.cpp:364] Write to disk failed. Will restore old document immich_typesense | E20231102 09:17:36.214782 425 collection.cpp:364] Write to disk failed. Will restore old document immich_typesense | E20231102 09:17:36.214882 425 collection.cpp:364] Write to disk failed. Will restore old document immich_typesense | E20231102 09:17:36.215003 425 collection.cpp:364] Write to disk failed. Will restore old document immich_typesense | E20231102 09:17:36.215093 425 collection.cpp:364] Write to disk failed. Will restore old document immich_typesense | E20231102 09:17:36.215206 425 collection.cpp:364] Write to disk failed. Will restore old document ```
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
accessibility
changelog:enhancement
changelog:security
changelog:translation
cli
date-time
documentation
external-library
format
good first issue
needs-answer
nice to have
sharing
tech-debt
📱mobile
🖥️web
🗄️server
🧠machine-learning
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
OVERLORD
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: immich-app/immich#1549
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?