[BUG] 500 error when open "explore" on web and mobile things search. #831

Closed
opened 2026-02-04 22:52:35 +03:00 by OVERLORD · 9 comments
Owner

Originally created by @zlo2k on GitHub (Apr 25, 2023).

The bug

Immich docker server log: ``` [Nest] 1 - 04/25/2023, 9:21:52 AM  ERROR [ExceptionsHandler] Request failed with HTTP code 404 | Server said: Could not find a field named `originalFileName` in the schema. ObjectNotFound: Request failed with HTTP code 404 | Server said: Could not find a field named `originalFileName` in the schema. at ObjectNotFound.TypesenseError [as constructor] (/usr/src/app/node_modules/typesense/lib/Typesense/Errors/TypesenseError.js:23:28) at new ObjectNotFound (/usr/src/app/node_modules/typesense/lib/Typesense/Errors/ObjectNotFound.js:25:42) at ApiCall.customErrorForResponse (/usr/src/app/node_modules/typesense/lib/Typesense/ApiCall.js:338:21) at /usr/src/app/node_modules/typesense/lib/Typesense/ApiCall.js:199:98 at step (/usr/src/app/node_modules/typesense/lib/Typesense/ApiCall.js:33:23) at Object.next (/usr/src/app/node_modules/typesense/lib/Typesense/ApiCall.js:14:53) at step (/usr/src/app/node_modules/typesense/lib/Typesense/ApiCall.js:18:139) at Object.next (/usr/src/app/node_modules/typesense/lib/Typesense/ApiCall.js:14:53) at fulfilled (/usr/src/app/node_modules/typesense/lib/Typesense/ApiCall.js:5:58) at processTicksAndRejections (node:internal/process/task_queues:96:5) [Nest] 1 - 04/25/2023, 9:22:00 AM  ERROR [ExceptionsHandler] Request failed with HTTP code 404 | Server said: Could not find a field named `originalFileName` in the schema. ObjectNotFound: Request failed with HTTP code 404 | Server said: Could not find a field named `originalFileName` in the schema. at ObjectNotFound.TypesenseError [as constructor] (/usr/src/app/node_modules/typesense/lib/Typesense/Errors/TypesenseError.js:23:28) at new ObjectNotFound (/usr/src/app/node_modules/typesense/lib/Typesense/Errors/ObjectNotFound.js:25:42) at ApiCall.customErrorForResponse (/usr/src/app/node_modules/typesense/lib/Typesense/ApiCall.js:338:21) at /usr/src/app/node_modules/typesense/lib/Typesense/ApiCall.js:199:98 at step (/usr/src/app/node_modules/typesense/lib/Typesense/ApiCall.js:33:23) at Object.next (/usr/src/app/node_modules/typesense/lib/Typesense/ApiCall.js:14:53) at step (/usr/src/app/node_modules/typesense/lib/Typesense/ApiCall.js:18:139) at Object.next (/usr/src/app/node_modules/typesense/lib/Typesense/ApiCall.js:14:53) at fulfilled (/usr/src/app/node_modules/typesense/lib/Typesense/ApiCall.js:5:58) at processTicksAndRejections (node:internal/process/task_queues:96:5) ```

The OS that Immich Server is running on

Ubuntu 20.04.5 LTS x86_64(Py3.7.9)

Version of Immich Server

v1.54.1

Version of Immich Mobile App

v1.54.0 build.77

Platform with the issue

  • Server
  • Web
  • Mobile

Your docker-compose.yml content

docker-compose.yml content
version: "3.8"

services:
  immich-server:
    container_name: immich_server
    image: ghcr.io/immich-app/immich-server:release
    entrypoint: [ "/bin/sh", "./start-server.sh" ]
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
    env_file:
      - .env
    environment:
      - NODE_ENV=production
    depends_on:
      - redis
      - database
      - typesense
    restart: always

  immich-microservices:
    container_name: immich_microservices
    image: ghcr.io/immich-app/immich-server:release
    entrypoint: [ "/bin/sh", "./start-microservices.sh" ]
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
    env_file:
      - .env
    environment:
      - NODE_ENV=production
    depends_on:
      - redis
      - database
      - typesense
    restart: always

  immich-machine-learning:
    container_name: immich_machine_learning
    image: ghcr.io/immich-app/immich-machine-learning:release
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - model-cache:/cache
    env_file:
      - .env
    environment:
      - NODE_ENV=production
    restart: always

  immich-web:
    container_name: immich_web
    image: ghcr.io/immich-app/immich-web:release
    entrypoint: [ "/bin/sh", "./entrypoint.sh" ]
    env_file:
      - .env
    restart: always

  typesense:
    container_name: immich_typesense
    image: typesense/typesense:0.24.0
    environment:
      - TYPESENSE_API_KEY=${TYPESENSE_API_KEY}
      - TYPESENSE_DATA_DIR=/data
    logging:
      driver: none
    volumes:
      - tsdata:/data
    restart: always  

  redis:
    container_name: immich_redis
    image: redis:6.2
    restart: always

  database:
    container_name: immich_postgres
    image: postgres:14
    env_file:
      - .env
    environment:
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      POSTGRES_USER: ${DB_USERNAME}
      POSTGRES_DB: ${DB_DATABASE_NAME}
      PG_DATA: /var/lib/postgresql/data
    volumes:
      - pgdata:/var/lib/postgresql/data
    restart: always

  immich-proxy:
    container_name: immich_proxy
    image: ghcr.io/immich-app/immich-proxy:release
    environment:
      # Make sure these values get passed through from the env file
      - IMMICH_SERVER_URL
      - IMMICH_WEB_URL
    ports:
      - 2283:8080
    logging:
      driver: none
    depends_on:
      - immich-server
    restart: always

volumes:
  pgdata:
  model-cache:
  tsdata:

Your .env content

.env content ```Shell ################################################################################### # Database ###################################################################################

DB_HOSTNAME=immich_postgres
DB_USERNAME=postgres
DB_PASSWORD=postgres
DB_DATABASE_NAME=immich

Optional Database settings:

DB_PORT=5432

###################################################################################

Redis

###################################################################################

REDIS_HOSTNAME=immich_redis

Optional Redis settings:

Note: these parameters are not automatically passed to the Redis Container

to do so, please edit the docker-compose.yml file as well. Redis is not configured

via environment variables, only redis.conf or the command line

REDIS_PORT=6379

REDIS_DBINDEX=0

REDIS_PASSWORD=

REDIS_SOCKET=

###################################################################################

Upload File Location

This is the location where uploaded files are stored.

###################################################################################

UPLOAD_LOCATION=/mnt/disk1/im_photos/

###################################################################################

Typesense

###################################################################################
TYPESENSE_API_KEY=sdfuysdfuywnrkvhvslkddgsb
#TYPESENSE_ENABLED=false

###################################################################################

Reverse Geocoding

Reverse geocoding is done locally which has a small impact on memory usage

This memory usage can be altered by changing the REVERSE_GEOCODING_PRECISION variable

This ranges from 0-3 with 3 being the most precise

3 - Cities > 500 population: ~200MB RAM

2 - Cities > 1000 population: ~150MB RAM

1 - Cities > 5000 population: ~80MB RAM

0 - Cities > 15000 population: ~40MB RAM

####################################################################################

DISABLE_REVERSE_GEOCODING=false

REVERSE_GEOCODING_PRECISION=3

####################################################################################

WEB - Optional

Custom message on the login page, should be written in HTML form.

For example:

PUBLIC_LOGIN_PAGE_MESSAGE="This is a demo instance of Immich.

Email: demo@demo.de
Password: demo"

####################################################################################

PUBLIC_LOGIN_PAGE_MESSAGE=

####################################################################################

Alternative Service Addresses - Optional

This is an advanced feature for users who may be running their immich services on different hosts.

It will not change which address or port that services bind to within their containers, but it will change where other services look for their peers.

Note: immich-microservices is bound to 3002, but no references are made

####################################################################################

IMMICH_WEB_URL=http://immich-web:3000
IMMICH_SERVER_URL=http://immich-server:3001
IMMICH_MACHINE_LEARNING_URL=http://immich-machine-learning:3003

####################################################################################

Alternative API's External Address - Optional

This is an advanced feature used to control the public server endpoint returned to clients during Well-known discovery.

You should only use this if you want mobile apps to access the immich API over a custom URL. Do not include trailing slash.

NOTE: At this time, the web app will not be affected by this setting and will continue to use the relative path: /api

Examples: http://localhost:3001, http://immich-api.example.com, etc

####################################################################################

#IMMICH_API_URL_EXTERNAL=http://localhost:3001

</details>

### Reproduction steps

```bash
Install the latest version, check "Explore" in the web version and things search in the mobile version.

Additional information

The database has over 169,000 photos and 2,000 videos.

Originally created by @zlo2k on GitHub (Apr 25, 2023). ### The bug <details><summary>Immich docker server log:</summary> ``` [Nest] 1 - 04/25/2023, 9:21:52 AM  ERROR [ExceptionsHandler] Request failed with HTTP code 404 | Server said: Could not find a field named `originalFileName` in the schema. ObjectNotFound: Request failed with HTTP code 404 | Server said: Could not find a field named `originalFileName` in the schema. at ObjectNotFound.TypesenseError [as constructor] (/usr/src/app/node_modules/typesense/lib/Typesense/Errors/TypesenseError.js:23:28) at new ObjectNotFound (/usr/src/app/node_modules/typesense/lib/Typesense/Errors/ObjectNotFound.js:25:42) at ApiCall.customErrorForResponse (/usr/src/app/node_modules/typesense/lib/Typesense/ApiCall.js:338:21) at /usr/src/app/node_modules/typesense/lib/Typesense/ApiCall.js:199:98 at step (/usr/src/app/node_modules/typesense/lib/Typesense/ApiCall.js:33:23) at Object.next (/usr/src/app/node_modules/typesense/lib/Typesense/ApiCall.js:14:53) at step (/usr/src/app/node_modules/typesense/lib/Typesense/ApiCall.js:18:139) at Object.next (/usr/src/app/node_modules/typesense/lib/Typesense/ApiCall.js:14:53) at fulfilled (/usr/src/app/node_modules/typesense/lib/Typesense/ApiCall.js:5:58) at processTicksAndRejections (node:internal/process/task_queues:96:5) [Nest] 1 - 04/25/2023, 9:22:00 AM  ERROR [ExceptionsHandler] Request failed with HTTP code 404 | Server said: Could not find a field named `originalFileName` in the schema. ObjectNotFound: Request failed with HTTP code 404 | Server said: Could not find a field named `originalFileName` in the schema. at ObjectNotFound.TypesenseError [as constructor] (/usr/src/app/node_modules/typesense/lib/Typesense/Errors/TypesenseError.js:23:28) at new ObjectNotFound (/usr/src/app/node_modules/typesense/lib/Typesense/Errors/ObjectNotFound.js:25:42) at ApiCall.customErrorForResponse (/usr/src/app/node_modules/typesense/lib/Typesense/ApiCall.js:338:21) at /usr/src/app/node_modules/typesense/lib/Typesense/ApiCall.js:199:98 at step (/usr/src/app/node_modules/typesense/lib/Typesense/ApiCall.js:33:23) at Object.next (/usr/src/app/node_modules/typesense/lib/Typesense/ApiCall.js:14:53) at step (/usr/src/app/node_modules/typesense/lib/Typesense/ApiCall.js:18:139) at Object.next (/usr/src/app/node_modules/typesense/lib/Typesense/ApiCall.js:14:53) at fulfilled (/usr/src/app/node_modules/typesense/lib/Typesense/ApiCall.js:5:58) at processTicksAndRejections (node:internal/process/task_queues:96:5) ``` </details> ### The OS that Immich Server is running on Ubuntu 20.04.5 LTS x86_64(Py3.7.9) ### Version of Immich Server v1.54.1 ### Version of Immich Mobile App v1.54.0 build.77 ### Platform with the issue - [X] Server - [X] Web - [X] Mobile ### Your docker-compose.yml content <details><summary>docker-compose.yml content</summary> ```YAML version: "3.8" services: immich-server: container_name: immich_server image: ghcr.io/immich-app/immich-server:release entrypoint: [ "/bin/sh", "./start-server.sh" ] volumes: - ${UPLOAD_LOCATION}:/usr/src/app/upload env_file: - .env environment: - NODE_ENV=production depends_on: - redis - database - typesense restart: always immich-microservices: container_name: immich_microservices image: ghcr.io/immich-app/immich-server:release entrypoint: [ "/bin/sh", "./start-microservices.sh" ] volumes: - ${UPLOAD_LOCATION}:/usr/src/app/upload env_file: - .env environment: - NODE_ENV=production depends_on: - redis - database - typesense restart: always immich-machine-learning: container_name: immich_machine_learning image: ghcr.io/immich-app/immich-machine-learning:release volumes: - ${UPLOAD_LOCATION}:/usr/src/app/upload - model-cache:/cache env_file: - .env environment: - NODE_ENV=production restart: always immich-web: container_name: immich_web image: ghcr.io/immich-app/immich-web:release entrypoint: [ "/bin/sh", "./entrypoint.sh" ] env_file: - .env restart: always typesense: container_name: immich_typesense image: typesense/typesense:0.24.0 environment: - TYPESENSE_API_KEY=${TYPESENSE_API_KEY} - TYPESENSE_DATA_DIR=/data logging: driver: none volumes: - tsdata:/data restart: always redis: container_name: immich_redis image: redis:6.2 restart: always database: container_name: immich_postgres image: postgres:14 env_file: - .env environment: POSTGRES_PASSWORD: ${DB_PASSWORD} POSTGRES_USER: ${DB_USERNAME} POSTGRES_DB: ${DB_DATABASE_NAME} PG_DATA: /var/lib/postgresql/data volumes: - pgdata:/var/lib/postgresql/data restart: always immich-proxy: container_name: immich_proxy image: ghcr.io/immich-app/immich-proxy:release environment: # Make sure these values get passed through from the env file - IMMICH_SERVER_URL - IMMICH_WEB_URL ports: - 2283:8080 logging: driver: none depends_on: - immich-server restart: always volumes: pgdata: model-cache: tsdata: ``` </details> ### Your .env content <details><summary> .env content</summary> ```Shell ################################################################################### # Database ################################################################################### DB_HOSTNAME=immich_postgres DB_USERNAME=postgres DB_PASSWORD=postgres DB_DATABASE_NAME=immich # Optional Database settings: # DB_PORT=5432 ################################################################################### # Redis ################################################################################### REDIS_HOSTNAME=immich_redis # Optional Redis settings: # Note: these parameters are not automatically passed to the Redis Container # to do so, please edit the docker-compose.yml file as well. Redis is not configured # via environment variables, only redis.conf or the command line # REDIS_PORT=6379 # REDIS_DBINDEX=0 # REDIS_PASSWORD= # REDIS_SOCKET= ################################################################################### # Upload File Location # # This is the location where uploaded files are stored. ################################################################################### UPLOAD_LOCATION=/mnt/disk1/im_photos/ ################################################################################### # Typesense ################################################################################### TYPESENSE_API_KEY=sdfuysdfuywnrkvhvslkddgsb #TYPESENSE_ENABLED=false ################################################################################### # Reverse Geocoding # # Reverse geocoding is done locally which has a small impact on memory usage # This memory usage can be altered by changing the REVERSE_GEOCODING_PRECISION variable # This ranges from 0-3 with 3 being the most precise # 3 - Cities > 500 population: ~200MB RAM # 2 - Cities > 1000 population: ~150MB RAM # 1 - Cities > 5000 population: ~80MB RAM # 0 - Cities > 15000 population: ~40MB RAM #################################################################################### # DISABLE_REVERSE_GEOCODING=false # REVERSE_GEOCODING_PRECISION=3 #################################################################################### # WEB - Optional # # Custom message on the login page, should be written in HTML form. # For example: # PUBLIC_LOGIN_PAGE_MESSAGE="This is a demo instance of Immich.<br><br>Email: <i>demo@demo.de</i><br>Password: <i>demo</i>" #################################################################################### PUBLIC_LOGIN_PAGE_MESSAGE= #################################################################################### # Alternative Service Addresses - Optional # # This is an advanced feature for users who may be running their immich services on different hosts. # It will not change which address or port that services bind to within their containers, but it will change where other services look for their peers. # Note: immich-microservices is bound to 3002, but no references are made #################################################################################### IMMICH_WEB_URL=http://immich-web:3000 IMMICH_SERVER_URL=http://immich-server:3001 IMMICH_MACHINE_LEARNING_URL=http://immich-machine-learning:3003 #################################################################################### # Alternative API's External Address - Optional # # This is an advanced feature used to control the public server endpoint returned to clients during Well-known discovery. # You should only use this if you want mobile apps to access the immich API over a custom URL. Do not include trailing slash. # NOTE: At this time, the web app will not be affected by this setting and will continue to use the relative path: /api # Examples: http://localhost:3001, http://immich-api.example.com, etc #################################################################################### #IMMICH_API_URL_EXTERNAL=http://localhost:3001 ``` </details> ### Reproduction steps ```bash Install the latest version, check "Explore" in the web version and things search in the mobile version. ``` ### Additional information The database has over 169,000 photos and 2,000 videos.
Author
Owner

@alextran1502 commented on GitHub (Apr 25, 2023):

Can you try removing the tsdata volume and restarting the stack? It seems to have an issue during indexing to show that error

@alextran1502 commented on GitHub (Apr 25, 2023): Can you try removing the tsdata volume and restarting the stack? It seems to have an issue during indexing to show that error
Author
Owner

@zlo2k commented on GitHub (Apr 25, 2023):

Can you try removing the tsdata volume and restarting the stack? It seems to have an issue during indexing to show that error

I delete the volume, after restarting the same error.

@zlo2k commented on GitHub (Apr 25, 2023): > Can you try removing the tsdata volume and restarting the stack? It seems to have an issue during indexing to show that error I delete the volume, after restarting the same error.
Author
Owner

@alextran1502 commented on GitHub (Apr 25, 2023):

This is pretty strange, can you bring down the whole stack, remove typesense container and typesense volume and bring them up? It would be helpful if you also capture screenshot along the way

@alextran1502 commented on GitHub (Apr 25, 2023): This is pretty strange, can you bring down the whole stack, remove typesense container and typesense volume and bring them up? It would be helpful if you also capture screenshot along the way
Author
Owner

@zlo2k commented on GitHub (Apr 26, 2023):

This is pretty strange, can you bring down the whole stack, remove typesense container and typesense volume and bring them up? It would be helpful if you also capture screenshot along the way

Ok, if I understand you correctly:

Ok, i down full stack
now i remove TS volume
There are no Immich containers
Now we pull and up it
And we have some error
@zlo2k commented on GitHub (Apr 26, 2023): > This is pretty strange, can you bring down the whole stack, remove typesense container and typesense volume and bring them up? It would be helpful if you also capture screenshot along the way Ok, if I understand you correctly: <details><summary>Ok, i down full stack</summary><img src="http://dl4.joxi.net/drive/2023/04/25/0007/2427/493947/47/da6195ad72.jpg"></details> <details><summary>now i remove TS volume</summary> <img src="http://joxi.ru/Q2K0wjnFLjnJXA.jpg"> <img src="http://joxi.ru/52aRwJ1SEOL3eA.jpg"> </details> <details><summary>There are no Immich containers</summary> <img src="http://joxi.ru/8235ygOU9Woj5r.jpg"> </details> <details><summary>Now we pull and up it</summary> <img src="http://joxi.ru/8AnBK3XhzlYGwA.jpg"> </details> <details><summary>And we have some error</summary> <img src="http://joxi.ru/KAxj1gesZkdXBm.jpg"> <img src="http://dl4.joxi.net/drive/2023/04/25/0007/2427/493947/47/1d7f17e6ee.jpg"> </details>
Author
Owner

@jrasm91 commented on GitHub (Apr 26, 2023):

I think indexing is going on in the background (microservices container). Can you share those logs?

@jrasm91 commented on GitHub (Apr 26, 2023): I think indexing is going on in the background (microservices container). Can you share those logs?
Author
Owner

@w00tlarr commented on GitHub (Apr 26, 2023):

Got that issue last week but is expected as I did TYPESENSE_ENABLED = false and I've uploaded more pictures via the web.
Fixed that after I did an Encode Clip job run after I reverted back to TYPESENSE_ENABLED = true;.

Try to run the Encode Clip job again on missing items.

@w00tlarr commented on GitHub (Apr 26, 2023): Got that issue last week but is expected as I did TYPESENSE_ENABLED = false and I've uploaded more pictures via the web. Fixed that after I did an Encode Clip job run after I reverted back to TYPESENSE_ENABLED = true;. Try to run the Encode Clip job again on missing items.
Author
Owner

@zlo2k commented on GitHub (Apr 26, 2023):

I think indexing is going on in the background (microservices container). Can you share those logs?

Yes, found some error in microservices log

microservices log Processor] Initializing Reverse Geocoding /usr/src/app/node_modules/local-reverse-geocoder/index.js:746 throw err; ^

CsvError: Invalid Record Length: expect 19, got 18 on line 60031
at Object.__onRecord (/usr/src/app/node_modules/csv-parse/dist/cjs/index.cjs:940:11)
at Object.parse (/usr/src/app/node_modules/csv-parse/dist/cjs/index.cjs:903:36)
at Parser._flush (/usr/src/app/node_modules/csv-parse/dist/cjs/index.cjs:1336:26)
at Parser.final [as _final] (node:internal/streams/transform:112:25)
at callFinal (node:internal/streams/writable:694:27)
at prefinish (node:internal/streams/writable:723:7)
at finishMaybe (node:internal/streams/writable:733:5)
at afterWrite (node:internal/streams/writable:504:3)
at onwrite (node:internal/streams/writable:477:7)
at Parser.Transform._read (node:internal/streams/transform:245:5) {
code: 'CSV_RECORD_INCONSISTENT_FIELDS_LENGTH',
bytes: 10502144,
comment_lines: 0,
empty_lines: 0,
invalid_field_length: 0,
lines: 60031,
records: 60030,
columns: false,
error: undefined,
header: false,
index: 18,
raw: undefined,
column: 18,
quoting: false,
record: [
'2965092',
'Courtown',
'Courtown',
'Baile na Cuirte,Baile na Cúirte,Courtown,Courtown Harbour,Cuan Bhaile na Cuirte,Cuan Bhaile na Cúirte',
'52.64424',
'-6.22899',
'P',
'PPL',
'IE',
'',
'L',
'30',
'',
'',
'3591',
'',
'9',
''
]
}
[Nest] 1 - 04/26/2023, 12:00:19 AM  LOG [NestFactory] Starting Nest application...
[Nest] 1 - 04/26/2023, 12:00:19 AM  LOG [InstanceLoader] TypeOrmModule dependencies initialized +83ms
[Nest] 1 - 04/26/2023, 12:00:19 AM  LOG [InstanceLoader] BullModule dependencies initialized +2ms
[Nest] 1 - 04/26/2023, 12:00:19 AM  LOG [InstanceLoader] ConfigHostModule dependencies initialized +1ms
[Nest] 1 - 04/26/2023, 12:00:19 AM  LOG [InstanceLoader] DiscoveryModule dependencies initialized +1ms
[Nest] 1 - 04/26/2023, 12:00:19 AM  LOG [InstanceLoader] ConfigModule dependencies initialized +12ms
[Nest] 1 - 04/26/2023, 12:00:19 AM  LOG [InstanceLoader] BullModule dependencies initialized +1ms
[Nest] 1 - 04/26/2023, 12:00:19 AM  LOG [InstanceLoader] BullModule dependencies initialized +1ms
[Nest] 1 - 04/26/2023, 12:00:19 AM  LOG [InstanceLoader] TypeOrmCoreModule dependencies initialized +217ms
[Nest] 1 - 04/26/2023, 12:00:19 AM  LOG [InstanceLoader] TypeOrmModule dependencies initialized +0ms
[Nest] 1 - 04/26/2023, 12:00:19 AM  LOG [InstanceLoader] TypeOrmModule dependencies initialized +0ms
[Nest] 1 - 04/26/2023, 12:00:19 AM  WARN [MetadataExtractionProcessor] Reverse geocoding is enabled
[Nest] 1 - 04/26/2023, 12:00:19 AM  LOG [MetadataExtractionProcessor] Initializing Reverse Geocoding
[Nest] 1 - 04/26/2023, 12:00:19 AM  LOG [InstanceLoader] InfraModule dependencies initialized +36ms
[Nest] 1 - 04/26/2023, 12:00:20 AM  LOG [InstanceLoader] DomainModule dependencies initialized +1ms
[Nest] 1 - 04/26/2023, 12:00:20 AM  LOG [InstanceLoader] MicroservicesModule dependencies initialized +0ms
[Nest] 1 - 04/26/2023, 12:00:20 AM  LOG [NestApplication] Nest application successfully started +24ms
[Nest] 1 - 04/26/2023, 12:00:20 AM  LOG [ImmichMicroservice] Running Immich Microservices in PRODUCTION environment - version 1.54.1 - Listening on port: 3002
[Nest] 1 - 04/26/2023, 12:00:32 AM  LOG [MetadataExtractionProcessor] Reverse Geocoding Initialized

<--- Last few GCs --->

[1:0x7f0f65ef1330] 128394 ms: Scavenge (reduce) 2046.4 (2083.0) -> 2045.5 (2083.2) MB, 13.3 / 0.0 ms (average mu = 0.239, current mu = 0.173) allocation failure
[1:0x7f0f65ef1330] 128903 ms: Mark-sweep (reduce) 2046.4 (2083.2) -> 2041.9 (2083.5) MB, 438.2 / 0.0 ms (+ 2285.4 ms in 592 steps since start of marking, biggest step 33.5 ms, walltime since start of marking 3380 ms) (average mu = 0.235, current mu = 0

<--- JS stacktrace --->

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
[Nest] 1 - 04/26/2023, 12:02:33 AM  LOG [NestFactory] Starting Nest application...
[Nest] 1 - 04/26/2023, 12:02:33 AM  LOG [InstanceLoader] TypeOrmModule dependencies initialized +77ms
[Nest] 1 - 04/26/2023, 12:02:33 AM  LOG [InstanceLoader] BullModule dependencies initialized +0ms
[Nest] 1 - 04/26/2023, 12:02:33 AM  LOG [InstanceLoader] ConfigHostModule dependencies initialized +1ms
[Nest] 1 - 04/26/2023, 12:02:33 AM  LOG [InstanceLoader] DiscoveryModule dependencies initialized +1ms
[Nest] 1 - 04/26/2023, 12:02:33 AM  LOG [InstanceLoader] ConfigModule dependencies initialized +10ms
[Nest] 1 - 04/26/2023, 12:02:33 AM  LOG [InstanceLoader] BullModule dependencies initialized +0ms
[Nest] 1 - 04/26/2023, 12:02:33 AM  LOG [InstanceLoader] BullModule dependencies initialized +1ms
[Nest] 1 - 04/26/2023, 12:02:33 AM  LOG [InstanceLoader] TypeOrmCoreModule dependencies initialized +205ms
[Nest] 1 - 04/26/2023, 12:02:33 AM  LOG [InstanceLoader] TypeOrmModule dependencies initialized +1ms
[Nest] 1 - 04/26/2023, 12:02:33 AM  LOG [InstanceLoader] TypeOrmModule dependencies initialized +0ms
[Nest] 1 - 04/26/2023, 12:02:33 AM  WARN [MetadataExtractionProcessor] Reverse geocoding is enabled
[Nest] 1 - 04/26/2023, 12:02:33 AM  LOG [MetadataExtractionProcessor] Initializing Reverse Geocoding
[Nest] 1 - 04/26/2023, 12:02:43 AM  LOG [InstanceLoader] InfraModule dependencies initialized +9424ms
[Nest] 1 - 04/26/2023, 12:02:43 AM  LOG [InstanceLoader] DomainModule dependencies initialized +1ms
[Nest] 1 - 04/26/2023, 12:02:43 AM  LOG [InstanceLoader] MicroservicesModule dependencies initialized +1ms
[Nest] 1 - 04/26/2023, 12:02:43 AM  LOG [NestApplication] Nest application successfully started +23ms
[Nest] 1 - 04/26/2023, 12:02:43 AM  LOG [ImmichMicroservice] Running Immich Microservices in PRODUCTION environment - version 1.54.1 - Listening on port: 3002
[Nest] 1 - 04/26/2023, 12:02:43 AM  LOG [MetadataExtractionProcessor] Reverse Geocoding Initialized
[Nest] 1 - 04/26/2023, 8:46:55 AM  LOG [NestFactory] Starting Nest application...
[Nest] 1 - 04/26/2023, 8:46:55 AM  LOG [InstanceLoader] TypeOrmModule dependencies initialized +78ms
[Nest] 1 - 04/26/2023, 8:46:55 AM  LOG [InstanceLoader] BullModule dependencies initialized +1ms
[Nest] 1 - 04/26/2023, 8:46:55 AM  LOG [InstanceLoader] ConfigHostModule dependencies initialized +1ms
[Nest] 1 - 04/26/2023, 8:46:55 AM  LOG [InstanceLoader] DiscoveryModule dependencies initialized +1ms
[Nest] 1 - 04/26/2023, 8:46:55 AM  LOG [InstanceLoader] ConfigModule dependencies initialized +9ms
[Nest] 1 - 04/26/2023, 8:46:55 AM  LOG [InstanceLoader] BullModule dependencies initialized +1ms
[Nest] 1 - 04/26/2023, 8:46:55 AM  LOG [InstanceLoader] BullModule dependencies initialized +1ms
[Nest] 1 - 04/26/2023, 8:46:56 AM  LOG [InstanceLoader] TypeOrmCoreModule dependencies initialized +240ms
[Nest] 1 - 04/26/2023, 8:46:56 AM  LOG [InstanceLoader] TypeOrmModule dependencies initialized +1ms
[Nest] 1 - 04/26/2023, 8:46:56 AM  LOG [InstanceLoader] TypeOrmModule dependencies initialized +0ms
[Nest] 1 - 04/26/2023, 8:46:56 AM  WARN [MetadataExtractionProcessor] Reverse geocoding is enabled
[Nest] 1 - 04/26/2023, 8:46:56 AM  LOG [MetadataExtractionProcessor] Initializing Reverse Geocoding
[Nest] 1 - 04/26/2023, 8:46:56 AM  LOG [InstanceLoader] InfraModule dependencies initialized +9ms
[Nest] 1 - 04/26/2023, 8:46:56 AM  LOG [InstanceLoader] DomainModule dependencies initialized +1ms
[Nest] 1 - 04/26/2023, 8:46:56 AM  LOG [InstanceLoader] MicroservicesModule dependencies initialized +0ms
[Nest] 1 - 04/26/2023, 8:46:56 AM  LOG [NestApplication] Nest application successfully started +25ms
[Nest] 1 - 04/26/2023, 8:46:56 AM  LOG [ImmichMicroservice] Running Immich Microservices in PRODUCTION environment - version 1.54.1 - Listening on port: 3002

@zlo2k commented on GitHub (Apr 26, 2023): > I think indexing is going on in the background (microservices container). Can you share those logs? Yes, found some error in microservices log <details><summary>microservices log</summary> Processor] Initializing Reverse Geocoding /usr/src/app/node_modules/local-reverse-geocoder/index.js:746 throw err; ^ CsvError: Invalid Record Length: expect 19, got 18 on line 60031 at Object.__onRecord (/usr/src/app/node_modules/csv-parse/dist/cjs/index.cjs:940:11) at Object.parse (/usr/src/app/node_modules/csv-parse/dist/cjs/index.cjs:903:36) at Parser._flush (/usr/src/app/node_modules/csv-parse/dist/cjs/index.cjs:1336:26) at Parser.final [as _final] (node:internal/streams/transform:112:25) at callFinal (node:internal/streams/writable:694:27) at prefinish (node:internal/streams/writable:723:7) at finishMaybe (node:internal/streams/writable:733:5) at afterWrite (node:internal/streams/writable:504:3) at onwrite (node:internal/streams/writable:477:7) at Parser.Transform._read (node:internal/streams/transform:245:5) { code: 'CSV_RECORD_INCONSISTENT_FIELDS_LENGTH', bytes: 10502144, comment_lines: 0, empty_lines: 0, invalid_field_length: 0, lines: 60031, records: 60030, columns: false, error: undefined, header: false, index: 18, raw: undefined, column: 18, quoting: false, record: [ '2965092', 'Courtown', 'Courtown', 'Baile na Cuirte,Baile na Cúirte,Courtown,Courtown Harbour,Cuan Bhaile na Cuirte,Cuan Bhaile na Cúirte', '52.64424', '-6.22899', 'P', 'PPL', 'IE', '', 'L', '30', '', '', '3591', '', '9', '' ] } [Nest] 1 - 04/26/2023, 12:00:19 AM  LOG [NestFactory] Starting Nest application... [Nest] 1 - 04/26/2023, 12:00:19 AM  LOG [InstanceLoader] TypeOrmModule dependencies initialized +83ms [Nest] 1 - 04/26/2023, 12:00:19 AM  LOG [InstanceLoader] BullModule dependencies initialized +2ms [Nest] 1 - 04/26/2023, 12:00:19 AM  LOG [InstanceLoader] ConfigHostModule dependencies initialized +1ms [Nest] 1 - 04/26/2023, 12:00:19 AM  LOG [InstanceLoader] DiscoveryModule dependencies initialized +1ms [Nest] 1 - 04/26/2023, 12:00:19 AM  LOG [InstanceLoader] ConfigModule dependencies initialized +12ms [Nest] 1 - 04/26/2023, 12:00:19 AM  LOG [InstanceLoader] BullModule dependencies initialized +1ms [Nest] 1 - 04/26/2023, 12:00:19 AM  LOG [InstanceLoader] BullModule dependencies initialized +1ms [Nest] 1 - 04/26/2023, 12:00:19 AM  LOG [InstanceLoader] TypeOrmCoreModule dependencies initialized +217ms [Nest] 1 - 04/26/2023, 12:00:19 AM  LOG [InstanceLoader] TypeOrmModule dependencies initialized +0ms [Nest] 1 - 04/26/2023, 12:00:19 AM  LOG [InstanceLoader] TypeOrmModule dependencies initialized +0ms [Nest] 1 - 04/26/2023, 12:00:19 AM  WARN [MetadataExtractionProcessor] Reverse geocoding is enabled [Nest] 1 - 04/26/2023, 12:00:19 AM  LOG [MetadataExtractionProcessor] Initializing Reverse Geocoding [Nest] 1 - 04/26/2023, 12:00:19 AM  LOG [InstanceLoader] InfraModule dependencies initialized +36ms [Nest] 1 - 04/26/2023, 12:00:20 AM  LOG [InstanceLoader] DomainModule dependencies initialized +1ms [Nest] 1 - 04/26/2023, 12:00:20 AM  LOG [InstanceLoader] MicroservicesModule dependencies initialized +0ms [Nest] 1 - 04/26/2023, 12:00:20 AM  LOG [NestApplication] Nest application successfully started +24ms [Nest] 1 - 04/26/2023, 12:00:20 AM  LOG [ImmichMicroservice] Running Immich Microservices in PRODUCTION environment - version 1.54.1 - Listening on port: 3002 [Nest] 1 - 04/26/2023, 12:00:32 AM  LOG [MetadataExtractionProcessor] Reverse Geocoding Initialized <--- Last few GCs ---> [1:0x7f0f65ef1330] 128394 ms: Scavenge (reduce) 2046.4 (2083.0) -> 2045.5 (2083.2) MB, 13.3 / 0.0 ms (average mu = 0.239, current mu = 0.173) allocation failure [1:0x7f0f65ef1330] 128903 ms: Mark-sweep (reduce) 2046.4 (2083.2) -> 2041.9 (2083.5) MB, 438.2 / 0.0 ms (+ 2285.4 ms in 592 steps since start of marking, biggest step 33.5 ms, walltime since start of marking 3380 ms) (average mu = 0.235, current mu = 0 <--- JS stacktrace ---> FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory [Nest] 1 - 04/26/2023, 12:02:33 AM  LOG [NestFactory] Starting Nest application... [Nest] 1 - 04/26/2023, 12:02:33 AM  LOG [InstanceLoader] TypeOrmModule dependencies initialized +77ms [Nest] 1 - 04/26/2023, 12:02:33 AM  LOG [InstanceLoader] BullModule dependencies initialized +0ms [Nest] 1 - 04/26/2023, 12:02:33 AM  LOG [InstanceLoader] ConfigHostModule dependencies initialized +1ms [Nest] 1 - 04/26/2023, 12:02:33 AM  LOG [InstanceLoader] DiscoveryModule dependencies initialized +1ms [Nest] 1 - 04/26/2023, 12:02:33 AM  LOG [InstanceLoader] ConfigModule dependencies initialized +10ms [Nest] 1 - 04/26/2023, 12:02:33 AM  LOG [InstanceLoader] BullModule dependencies initialized +0ms [Nest] 1 - 04/26/2023, 12:02:33 AM  LOG [InstanceLoader] BullModule dependencies initialized +1ms [Nest] 1 - 04/26/2023, 12:02:33 AM  LOG [InstanceLoader] TypeOrmCoreModule dependencies initialized +205ms [Nest] 1 - 04/26/2023, 12:02:33 AM  LOG [InstanceLoader] TypeOrmModule dependencies initialized +1ms [Nest] 1 - 04/26/2023, 12:02:33 AM  LOG [InstanceLoader] TypeOrmModule dependencies initialized +0ms [Nest] 1 - 04/26/2023, 12:02:33 AM  WARN [MetadataExtractionProcessor] Reverse geocoding is enabled [Nest] 1 - 04/26/2023, 12:02:33 AM  LOG [MetadataExtractionProcessor] Initializing Reverse Geocoding [Nest] 1 - 04/26/2023, 12:02:43 AM  LOG [InstanceLoader] InfraModule dependencies initialized +9424ms [Nest] 1 - 04/26/2023, 12:02:43 AM  LOG [InstanceLoader] DomainModule dependencies initialized +1ms [Nest] 1 - 04/26/2023, 12:02:43 AM  LOG [InstanceLoader] MicroservicesModule dependencies initialized +1ms [Nest] 1 - 04/26/2023, 12:02:43 AM  LOG [NestApplication] Nest application successfully started +23ms [Nest] 1 - 04/26/2023, 12:02:43 AM  LOG [ImmichMicroservice] Running Immich Microservices in PRODUCTION environment - version 1.54.1 - Listening on port: 3002 [Nest] 1 - 04/26/2023, 12:02:43 AM  LOG [MetadataExtractionProcessor] Reverse Geocoding Initialized [Nest] 1 - 04/26/2023, 8:46:55 AM  LOG [NestFactory] Starting Nest application... [Nest] 1 - 04/26/2023, 8:46:55 AM  LOG [InstanceLoader] TypeOrmModule dependencies initialized +78ms [Nest] 1 - 04/26/2023, 8:46:55 AM  LOG [InstanceLoader] BullModule dependencies initialized +1ms [Nest] 1 - 04/26/2023, 8:46:55 AM  LOG [InstanceLoader] ConfigHostModule dependencies initialized +1ms [Nest] 1 - 04/26/2023, 8:46:55 AM  LOG [InstanceLoader] DiscoveryModule dependencies initialized +1ms [Nest] 1 - 04/26/2023, 8:46:55 AM  LOG [InstanceLoader] ConfigModule dependencies initialized +9ms [Nest] 1 - 04/26/2023, 8:46:55 AM  LOG [InstanceLoader] BullModule dependencies initialized +1ms [Nest] 1 - 04/26/2023, 8:46:55 AM  LOG [InstanceLoader] BullModule dependencies initialized +1ms [Nest] 1 - 04/26/2023, 8:46:56 AM  LOG [InstanceLoader] TypeOrmCoreModule dependencies initialized +240ms [Nest] 1 - 04/26/2023, 8:46:56 AM  LOG [InstanceLoader] TypeOrmModule dependencies initialized +1ms [Nest] 1 - 04/26/2023, 8:46:56 AM  LOG [InstanceLoader] TypeOrmModule dependencies initialized +0ms [Nest] 1 - 04/26/2023, 8:46:56 AM  WARN [MetadataExtractionProcessor] Reverse geocoding is enabled [Nest] 1 - 04/26/2023, 8:46:56 AM  LOG [MetadataExtractionProcessor] Initializing Reverse Geocoding [Nest] 1 - 04/26/2023, 8:46:56 AM  LOG [InstanceLoader] InfraModule dependencies initialized +9ms [Nest] 1 - 04/26/2023, 8:46:56 AM  LOG [InstanceLoader] DomainModule dependencies initialized +1ms [Nest] 1 - 04/26/2023, 8:46:56 AM  LOG [InstanceLoader] MicroservicesModule dependencies initialized +0ms [Nest] 1 - 04/26/2023, 8:46:56 AM  LOG [NestApplication] Nest application successfully started +25ms [Nest] 1 - 04/26/2023, 8:46:56 AM  LOG [ImmichMicroservice] Running Immich Microservices in PRODUCTION environment - version 1.54.1 - Listening on port: 3002 </details>
Author
Owner

@zlo2k commented on GitHub (Apr 26, 2023):

Got that issue last week but is expected as I did TYPESENSE_ENABLED = false and I've uploaded more pictures via the web. Fixed that after I did an Encode Clip job run after I reverted back to TYPESENSE_ENABLED = true;.

Try to run the Encode Clip job again on missing items.

I tried this and it didn't work.

@zlo2k commented on GitHub (Apr 26, 2023): > Got that issue last week but is expected as I did TYPESENSE_ENABLED = false and I've uploaded more pictures via the web. Fixed that after I did an Encode Clip job run after I reverted back to TYPESENSE_ENABLED = true;. > > Try to run the Encode Clip job again on missing items. I tried this and it didn't work.
Author
Owner

@jrasm91 commented on GitHub (Apr 26, 2023):

Your container is in an invalid state due to another bug. Delete and recreate the container and it should start up. Then wait for indexing to finish.

@jrasm91 commented on GitHub (Apr 26, 2023): Your container is in an invalid state due to another bug. Delete and recreate the container and it should start up. Then wait for indexing to finish.
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: immich-app/immich#831