[BUG] Large videos are not uploaded to the server #226

Closed
opened 2026-02-04 18:50:55 +03:00 by OVERLORD · 8 comments
Owner

Originally created by @kai23 on GitHub (Aug 24, 2022).

Describe the bug
Some videos (3 in my collection) can't be uploaded

Task List

  • I have read thoroughly the README setup and installation instructions.
  • I have included my docker-compose file.
immich-server:
  <<: *common-keys-core # See EXTENSION FIELDS at the top
  image: altran1502/immich-server:release
  entrypoint: ["/bin/sh", "./start-server.sh"]
  volumes:
    - ${UPLOAD_LOCATION}:/usr/src/app/upload
  env_file:
    - .env
  environment:
    - NODE_ENV=production
    - DB_PASSWORD=${DB_IMMICH_PASSWORD}
    - DB_USERNAME=${DB_IMMICH_USERNAME}
    - DB_DATABASE_NAME=${DB_IMMICH_DATABASE_NAME}
  depends_on:
    - redis
    - database

immich-microservices:
  <<: *common-keys-core # See EXTENSION FIELDS at the top
  image: altran1502/immich-server:release
  entrypoint: ["/bin/sh", "./start-microservices.sh"]
  volumes:
    - ${UPLOAD_LOCATION}:/usr/src/app/upload
  env_file:
    - .env
  environment:
    - NODE_ENV=production
    - DB_PASSWORD=${DB_IMMICH_PASSWORD}
    - DB_USERNAME=${DB_IMMICH_USERNAME}
    - DB_DATABASE_NAME=${DB_IMMICH_DATABASE_NAME}
  depends_on:
    - redis
    - database
  restart: always

immich-machine-learning:
  <<: *common-keys-core # See EXTENSION FIELDS at the top
  image: altran1502/immich-machine-learning:release
  entrypoint: ["/bin/sh", "./entrypoint.sh"]
  volumes:
    - ${UPLOAD_LOCATION}:/usr/src/app/upload
  env_file:
    - .env
  environment:
    - NODE_ENV=production
    - DB_PASSWORD=${DB_IMMICH_PASSWORD}
    - DB_USERNAME=${DB_IMMICH_USERNAME}
    - DB_DATABASE_NAME=${DB_IMMICH_DATABASE_NAME}
  depends_on:
    - database

immich-web:
  <<: *common-keys-core # See EXTENSION FIELDS at the top
  image: altran1502/immich-web:release
  entrypoint: ["/bin/sh", "./entrypoint.sh"]
  env_file:
    - .env

redis:
  <<: *common-keys-core # See EXTENSION FIELDS at the top
  container_name: immich_redis
  image: redis:6.2

database:
  <<: *common-keys-core # See EXTENSION FIELDS at the top
  container_name: immich_postgres
  image: postgres:14
  env_file:
    - .env
  environment:
    POSTGRES_PASSWORD: ${DB_IMMICH_PASSWORD}
    POSTGRES_USER: ${DB_IMMICH_USERNAME}
    POSTGRES_DB: ${DB_IMMICH_DATABASE_NAME}
    PG_DATA: /var/lib/postgresql/data
  volumes:
    - pgdata:/var/lib/postgresql/data

immich-proxy:
  <<: *common-keys-core # See EXTENSION FIELDS at the top
  container_name: immich_proxy
  image: altran1502/immich-proxy:release
  # ports:
  #   - 2283:80
  logging:
    driver: none
  depends_on:
    - immich-server
  • I have included my redacted .env file.
###################################################################################
# IMMICH
###################################################################################

DB_IMMICH_HOSTNAME=immich_postgres
DB_IMMICH_USERNAME=XXX # <- obfuscated
DB_IMMICH_PASSWORD=XXX # <- obfuscated
DB_IMMICH_DATABASE_NAME=immich

REDIS_IMMICH_HOSTNAME=immich_redis

UPLOAD_LOCATION=XXX # <- obfuscated

JWT_SECRET=XXX # <- obfuscated

ENABLE_MAPBOX=false
MAPBOX_KEY=

VITE_LOGIN_PAGE_MESSAGE=
  • I have included information on my machine, and environment.
Linux 5.4.0-124-generic #140-Ubuntu SMP Thu Aug 4 02:23:37 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

No LSB modules are available.
Distributor ID:	Ubuntu
Description:	Ubuntu 20.04.4 LTS
Release:	20.04
Codename:	focal

To Reproduce

I don't really know why these three videos in particular to be honest. Maybe because of the extension ?

Expected behavior
I expect to have the videos uploaded

Screenshots
If applicable, add screenshots to help explain your problem.

System

  • Phone OS [Android]: 12 (OneUI 4.1)
  • Server Version: v1.25.0
  • Mobile App Version: 1.25.0

Additional context

Here is the stacktrace of the app :

08-24 12:07:38.492 10732 10764 I flutter : ERROR backupAsset: SocketException: Write failed (OS Error: Connection reset by peer, errno = 104), address = photos.milo.ovh, port = 41506

And in the microservice app :

today at 01:00:02[Nest] 6  - 08/24/2022, 1:00:02 AM     LOG [mp4Conversion] Start Converting Video
today at 01:00:02[Nest] 6  - 08/24/2022, 1:00:02 AM   ERROR [mp4Conversion] Cannot Convert Video Error: ffmpeg exited with code 1: upload/961f1e40-7dc7-45d9-b568-b4f3b0bff525/original/fb493a08-1700-535e-94a9-c55d8a50d789/6c601d5b-62b6-4aa3-bad9-7e3034c9261e.mov: Invalid data found when processing input
Originally created by @kai23 on GitHub (Aug 24, 2022). **Describe the bug** Some videos (3 in my collection) can't be uploaded **Task List** - [X] I have read thoroughly the README setup and installation instructions. - [X] I have included my `docker-compose` file. ```yaml immich-server: <<: *common-keys-core # See EXTENSION FIELDS at the top image: altran1502/immich-server:release entrypoint: ["/bin/sh", "./start-server.sh"] volumes: - ${UPLOAD_LOCATION}:/usr/src/app/upload env_file: - .env environment: - NODE_ENV=production - DB_PASSWORD=${DB_IMMICH_PASSWORD} - DB_USERNAME=${DB_IMMICH_USERNAME} - DB_DATABASE_NAME=${DB_IMMICH_DATABASE_NAME} depends_on: - redis - database immich-microservices: <<: *common-keys-core # See EXTENSION FIELDS at the top image: altran1502/immich-server:release entrypoint: ["/bin/sh", "./start-microservices.sh"] volumes: - ${UPLOAD_LOCATION}:/usr/src/app/upload env_file: - .env environment: - NODE_ENV=production - DB_PASSWORD=${DB_IMMICH_PASSWORD} - DB_USERNAME=${DB_IMMICH_USERNAME} - DB_DATABASE_NAME=${DB_IMMICH_DATABASE_NAME} depends_on: - redis - database restart: always immich-machine-learning: <<: *common-keys-core # See EXTENSION FIELDS at the top image: altran1502/immich-machine-learning:release entrypoint: ["/bin/sh", "./entrypoint.sh"] volumes: - ${UPLOAD_LOCATION}:/usr/src/app/upload env_file: - .env environment: - NODE_ENV=production - DB_PASSWORD=${DB_IMMICH_PASSWORD} - DB_USERNAME=${DB_IMMICH_USERNAME} - DB_DATABASE_NAME=${DB_IMMICH_DATABASE_NAME} depends_on: - database immich-web: <<: *common-keys-core # See EXTENSION FIELDS at the top image: altran1502/immich-web:release entrypoint: ["/bin/sh", "./entrypoint.sh"] env_file: - .env redis: <<: *common-keys-core # See EXTENSION FIELDS at the top container_name: immich_redis image: redis:6.2 database: <<: *common-keys-core # See EXTENSION FIELDS at the top container_name: immich_postgres image: postgres:14 env_file: - .env environment: POSTGRES_PASSWORD: ${DB_IMMICH_PASSWORD} POSTGRES_USER: ${DB_IMMICH_USERNAME} POSTGRES_DB: ${DB_IMMICH_DATABASE_NAME} PG_DATA: /var/lib/postgresql/data volumes: - pgdata:/var/lib/postgresql/data immich-proxy: <<: *common-keys-core # See EXTENSION FIELDS at the top container_name: immich_proxy image: altran1502/immich-proxy:release # ports: # - 2283:80 logging: driver: none depends_on: - immich-server ``` - [X] I have included my redacted `.env` file. ```env ################################################################################### # IMMICH ################################################################################### DB_IMMICH_HOSTNAME=immich_postgres DB_IMMICH_USERNAME=XXX # <- obfuscated DB_IMMICH_PASSWORD=XXX # <- obfuscated DB_IMMICH_DATABASE_NAME=immich REDIS_IMMICH_HOSTNAME=immich_redis UPLOAD_LOCATION=XXX # <- obfuscated JWT_SECRET=XXX # <- obfuscated ENABLE_MAPBOX=false MAPBOX_KEY= VITE_LOGIN_PAGE_MESSAGE= ``` - [X] I have included information on my machine, and environment. ``` Linux 5.4.0-124-generic #140-Ubuntu SMP Thu Aug 4 02:23:37 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux No LSB modules are available. Distributor ID: Ubuntu Description: Ubuntu 20.04.4 LTS Release: 20.04 Codename: focal ``` **To Reproduce** I don't really know why these three videos in particular to be honest. Maybe because of the extension ? **Expected behavior** I expect to have the videos uploaded **Screenshots** [If applicable, add screenshots to help explain your problem.](https://streamable.com/hfurpc) **System** - Phone OS [Android]: `12 (OneUI 4.1)` - Server Version: `v1.25.0` - Mobile App Version: `1.25.0` **Additional context** Here is the stacktrace of the app : ``` 08-24 12:07:38.492 10732 10764 I flutter : ERROR backupAsset: SocketException: Write failed (OS Error: Connection reset by peer, errno = 104), address = photos.milo.ovh, port = 41506 ``` And in the microservice app : ```logs today at 01:00:02[Nest] 6 - 08/24/2022, 1:00:02 AM LOG [mp4Conversion] Start Converting Video today at 01:00:02[Nest] 6 - 08/24/2022, 1:00:02 AM ERROR [mp4Conversion] Cannot Convert Video Error: ffmpeg exited with code 1: upload/961f1e40-7dc7-45d9-b568-b4f3b0bff525/original/fb493a08-1700-535e-94a9-c55d8a50d789/6c601d5b-62b6-4aa3-bad9-7e3034c9261e.mov: Invalid data found when processing input ```
Author
Owner

@alextran1502 commented on GitHub (Aug 24, 2022):

Is the video something you can share?

@alextran1502 commented on GitHub (Aug 24, 2022): Is the video something you can share?
Author
Owner

@kai23 commented on GitHub (Aug 24, 2022):

To check, I've downloaded the 6c601d5b-62b6-4aa3-bad9-7e3034c9261e.mov but it can't be opened with VLC, which is surely really strange.

I'll try to delete it on the server and do the upload again

@kai23 commented on GitHub (Aug 24, 2022): To check, I've downloaded the `6c601d5b-62b6-4aa3-bad9-7e3034c9261e.mov` but it can't be opened with VLC, which is surely really strange. I'll try to delete it on the server and do the upload again
Author
Owner

@alextran1502 commented on GitHub (Aug 25, 2022):

Can you help me check if the video is uploading can be played on your phone?
Screen Shot 2022-08-24 at 21 33 00

@alextran1502 commented on GitHub (Aug 25, 2022): Can you help me check if the video is uploading can be played on your phone? <img width="303" alt="Screen Shot 2022-08-24 at 21 33 00" src="https://user-images.githubusercontent.com/27055614/186575740-ae59c8bb-0ca6-435e-b546-04c1c6d96db9.png">
Author
Owner

@kai23 commented on GitHub (Aug 25, 2022):

Maybe I was misled by the logs on the microservices part, as it should be three different files. It's maybe another bug.

Here are the "true" logs for the three files that can't be uploaded :

08-25 09:29:06.524 15134  8820 I flutter : ERROR backupAsset: SocketException: Write failed (OS Error: Connection reset by peer, errno = 104), address = photos.milo.ovh, port = 41926
08-25 09:29:36.527 15134  8820 I flutter : ERROR backupAsset: SocketException: Write failed (OS Error: Connection reset by peer, errno = 104), address = photos.milo.ovh, port = 41944
08-25 09:30:06.522 15134  8820 I flutter : ERROR backupAsset: SocketException: Read failed (OS Error: Connection reset by peer, errno = 104), address = photos.milo.ovh, port = 41948

Do you have any logs on the server I could "activate" to see what happen during the upload ? It always stuck at any point, like here

image

It stops uploading just after the logs I've just sent. Pretty weird.

@kai23 commented on GitHub (Aug 25, 2022): Maybe I was misled by the logs on the microservices part, as it should be three different files. It's maybe another bug. Here are the "true" logs for the three files that can't be uploaded : ``` 08-25 09:29:06.524 15134 8820 I flutter : ERROR backupAsset: SocketException: Write failed (OS Error: Connection reset by peer, errno = 104), address = photos.milo.ovh, port = 41926 08-25 09:29:36.527 15134 8820 I flutter : ERROR backupAsset: SocketException: Write failed (OS Error: Connection reset by peer, errno = 104), address = photos.milo.ovh, port = 41944 08-25 09:30:06.522 15134 8820 I flutter : ERROR backupAsset: SocketException: Read failed (OS Error: Connection reset by peer, errno = 104), address = photos.milo.ovh, port = 41948 ``` Do you have any logs on the server I could "activate" to see what happen during the upload ? It always stuck at any point, like here ![image](https://user-images.githubusercontent.com/796803/186604209-a8f8cce0-4d25-4f18-b62e-704067397620.png) It stops uploading just after the logs I've just sent. Pretty weird.
Author
Owner

@alextran1502 commented on GitHub (Aug 25, 2022):

If read and write failed on the mobile side, I assume the files are corrupted. Can you pull those files from your phone to try playing them on your computer?

@alextran1502 commented on GitHub (Aug 25, 2022): If read and write failed on the mobile side, I assume the files are corrupted. Can you pull those files from your phone to try playing them on your computer?
Author
Owner

@kai23 commented on GitHub (Aug 25, 2022):

I can confirm you that the videos in question are playable.
But they are also really big : 2.2 Gb, 3.29 Gb and 2.11 Gb.

There are the only videos more than 800 Mb on my phone (don't know if it helps)

@kai23 commented on GitHub (Aug 25, 2022): I can confirm you that the videos in question are playable. But they are also really big : 2.2 Gb, 3.29 Gb and 2.11 Gb. There are the only videos more than 800 Mb on my phone (don't know if it helps)
Author
Owner

@alextran1502 commented on GitHub (Aug 26, 2022):

Are you using your own reverse proxy? Is there any size limitation on your frontend? Can you try upload using local ip instead of your dns?

@alextran1502 commented on GitHub (Aug 26, 2022): Are you using your own reverse proxy? Is there any size limitation on your frontend? Can you try upload using local ip instead of your dns?
Author
Owner

@kai23 commented on GitHub (Aug 26, 2022):

It was a nice lead, thank you !

For those stumble upon this issue, if use Nginx Package Manager (npm) you should add this in the advanced tab of the proxy :

proxy_set_header Host $host;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_max_temp_file_size 16384m;
client_max_body_size 0;
image

Then, is seems to work as I have some other errors in the "microservices" service :

[Nest] 6  - 08/26/2022, 9:13:44 AM     LOG [generateJPEGThumbnail] Start Generating Video Thumbnail
Error in video metadata extraction QueryFailedError: value "2677588375" is out of range for type integer
    at PostgresQueryRunner.query (/usr/src/app/node_modules/typeorm/driver/postgres/PostgresQueryRunner.js:211:19)
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at async InsertQueryBuilder.execute (/usr/src/app/node_modules/typeorm/query-builder/InsertQueryBuilder.js:106:33)
    at async SubjectExecutor.executeInsertOperations (/usr/src/app/node_modules/typeorm/persistence/SubjectExecutor.js:260:42)
    at async SubjectExecutor.execute (/usr/src/app/node_modules/typeorm/persistence/SubjectExecutor.js:92:9)
    at async EntityPersistExecutor.execute (/usr/src/app/node_modules/typeorm/persistence/EntityPersistExecutor.js:147:21)
    at async MetadataExtractionProcessor.extractVideoMetadata (/usr/src/app/dist/apps/microservices/apps/microservices/src/processors/metadata-extraction.processor.js:259:13) {
  query: 'INSERT INTO "exif"("assetId", "description", "exifImageWidth", "exifImageHeight", "fileSizeInByte", "orientation", "dateTimeOriginal", "modifyDate", "latitude", "longitude", "city", "state", "country", "make", "model", "imageName", "lensModel", "fNumber", "focalLength", "iso", "exposureTime", "fps", "exifTextSearchableColumn") VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13, DEFAULT, DEFAULT, $14, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, $15, DEFAULT) RETURNING "id", "description"',
  parameters: [
    '2422f446-6cf3-4450-ba4f-4f2a7b467e4b',
    '',
    3840,
    2160,
    2677588375,
    '-90',
    2022-06-04T14:03:48.000Z,
    null,
    null,
    null,
    null,
    null,
    null,
    '20220604_155825',
    60
  ],
  driverError: error: value "2677588375" is out of range for type integer
      at Parser.parseErrorMessage (/usr/src/app/node_modules/pg-protocol/dist/parser.js:287:98)
      at Parser.handlePacket (/usr/src/app/node_modules/pg-protocol/dist/parser.js:126:29)
      at Parser.parse (/usr/src/app/node_modules/pg-protocol/dist/parser.js:39:38)
      at Socket.<anonymous> (/usr/src/app/node_modules/pg-protocol/dist/index.js:11:42)
      at Socket.emit (node:events:527:28)
      at addChunk (node:internal/streams/readable:315:12)
      at readableAddChunk (node:internal/streams/readable:289:9)
      at Socket.Readable.push (node:internal/streams/readable:228:10)
      at TCP.onStreamRead (node:internal/stream_base_commons:190:23) {
    length: 148,
    severity: 'ERROR',
    code: '22003',
    detail: undefined,
    hint: undefined,
    position: undefined,
    internalPosition: undefined,
    internalQuery: undefined,
    where: "unnamed portal parameter $5 = '...'",
    schema: undefined,
    table: undefined,
    column: undefined,
    dataType: undefined,
    constraint: undefined,
    file: 'numutils.c',
    line: '314',
    routine: 'pg_strtoint32'
  },
  length: 148,
  severity: 'ERROR',
  code: '22003',
  detail: undefined,
  hint: undefined,
  position: undefined,
  internalPosition: undefined,
  internalQuery: undefined,
  where: "unnamed portal parameter $5 = '...'",
  schema: undefined,
  table: undefined,
  column: undefined,
  dataType: undefined,
  constraint: undefined,
  file: 'numutils.c',
  line: '314',
  routine: 'pg_strtoint32'
}
[Nest] 6  - 08/26/2022, 9:13:50 AM     LOG [generateJPEGThumbnail] Generating Video Thumbnail Success 2422f446-6cf3-4450-ba4f-4f2a7b467e4b
@kai23 commented on GitHub (Aug 26, 2022): It was a nice lead, thank you ! For those stumble upon this issue, if use Nginx Package Manager (npm) you should add this in the advanced tab of the proxy : ``` proxy_set_header Host $host; proxy_set_header X-Forwarded-Proto $scheme; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_max_temp_file_size 16384m; client_max_body_size 0; ``` <img width="510" alt="image" src="https://user-images.githubusercontent.com/796803/186844782-eec056d6-0292-4af7-a3e0-4f31a6f9ba42.png"> Then, is _seems_ to work as I have some other errors in the "microservices" service : ``` [Nest] 6 - 08/26/2022, 9:13:44 AM LOG [generateJPEGThumbnail] Start Generating Video Thumbnail Error in video metadata extraction QueryFailedError: value "2677588375" is out of range for type integer at PostgresQueryRunner.query (/usr/src/app/node_modules/typeorm/driver/postgres/PostgresQueryRunner.js:211:19) at runMicrotasks (<anonymous>) at processTicksAndRejections (node:internal/process/task_queues:96:5) at async InsertQueryBuilder.execute (/usr/src/app/node_modules/typeorm/query-builder/InsertQueryBuilder.js:106:33) at async SubjectExecutor.executeInsertOperations (/usr/src/app/node_modules/typeorm/persistence/SubjectExecutor.js:260:42) at async SubjectExecutor.execute (/usr/src/app/node_modules/typeorm/persistence/SubjectExecutor.js:92:9) at async EntityPersistExecutor.execute (/usr/src/app/node_modules/typeorm/persistence/EntityPersistExecutor.js:147:21) at async MetadataExtractionProcessor.extractVideoMetadata (/usr/src/app/dist/apps/microservices/apps/microservices/src/processors/metadata-extraction.processor.js:259:13) { query: 'INSERT INTO "exif"("assetId", "description", "exifImageWidth", "exifImageHeight", "fileSizeInByte", "orientation", "dateTimeOriginal", "modifyDate", "latitude", "longitude", "city", "state", "country", "make", "model", "imageName", "lensModel", "fNumber", "focalLength", "iso", "exposureTime", "fps", "exifTextSearchableColumn") VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13, DEFAULT, DEFAULT, $14, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, $15, DEFAULT) RETURNING "id", "description"', parameters: [ '2422f446-6cf3-4450-ba4f-4f2a7b467e4b', '', 3840, 2160, 2677588375, '-90', 2022-06-04T14:03:48.000Z, null, null, null, null, null, null, '20220604_155825', 60 ], driverError: error: value "2677588375" is out of range for type integer at Parser.parseErrorMessage (/usr/src/app/node_modules/pg-protocol/dist/parser.js:287:98) at Parser.handlePacket (/usr/src/app/node_modules/pg-protocol/dist/parser.js:126:29) at Parser.parse (/usr/src/app/node_modules/pg-protocol/dist/parser.js:39:38) at Socket.<anonymous> (/usr/src/app/node_modules/pg-protocol/dist/index.js:11:42) at Socket.emit (node:events:527:28) at addChunk (node:internal/streams/readable:315:12) at readableAddChunk (node:internal/streams/readable:289:9) at Socket.Readable.push (node:internal/streams/readable:228:10) at TCP.onStreamRead (node:internal/stream_base_commons:190:23) { length: 148, severity: 'ERROR', code: '22003', detail: undefined, hint: undefined, position: undefined, internalPosition: undefined, internalQuery: undefined, where: "unnamed portal parameter $5 = '...'", schema: undefined, table: undefined, column: undefined, dataType: undefined, constraint: undefined, file: 'numutils.c', line: '314', routine: 'pg_strtoint32' }, length: 148, severity: 'ERROR', code: '22003', detail: undefined, hint: undefined, position: undefined, internalPosition: undefined, internalQuery: undefined, where: "unnamed portal parameter $5 = '...'", schema: undefined, table: undefined, column: undefined, dataType: undefined, constraint: undefined, file: 'numutils.c', line: '314', routine: 'pg_strtoint32' } [Nest] 6 - 08/26/2022, 9:13:50 AM LOG [generateJPEGThumbnail] Generating Video Thumbnail Success 2422f446-6cf3-4450-ba4f-4f2a7b467e4b ```
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: immich-app/immich#226