immich-app/immich
1
0
Fork 0
mirror of https://github.com/immich-app/immich.git synced 2026-03-01 11:20:12 +03:00
Code Issues 783 Packages Projects Releases 12 Wiki Activity

[PR #25905] fix: set fps_mode=cfr to prevent choppy hw transcodes #18397

New Issue
Open
opened 2026-02-05 16:37:33 +03:00 by OVERLORD · 0 comments
OVERLORD commented 2026-02-05 16:37:33 +03:00
Owner
Copy Link

📋 Pull Request Information

Original PR: https://github.com/immich-app/immich/pull/25905
Author: @aclerici38
Created: 2/5/2026
Status: 🔄 Open

Base: main ← Head: cfr-fix

📝 Commits (1)

  • d783062 fix: set fps_mode=cfr to prevent choppy hw transcodes

📊 Changes

5 files changed (+27 additions, -6 deletions)

View changed files

📝 server/src/repositories/media.repository.ts (+1 -0)
📝 server/src/services/media.service.spec.ts (+10 -5)
📝 server/src/types.ts (+1 -0)
📝 server/src/utils/media.ts (+3 -1)
📝 server/test/fixtures/media.stub.ts (+12 -0)

📄 Description

Description

Fixes https://github.com/immich-app/immich/issues/8657
Possibly https://github.com/immich-app/immich/issues/21484

Disclaimer: I am far from an expert in ffmpeg, hw stacks, etc. This is just an issue that's been bothering me and I took a stab at fixing it. let me know if it's completely off base!

HW Transcoding (QSV specifically for me) on some videos (newer iPhone vids that use variable fps are particularly bad) produces an extremely choppy video across all clients I tried (firefox/chrome, iOS app, immich-gallery on tvOS). For some reason the hw encode/decode paths on these videos produce outputs with out-of-order or nearly identical timestamps. Swapping the -fps_mode arg from passthrough to cfr (constant frame rate) works around this by forcing ffmpeg to interpolate or duplicate frames as needed to output a vid at the specified fps. The trade-off is potential for frame duplication/dropping to enforce CFR.
NOTE: it's possible to just set CFR and ffmpeg will match the original's framerate, but for videos with variable frame rates this can cause the transcode to be much larger than it needs to be. For example, in the video below the frame rate is spec'd at 120 but the average frame rate is about 29 (3597000/123509). It makes more sense to me to use the average framerate for a lossy transcode.

Alternate solutions: In my testing disabling B-frames also helped the choppiness. I feel setting cfr is a more complete fix, however.

This example vid was filmed on an iphone 13. I cut it to 10 seconds and removed the location metadata. It's too big to upload straight to github unfortunately.
original.mov
Here are the timestamps:

ffprobe -v error -select_streams v:0 -show_entries frame=pts_time -of csv=p=0 original.mov | head -10
0.000000,,,,
0.041667,,,,
0.083333,,,,
0.125000,,,,
0.166667,,,,
0.208333,,,,
0.241667,,,,
0.275000,,,,
0.308333,,,,
0.341667,,,,

Before, immich would invoke this ffmpeg command to transcode:

ffmpeg -n 10 /usr/bin/ffmpeg -hwaccel qsv -hwaccel_output_format qsv -async_depth 4 -noautorotate -qsv_device /dev/dri/renderD128 -threads 1 -i /data/library/anthony/2025/12/IMG_3104.mov -y -c:v hevc_qsv -c:a copy -movflags faststart -fps_mode passthrough -map 0:0 -map_metadata -1 -map 0:1 -bf 7 -refs 5 -g 256 -tag:v hvc1 -v verbose -vf scale_qsv=-1:1440:async_depth=4:mode=hq,hwmap=derive_device=opencl,tonemap_opencl=desat=0:format=nv12:matrix=bt709:primaries=bt709:transfer=bt709:range=pc:tonemap=reinhard:tonemap_mode=lum:peak=100,hwmap=derive_device=qsv:reverse=1,format=qsv -preset 4 -global_quality:v 28 /data/encoded-video/d5e4be30-b38f-4c6b-a7b1-0721a11992c5/e9/8a/e98a9912-82db-4bd0-952b-e9f12f6d4214.mp4

Which produced this choppy output
https://github.com/user-attachments/assets/8c0494ff-7bf6-4fd9-99a1-48019d958f5e
And has some frames out of order along with 5 crammed into just 0.0003s

ffprobe -v error -select_streams v:0 -show_entries frame=pts_time -of csv=p=0 before-transcode.mp4 | head -10
0.000000,
0.225195,
0.225130,
0.225260,
0.225065,
0.225391,
0.241667,
0.275000,
0.308333,
0.491862,

After the change to use cfr the ffmpeg command becomes

ffmpeg -n 10 /usr/bin/ffmpeg -hwaccel qsv -hwaccel_output_format qsv -async_depth 4 -noautorotate -qsv_device /dev/dri/renderD128 -threads 1 -i /data/library/anthony/2025/12/IMG_3104.mov -y -c:v hevc_qsv -c:a copy -movflags faststart -fps_mode cfr -r 3597000/123509 -map 0:0 -map_metadata -1 -map 0:1 -bf 7 -refs 5 -g 256 -tag:v hvc1 -v verbose -vf scale_qsv=-1:1440:async_depth=4:mode=hq,hwmap=derive_device=opencl,tonemap_opencl=desat=0:format=nv12:matrix=bt709:primaries=bt709:transfer=bt709:range=pc:tonemap=reinhard:tonemap_mode=lum:peak=100,hwmap=derive_device=qsv:reverse=1,format=qsv -preset 4 -global_quality:v 28 /data/encoded-video/d5e4be30-b38f-4c6b-a7b1-0721a11992c5/e9/8a/e98a9912-82db-4bd0-952b-e9f12f6d4214.mp4

The encoded vid is no longer choppy
https://github.com/user-attachments/assets/82d5c6f4-658c-49d5-bbcf-c39586483be2
And the timestamps are normal-looking

ffprobe -v error -select_streams v:0 -show_entries frame=pts_time -of csv=p=0 cfr.mp4 | head -10
0.000000,
0.034337,
0.068673,
0.103010,
0.137347,
0.171683,
0.206020,
0.240357,
0.274693,
0.309030,

How Has This Been Tested?

  • Pushed an image to docker.io/ant385525/immich-server:cfr-fix and retranscoded my library, verified all videos play well
  • Reviewed ffprobe outputs to see corrected timestamps

Screenshots (if appropriate)

Checklist:

  • I have performed a self-review of my own code
  • I have made corresponding changes to the documentation if applicable
  • I have no unrelated changes in the PR.
  • I have confirmed that any new dependencies are strictly necessary.
  • I have written tests for new code (if applicable)
  • I have followed naming conventions/patterns in the surrounding code
  • All code in src/services/ uses repositories implementations for database calls, filesystem operations, etc.
  • All code in src/repositories/ is pretty basic/simple and does not have any immich specific logic (that belongs in src/services/)

Please describe to which degree, if any, an LLM was used in creating this pull request.

Used claude code to debug the video files and parse ffprobe/ffmpeg output. Impl logic is my own

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/immich-app/immich/pull/25905 **Author:** [@aclerici38](https://github.com/aclerici38) **Created:** 2/5/2026 **Status:** 🔄 Open **Base:** `main` ← **Head:** `cfr-fix` --- ### 📝 Commits (1) - [`d783062`](https://github.com/immich-app/immich/commit/d783062cfbdb3244e5762398ff23efe9ed4c30dc) fix: set fps_mode=cfr to prevent choppy hw transcodes ### 📊 Changes **5 files changed** (+27 additions, -6 deletions) <details> <summary>View changed files</summary> 📝 `server/src/repositories/media.repository.ts` (+1 -0) 📝 `server/src/services/media.service.spec.ts` (+10 -5) 📝 `server/src/types.ts` (+1 -0) 📝 `server/src/utils/media.ts` (+3 -1) 📝 `server/test/fixtures/media.stub.ts` (+12 -0) </details> ### 📄 Description ## Description <!--- Describe your changes in detail --> <!--- Why is this change required? What problem does it solve? --> <!--- If it fixes an open issue, please link to the issue here. --> Fixes https://github.com/immich-app/immich/issues/8657 Possibly https://github.com/immich-app/immich/issues/21484 Disclaimer: I am far from an expert in ffmpeg, hw stacks, etc. This is just an issue that's been bothering me and I took a stab at fixing it. let me know if it's completely off base! HW Transcoding (QSV specifically for me) on some videos (newer iPhone vids that use variable fps are particularly bad) produces an extremely choppy video across all clients I tried (firefox/chrome, iOS app, immich-gallery on tvOS). For some reason the hw encode/decode paths on these videos produce outputs with out-of-order or nearly identical timestamps. Swapping the `-fps_mode` arg from `passthrough` to `cfr` (constant frame rate) works around this by forcing ffmpeg to interpolate or duplicate frames as needed to output a vid at the specified fps. The trade-off is potential for frame duplication/dropping to enforce CFR. NOTE: it's possible to just set CFR and ffmpeg will match the original's framerate, but for videos with variable frame rates this can cause the transcode to be much larger than it needs to be. For example, in the video below the frame rate is spec'd at 120 but the average frame rate is about 29 (3597000/123509). It makes more sense to me to use the average framerate for a lossy transcode. Alternate solutions: In my testing disabling B-frames also helped the choppiness. I feel setting `cfr` is a more complete fix, however. This example vid was filmed on an iphone 13. I cut it to 10 seconds and removed the location metadata. It's too big to upload straight to github unfortunately. [original.mov](https://drive.google.com/file/d/10ilE2bbbeE6FAryfejQeWCt63Vbaby6n/view?usp=drive_link) Here are the timestamps: ``` ffprobe -v error -select_streams v:0 -show_entries frame=pts_time -of csv=p=0 original.mov | head -10 0.000000,,,, 0.041667,,,, 0.083333,,,, 0.125000,,,, 0.166667,,,, 0.208333,,,, 0.241667,,,, 0.275000,,,, 0.308333,,,, 0.341667,,,, ``` Before, immich would invoke this ffmpeg command to transcode: ``` ffmpeg -n 10 /usr/bin/ffmpeg -hwaccel qsv -hwaccel_output_format qsv -async_depth 4 -noautorotate -qsv_device /dev/dri/renderD128 -threads 1 -i /data/library/anthony/2025/12/IMG_3104.mov -y -c:v hevc_qsv -c:a copy -movflags faststart -fps_mode passthrough -map 0:0 -map_metadata -1 -map 0:1 -bf 7 -refs 5 -g 256 -tag:v hvc1 -v verbose -vf scale_qsv=-1:1440:async_depth=4:mode=hq,hwmap=derive_device=opencl,tonemap_opencl=desat=0:format=nv12:matrix=bt709:primaries=bt709:transfer=bt709:range=pc:tonemap=reinhard:tonemap_mode=lum:peak=100,hwmap=derive_device=qsv:reverse=1,format=qsv -preset 4 -global_quality:v 28 /data/encoded-video/d5e4be30-b38f-4c6b-a7b1-0721a11992c5/e9/8a/e98a9912-82db-4bd0-952b-e9f12f6d4214.mp4 ``` Which produced this choppy output https://github.com/user-attachments/assets/8c0494ff-7bf6-4fd9-99a1-48019d958f5e And has some frames out of order along with 5 crammed into just 0.0003s ``` ffprobe -v error -select_streams v:0 -show_entries frame=pts_time -of csv=p=0 before-transcode.mp4 | head -10 0.000000, 0.225195, 0.225130, 0.225260, 0.225065, 0.225391, 0.241667, 0.275000, 0.308333, 0.491862, ``` After the change to use `cfr` the ffmpeg command becomes ``` ffmpeg -n 10 /usr/bin/ffmpeg -hwaccel qsv -hwaccel_output_format qsv -async_depth 4 -noautorotate -qsv_device /dev/dri/renderD128 -threads 1 -i /data/library/anthony/2025/12/IMG_3104.mov -y -c:v hevc_qsv -c:a copy -movflags faststart -fps_mode cfr -r 3597000/123509 -map 0:0 -map_metadata -1 -map 0:1 -bf 7 -refs 5 -g 256 -tag:v hvc1 -v verbose -vf scale_qsv=-1:1440:async_depth=4:mode=hq,hwmap=derive_device=opencl,tonemap_opencl=desat=0:format=nv12:matrix=bt709:primaries=bt709:transfer=bt709:range=pc:tonemap=reinhard:tonemap_mode=lum:peak=100,hwmap=derive_device=qsv:reverse=1,format=qsv -preset 4 -global_quality:v 28 /data/encoded-video/d5e4be30-b38f-4c6b-a7b1-0721a11992c5/e9/8a/e98a9912-82db-4bd0-952b-e9f12f6d4214.mp4 ``` The encoded vid is no longer choppy https://github.com/user-attachments/assets/82d5c6f4-658c-49d5-bbcf-c39586483be2 And the timestamps are normal-looking ``` ffprobe -v error -select_streams v:0 -show_entries frame=pts_time -of csv=p=0 cfr.mp4 | head -10 0.000000, 0.034337, 0.068673, 0.103010, 0.137347, 0.171683, 0.206020, 0.240357, 0.274693, 0.309030, ``` ## How Has This Been Tested? <!-- Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration --> - [x] Pushed an image to `docker.io/ant385525/immich-server:cfr-fix` and retranscoded my library, verified all videos play well - [x] Reviewed ffprobe outputs to see corrected timestamps <details><summary><h2>Screenshots (if appropriate)</h2></summary> <!-- Images go below this line. --> </details> <!-- API endpoint changes (if relevant) ## API Changes The `/api/something` endpoint is now `/api/something-else` --> ## Checklist: - [x] I have performed a self-review of my own code - [ ] I have made corresponding changes to the documentation if applicable - [x] I have no unrelated changes in the PR. - [ ] I have confirmed that any new dependencies are strictly necessary. - [ ] I have written tests for new code (if applicable) - [ ] I have followed naming conventions/patterns in the surrounding code - [ ] All code in `src/services/` uses repositories implementations for database calls, filesystem operations, etc. - [x] All code in `src/repositories/` is pretty basic/simple and does not have any immich specific logic (that belongs in `src/services/`) ## Please describe to which degree, if any, an LLM was used in creating this pull request. Used claude code to debug the video files and parse ffprobe/ffmpeg output. Impl logic is my own --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
OVERLORD added the pull-request label 2026-02-05 16:37:33 +03:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
accessibility
changelog:enhancement
changelog:security
changelog:skip
changelog:translation
cli
date-time
dependencies
documentation
external-library
format
good first issue
mobile-beta
mobile-beta
mobile-beta
needs-answer
nice to have
pull-request
Mirrored from GitHub Pull Request
 sharing
tech-debt
📱mobile
🖥️web
🗄️server
🧠machine-learning
No Label pull-request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
OVERLORD
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: immich-app/immich#18397
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?