mirror of
https://github.com/immich-app/immich.git
synced 2026-02-05 08:41:28 +03:00
[BUG] Web client has major performance issues with large image library #1354
Closed
opened 2026-02-05 01:26:41 +03:00 by OVERLORD
·
57 comments
No Branch/Tag Specified
main
fix/web-people-hidden-state
fix-filename-search-label
chore/yank-cloud-id
chore/oauth-labels
renovate/machine-learning
uhthomas/mobile-fix-app-bar-fade
feat/debug-schema
renovate/typescript-projects
fix/25803
feat/asset-file-apis
chore/translations
fix/web-switch-label-clickable
release/next
fix/timezones
fix/time-zone-upserts
midzelis/wip
push-zpwsovysllvn
push-nwxlpmyzkyrl
push-nvnkszuqwppm
renovate/github-actions
push-smstsuupsowp
refactor/adaptive_image
push-olwpzvrxnomt
push-lmxsupnmxspl
feat/web-chromecast-video-looping
feat/use-native-clients
renovate/flutter
fix/create-face-edited
fix/mobile-ios-mtls
docs/contributing
docs/mise-mobile
renovate/grafana-monorepo
feature/bottom-buttons-order
feat/immich-mobile-ui-showcase
refactor/consolidate-image-requests
renovate/connectivity_plus-7.x
renovate/major-vitest-monorepo
renovate/pypi-python-multipart-vulnerability
fix/mobile-people-query
sqlite_thumbs
feat/html-text
chore/no-macro-validation
refactor/purchase-store
uhthomas/mobile-fix-asset-jump
feat/pano-ocr
feat/shared-link-login
fix/database-backup-db-names
fix-keep-correct-ios-shared-album-asset
fix-memory-generation-and-display
feat/verify-permissions
refactor/album-service-small-tests
fix/ml-rocm-build
fix/flipped-dimensions-mobile
push-vpxwmwwxwnvw
fix-migration-width-height
refactor/more-queries
revert/prettier-translations
refactor/asset-service-queries
fix/locale-settings-desc
chore/add-debug-log
feat/edit-filters
shared-deep-link-handler
feat/mobile-editing
feat/thumbnail-native-clients
feat/platform-clients
feat/integrity-checks-izzy
fix/foreground-cloud-sync
feat/dynamic-layout
filter-by-person
feat/csp
refactor/sidebar
fix/disable-editing
fix/view-timeline-deeplink
image-zoom-on-slow-connection
fix-consider-dar-for-video-dimension
fix/merged-edited-assets
perf/optimize-album-sort
open-api-fix
feat/create-job-with-dto
use-toast-primary
feat/vitest-4
feat/ios-fastlane-match
match-signing
fix-update-time-update-timeline
chore/translation-keys
feat/modal-routes
feat/panorama-tiles
feature/mobile-view-asset-owner
feat/system-settings
feature/show-activity-count
better-info-in-asset-viewer
fix/all-people-count
feat/location-favorites
feature/rearrange-buttons-2
fix/download-storage-template
feat/kb-shortcuts-mobile
fix/people-count
push-qolzzzzxrvvn
chore/originals-in-asset-files
feat/asset-size-columns
ben/tree-a11y
new-search-filter-ui
refactor/expectSelectedReadonly
refactor/mobile-grdb
push-qvuktpxmkknu
feat/mobile-native-local-sync
refactor/timeline_ops
fix/scrubber_end
feat/version.txt
feat/context-menus
feat/server-chunked-uploads
refactor/virtualsegment
refactor/rename_daymonth_groups
fix/restrict-android-bg-worker
feat/android-periodic-worker
fix-remote-sync-clean-up
refactor/timeline_move_ops
renovate/mapbox-mapbox-gl-rtl-text-0.x
fix/timeline_split_selectable
feat/keyboard_actions_help_modal
feat/static_frontend
feat/notification-warnign-android
feat/plugins2
feat/plugins
test/create-workflow-token-action
fix/docs-force
debug/search-result-similarity
debug/cf-chunked-uploads
feat/eslint_rule
feat/search-filter-album/web
refactor/timeline_photostream
refactor/timelineasset_asset
feat/session-permissions
feat/timeline_photostream_assetnav
feat/timeline_minor_optimize
feat/timeline_perf_nocomp
feat/timeline_search_results_actions
feat/timeline_search_results_page
fix/timeline_padding
fix/timeline_search_reactivity_warnings
feat/timeline_scrollbar
feat/timeline_stream_withviewer
fix/timeline_back_forth_nav
refactor/timeline_photostream_component
fix/generated-files-checks
fix/locate-button-local
chore/base-image-mimalloc
refactor/timeline_assetlayout
refactor/timeline_selectable
refactor/timeline_aware_actions
refactor/timeline_monthsegment
feat/remove-old-pages
chore/deps-gradle
tmp_photostream
tmp/lcms
feat/mobile-dynamic-thumbnails
fix/mobile-finer-thumbnail-concurrency
refactor/timeline1
refactor/extract_photostream
refactor/rename_load_api
refactor/timeline2
refactor/timeline3
feat/multi-select-asset-viewer
feat-no-thumbhash-cache
refactor/asset_grid
feat/faster-access-checks
fix/18991
fix/19543
chore/temp-remove
fix/21419
feat/mobile-hdr-images
chore/update-mise-lockfile
feat/mise-server-checks
feat/mise-ci
feat/windows-2025
feat/dev_cli
refactor/mobile-migrate-clients
fix/map-theme
fix/require-checkbox
chore/use_swc
feat/efficient-thumbnail-decoding
refactor/mobile-thumbhash
refactor/mobile-thumbhash-new
fix/mobile-uncached-zoom
feat/beta-background-upload
fix/beta-timeline-memories-setting
fix/failed-uploads-not-removed
feat/mobile-shared-album
feat/groups
drift-map-page
drift-auth-user-sync
fix/disable-memory
feat/add-to-album-action
edit-date-time-action
drift-people-page
sqlite-remove-isIn
feat/inline-storage-columns
chore/required-reviewers
refact/asset-manager
fix/folder-sort
pnpm
feat/widget-multiple-server-urls
chore/medium-tests-dbname
fix/web-no-iterator-find
fix/map-pan-interruption
track-livephotos
timeline_events
chore/oxlint-migration
feat/maintenance-worker
feat/dav
chore/demo-snapshot
refactor/server-side-dedupe
feat/integrity-checks
dev/recognition-eval
lighter_buckets_test
perf/postgres-queue
postgres-queue
focus_rings
refactor/web-stores-1
refactor/add-to-taken
feat/sort-places
feat/sidecar-asset-file
vet
tmp/demo-snapshot-preview
fix/server-migration-file-extension
refactor/mobile-v2
fix/asset-update-race-condition
rknn-toolkit-lite2
refactor/mobile-split-up-search-page
feature/Add-rocm-support-for-machine-learning
feat/rocm
chore/async-hash-file
feat/shared-link-view-count
feat/rotation
feat/graphql
feat/job-ids
feat/ignore-library-permission-error
feat/docker-compose-builder
feat/kysely-typeorm
mobile/onboarding
no-video-player
fix/server-qsv-output-format
chore/server-geodata-tweaks
mobile/native-video-player-no-hero
feat/xxhash
fix/docs-concurrency
feat/preload-ml-textual-model
feat/local-tileserver
refactor/exif-orientation
original-path-infix
refactor/mobile/login-form-1
feat/server-editor-endpoints
fix/server-qsv-vbr
fix-mobile-db-problems
feat/ml-armnn-conversion
feat/mobile/backup-with-album-info
feat/fast-initial-sync-1
chore/handle-output_dims
feat/server-more-robust-generation
feat/unassign-faces
feat/shortcuts-on-asset-grid
feat/background-upload
feat/capacitor-mobile-app-poc
feat/server-nvenc-hw-decoding
release/v1.105
fix/mobile-fetch-non-archive
feat/fine-grained-access-controls
web/automation-ui
feat/mobile-server-endpoint-save-dropdown
feat/blurhash-thumbnail
object-storage
feat/memories-animations
dev/metrics
ml/tflite
feat/ml-export-cli
v2.5.3
v2.5.2
v2.5.1
v2.5.0
v2.4.1
v2.4.0
v2.3.1
v2.3.0
v2.2.3
v2.2.2
v2.2.1
v2.2.0
v2.1.0
v2.0.1
v2.0.0
v1.144.1
v1.144.0
v1.143.1
v1.143.0
v1.142.1
v1.142.0
v1.141.1
v1.141.0
v1.140.1
v1.140.0
v1.139.4
v1.139.3
v1.139.2
v1.139.1
v1.139.0
v1.138.1
v1.138.0
v1.137.3
v1.137.2
v1.137.1
v1.137.0
v1.136.0
v1.135.3
v1.135.2
v1.135.1
v1.135.0
v1.134.0
v1.133.1
v1.133.0
v1.132.3
v1.132.2
v1.132.1
v1.132.0
v1.131.3
v1.131.2
v1.131.1
v1.131.0
v1.130.3
v1.130.2
v1.130.1
v1.130.0
v1.129.0
v1.128.0
v1.127.0
v1.126.1
v1.126.0
v1.125.7
v1.125.6
v1.125.5
v1.125.4
v1.125.3
v1.125.2
v1.125.1
v1.125.0
v1.124.2
v1.124.1
v1.124.0
v1.123.0
v1.122.3
v1.122.2
v1.122.1
v1.122.0
v1.121.0
v1.120.2
v1.120.1
v1.120.0
v1.119.1
v1.119.0
v1.118.2
v1.118.1
v1.118.0
v1.117.0
v1.116.2
v1.116.1
v1.116.0
v1.115.0
v1.114.0
v1.113.1
v1.113.0
v1.112.1
v1.112.0
v1.111.0
v1.110.0
v1.109.2
v1.109.1
v1.109.0
v1.108.0
v1.107.2
v1.107.1
v1.107.0
v1.106.4
v1.106.3
v1.106.2
v1.106.1
v1.106.0
v1.105.1
v1.105.0
v1.104.0
v1.103.1
v1.103.0
v1.102.3
v1.102.2
v1.102.1
v1.102.0
v1.101.0
v1.100.0
v1.99.0
v1.98.2
v1.98.1
v1.98.0
v1.97.0
v1.96.0
v1.95.1
v1.95.0
v1.94.1
v1.94.0
v1.93.3
v1.93.2
v1.93.1
v1.93.0
v1.92.1
v1.92.0
v1.91.4
v1.91.3
v1.91.2
v1.91.1
v1.91.0
v1.90.2
v1.90.1
v1.90.0
v1.89.0
v1.88.2
v1.88.1
v1.88.0
v1.87.0
v1.86.0
v1.85.0
v1.84.0
v1.83.0
v1.82.1
v1.82.0
v1.81.1
v1.81.0
v1.80.0
v1.79.1
v1.79.0
v1.78.1
v1.78.0
v1.77.0
v1.76.1
v1.76.0
v1.75.2
v1.75.1
v1.75.0
v1.74.0
v1.73.0
v1.72.2
v1.72.1
v1.72.0
v1.71.0
v1.70.0
v1.69.0
v1.68.0
v1.67.2
v1.67.1
v1.67.0
v1.66.1
v1.66.0
v1.65.0
v1.64.0
v1.63.2
v1.63.1
v1.63.0
v1.62.1
v1.62.0
v1.61.0
v1.60.0
v1.59.1
v1.59.0
v1.58.0
v1.57.1
v1.57.0
v1.56.2
v1.56.1
v1.56.0
v1.55.1
v1.55.0
v1.54.1
v1.54.0
v1.53.0
v1.52.1
v1.52.0
v1.51.2
v1.51.1
v1.51.0
v1.50.1
v1.50.0
v1.49.0
v1.48.1
v1.48.0
v1.47.3
v1.47.2
v1.47.1
v1.47.0
v1.46.1
v1.46.0
v1.45.0
v1.44.0
v1.43.1
v1.43.0
v1.42.0_65-dev
v1.41.1_64-dev
v1.41.0_64-dev
v1.40.1_63-dev
v1.40.0_63-dev
v1.39.0_61-dev
v1.38.2_60-dev
v1.38.1_60-dev
v1.38.0_60-dev
v1.37.0_58-dev
v1.36.2_56-dev
v1.36.1_55-dev
v1.36.0_55-dev
v1.35.0_54-dev
v1.34.0_53-dev
v1.33.1_52-dev
v1.33.0_52-dev
v1.32.1_51-dev
v1.32.0_50-dev
v1.31.1_49-dev
v1.31.0_49-dev
v1.30.2_48-dev
v1.30.0_46-dev
v1.29.6_45-dev
v1.29.6_44-dev
v1.29.5_44-dev
v1.29.4_44-dev
v1.29.3_43-dev
v1.29.2_43-dev
v1.29.1_43-dev
v1.29.0_42-dev
v1.28.4_41-dev
v1.28.4_42-dev
v1.28.3_41-dev
v1.28.2_40-dev
v1.28.1_39-dev
v1.28.0_38-dev
v1.27.0_37-dev
v1.26.0_36-dev
v1.25.0_35-dev
v1.24.0_34-dev
v1.23.0_33-dev
v1.22.0_32-dev
v1.21.1_31-dev
v1.21.0_31-dev
v1.20.3_30-dev
v1.20.2_30-dev
v1.20.1_30-dev
v1.20.0_30-dev
v1.19.1_29-dev
v1.19.0_29-dev
v1.18.0_27-dev
v1.17.0_25-dev
v1.16.0_23-dev
v1.15.1_21-dev
v1.15.0_21-dev
v1.14.0_21-dev
v1.13.0_20-dev
v1.12.0_18-dev
v1.11.0_17-dev
v1.10.0_15-dev
v1.9.1_14-dev
v1.9.0_13-dev
v1.8.0_12-dev
v1.7.0_11-dev
v1.6.0_10-dev
v1.5.1+9-dev
v1.5.0+8-dev
v1.4.0+7-dev
v1.4.0+6-dev
v1.4.0-dev
v1.3.0-dev
v1.3.1-dev
v0.6-dev
v0.5-dev
v0.4-dev
v0.3-dev
v0.2-dev
first-android-release
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: immich-app/immich#1354
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Delete Branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @rcdailey on GitHub (Sep 22, 2023).
Originally assigned to: @alextran1502 on GitHub.
The bug
Using Immich v1.79.1, I am observing the following issues:
After installing using the
docker-compose.ymlprovided in the installation docs, I introduced an external photo directory. Immich reports about 65,000 photos. I suspect the web client is trying to load information for all of these photos which would explain the symptoms. But I don't know for sure.I have NOT uploaded any photos to Immich at this point. All I have done is added existing photos from a different mounted directory.
The OS that Immich Server is running on
Unraid (using docker compose)
Version of Immich Server
v1.79.1
Version of Immich Mobile App
N/A
Platform with the issue
Your docker-compose.yml content
Your .env content
Reproduction steps
Additional information
No response
@jrasm91 commented on GitHub (Sep 23, 2023):
It definitely does not load everything for the main timeline. Can you record a video of what you are seeing? You can DM it to me on discord if needed.
@rcdailey commented on GitHub (Sep 23, 2023):
I can't take a video in this case, there's just too many personal photos of mine that would be visible and I'm not comfortable sharing that. I don't know if it's helpful but I can show you this from the Networking tab of dev tools:
It clearly shows which requests are taking the longest
@alextran1502 commented on GitHub (Sep 23, 2023):
Can you click on that large request and get the request content body? Did you get the same issue on previous version?
My guess is that you have a very large collection in the month of Dec 2022, is that the correct observation?
@rcdailey commented on GitHub (Sep 23, 2023):
The response was so big that the inspector evicted the data. I used Postman to send the request with an API Key and the total size of the JSON data returned is 60MB!
I can't even pretty format it in VS Code without causing it to crash.
How can this not be the whole library of 65k images?
@rcdailey commented on GitHub (Sep 23, 2023):
Based on the little timeline view on the right side, November 2022 appears to be very large. I don't know why the photos are all grouped under that 1 month but I think that's wrong. Maybe the metadata in the image files is wrong.
@alextran1502 commented on GitHub (Sep 23, 2023):
When the page finally loaded, do you see all images are grouped under that one month?
@jrasm91 commented on GitHub (Sep 23, 2023):
There are two time bucket endpoints. One returns after details by month, that is the very large one you are seeing. The other one returns counts by month. Can you try to capture that one? It should be much smaller and have more details about the distribution of photos
@rcdailey commented on GitHub (Sep 23, 2023):
@alextran1502 commented on GitHub (Sep 24, 2023):
I suspect this is related to #4191, which might fix this in the next release. When the next release is out, can you create a new instance and import the photos again?
@rcdailey commented on GitHub (Sep 24, 2023):
I'd be more than happy to try this again. I appreciate everyone's attention to this!
@JosiahBull commented on GitHub (Sep 26, 2023):
From my current understanding #4191 won't fix this issue? What if a user legitimately has several thousand photos on a single day, or even from a single hour?
A common instance I can think of is a user who copies a Gphotos library over without applying exif - all items will be on the same date. There's needs to be some item-count based batching instead of date-based batching to resolve this permanently.
@jrasm91 commented on GitHub (Sep 26, 2023):
Practically, it hasn't happened yet where somebody has so many photos that the segment by month strategy has failed, minus this case. It doesn't sound like the expected results are 50k photos either. For now, I am fine leaving the implementation as is and seeing if there is a legitimate use case for 5k+, 10k+, etc. photos in a single month by a single user.
@jrasm91 commented on GitHub (Sep 26, 2023):
As far as handling the specific situation where an import or similar causes this situation, I think it would be acceptable to detect the situation, truncate the results to some reasonable amount and show the user a warning about the situation. Ideally they can correct the exif and then continue to use the system as normal.
We've not designed it to work with this volume of photos and it would require quite a bit of work to change that. Unless there are some real life use cases for it I think we can leave it as is for now.
@curtwagner1984 commented on GitHub (Sep 26, 2023):
If you're a photographer and you take hundreds/thousands of pictures a day, it's more than feasible that you'll have that number of images per month.
Regardless of the use case, from my understanding, if the user has 5k images that are dated at the same month, the Web App will try to load all of those images? If that's true, then it seems it might be a design issue, regardless of the use case.
Perhaps the loaded images should be per amount of loaded images regardless of the month. And when you scroll down more images are loaded. (from the same month, if that particular month had a lot of images). This way, this issue will never occur, regardless of the amount of images per month.
It's unreasonable for the app to freeze if it encounters a month with a lot of photos. This makes the web app unusable for users who encountered this issue.
I'm also usure if it's feasible to fix the exif on 5k images, the dates of the photos might be unbailable or never recorded.
@jrasm91 commented on GitHub (Sep 26, 2023):
I think it's great that Immich even exists at all. I also think that it is OK, that it doesn't service every group of users out the gate. If you want to call that a "design issue", that's up to you.
The group of users it does service, it does so really well. Unfortunately, the 5k+ / month professional photographers isn't currently in that category. Fortunately, I have yet to hear of one that is running into this limitation in the first place.
Assets are grouped by month in part so that the virtual scrollbar has some semi-accurate spacing and the justified layout can be calculated for the time period. Neither of those features (justified layout or virtual scrollbar) can exist very easily when you shift to a "per amount" strategy.
There is also the possibility to explore loading statistics per day and load groups of photos per day. I don't know if anybody wants to work on this though. Priorities on an open source project are a bit different since people work on what is interesting to them and/or what issues current users are facing.
@rcdailey commented on GitHub (Sep 26, 2023):
I want to clarify that the results are intended, or at least, not something I'm willing or able to fix. Most of these photos were scanned in IIRC, so the date information probably isn't right. In either case, fixing 65k images is out of the question for me.
I agree with all of these points. It's unfortunate that I'm apparently a corner case here and because of that there's no apparent motivation to fix this. I was excited to use Immich but I'll apparently have to continue using Google Photos or find some other alternative that properly handles my situation.
@alextran1502 commented on GitHub (Sep 26, 2023):
I think we have option to group and fetch date bucket by date instead of by month, correct?
@jrasm91 commented on GitHub (Sep 26, 2023):
@rcdailey - we can certainly look into this if it is a blocker for you. I assumed, perhaps incorrectly that you would look into using exiftool or another solution to correct the dates for those images. Exiftool can bulk update metadata following a pattern.
@curtwagner1984 commented on GitHub (Sep 26, 2023):
First and foremost, I'm genuinely appreciative of Immich's existence and the contributions everyone has made to it. I'm in complete agreement that it's okay for a software to not cater to every user group immediately. My mention of a "design issue" wasn't aimed at critiquing this aspect.
The core of my concern is the application's handling of unexpected volumes of input. If Immich is designed to handle, let's say, X images at once, encountering a scenario with more than X images should be managed gracefully. One possible solution could be limiting the image requests to X and providing a user with a message for the surplus (e.g., 5 images not loaded). This threshold, X, could perhaps even be user-configurable based on individual hardware capacities.
Addressing your point about assets being grouped by month for the virtual scrollbar and justified layout: might it be possible to simply deactivate the virtual scrollbar in situations where the image volume of a month necessitates batching? In my perspective, having Immich operate without the virtual scrollbar is far more preferable than having it hang to the point of tab closure.
Regarding loading images by day, I'm doubtful this will resolve the issue, as you pointed out, a large batch of images imported/uploaded without Exif data would be treated as coming from the same day.
To wrap up, I'd like to emphasize that my intention is never to belittle or discredit the effort that's gone into Immich. When highlighting what I see as a design challenge, it's purely from a constructive feedback standpoint and in no way a critique of the project's overall value or the dedication of its contributors.
@JosiahBull commented on GitHub (Sep 26, 2023):
Similar to other comments in this thread I'm extremely excited about Immich existing, and appreciative of the work that's gone into it. It's refreshing to have such a professional FOSS solution to host and share my family photos.
That being said, I think it's possible for non-professional photographers to exceed the limits imposed by the current design, and I think Immich should be able to gracefully handle large photo uploads across a single month.
I took a team trip a couple of years ago and we had a shared album for photos, which accumulated ~1500 photos in only 3 days. I think it's extremely feasible for 'vanilla' users to have a month that crashes the app when they scroll past it if they went traveling with a group at some point.
Notably I had a botched Google Takeout export which failed to provide
jsondata for ~15k images. I'm aware this is an upstream issue with Google, but for me all these images are stacked on a single day, and I don't have recourse to resolve it beyond manually tagging them... :/@jrasm91 commented on GitHub (Sep 26, 2023):
@curtwagner1984 @JosiahBull - I think those are all valid points and the system should definitely handle those situations (better at least). We can look at improving this in the future for sure. Some ideas that have been brought up include:
Thanks you for your feedback!
@lachlan2k commented on GitHub (Sep 26, 2023):
Would it perhaps be viable to chunk the API requests, using some sort of offset-based pagination, for instance with optional
limitandoffsetparameters? Such that the initial request would contain&limit=5000&offset=0followed by&limit=5000&offset=5000, etc. until all the data has loaded.@dodg3r commented on GitHub (Oct 25, 2023):
Hi
This error is consistent with the problem I'm having.
The difference is that my main timelineview freezes completely and I can't move forward.
If I quickly click on "people" or "albums" or "explore" after logging in, I can scroll further, but as soon as I go back to the timeline, it freezes again.
I have 140,000 pictures and 11,000 of them are from the year 2023-10. Could that be the problem?
Can I edit the files directly in the library folder with EXIF?
@curtwagner1984 commented on GitHub (Oct 25, 2023):
Yeah, this is definitely the same issue. For me the browser tab completely
freezes and had to be killed. I have to navigate to the explore it people
tab manually.
On Wed, Oct 25, 2023, 13:56 Jerry Johansson @.***>
wrote:
@jrasm91 commented on GitHub (Oct 26, 2023):
I actually did some more research into this and I don't think it's actually a problem with loading that many assets at once, or sending that much data across the wire, it comes down to how granular of groups we use to render the assets in the web. We use an intersection observer, but it looks like it works at the per day level, so it will try to render all the assets for a given day as you are scrolling.
@curtwagner1984 commented on GitHub (Oct 26, 2023):
It would have been really, really, really, great if the user could override
this to a specific count. Like, render at most 20 images as I'm scrolling.
On Thu, Oct 26, 2023 at 12:00 AM Jason Rasmussen @.***>
wrote:
@alextran1502 commented on GitHub (Oct 26, 2023):
@curtwagner1984 it will be an optimization for this mechanism. We will find sometimes to work on this
@curtwagner1984 commented on GitHub (Oct 27, 2023):
Thank you for considering my request for this optimization. 🙏
On Thu, Oct 26, 2023, 18:43 Alex @.***> wrote:
@peca89 commented on GitHub (Nov 3, 2023):
I'm also very much excited about Immich being actively developped. Since local library feature is introduced, I'm in the process of evaluating Immich's performance compared to Plex in order for it to become main picture archive viewer. I find myself quite bothered about existance of this bug/feature that I need to point out why this would be a dealbreaker for me unless it's fixed.
My personal library contains a folder with a bunch of scanned films where all files share the same date. Because for the most of films even the approximate date of capturing is unknown, I decided to batch set file and exif date for all these files to a random date in 1990. Anything else, like randomizing the date, would sort the pictures completely incorrectly. It's about 10000 photos (more than 200 films were scanned) that would show as if they were the same date.
@jrasm91 commented on GitHub (Nov 3, 2023):
That doesn't really make sense IMO. Having specific settings like "how many images to load at once" it just a symptom of a problem not a valid solution, especially if the implementation is non-trivial. Just fix the original issue at that point.
@curtwagner1984 commented on GitHub (Dec 20, 2023):
Yeah, you're right. Resolving the underlying issue would defiantly be preferable. However, if it's too time consuming, at the very least, a temporary patch that would allow setting the max images to fetch and render would enable the app to run instead of crushing the browser tab.
@mertalev commented on GitHub (Feb 3, 2024):
Building on some earlier suggestions, we could truncate the number of assets per bucket to 1000 and allow buckets to be interacted with (e.g. by making the date clickable) to change to a paginated gallery viewer without the timeline on the side.
Possible UX refinements include having the interaction at the end of a bucket instead, only displaying this button if the assets were truncated, and possibly skipping to the second page when fetching in the gallery viewer (the first page already having been viewed).
@Handrail9 commented on GitHub (Feb 28, 2024):
Hello, I am popping into this issue to say this is the exact issue I'm facing at the moment, a lot of my photos carried over from a GPhotos archive from a deleted G account from roughly 2021 are all showing as being from June '23, roughly 43k images actually. I can get as far as selecting all of them after about 10 minutes of waiting, however I can't move them to an album, archive them, or delete them. On the Android app only 2,009 of them show up, which I tried to archive to get more to show up, and none did, so roughly 41k just do not show up on the Android app. I'm hoping this can be an issue that's fixed rather than left for people in this scenario to "find a better solution" because Immich is a beautiful and amazing looking app from a UI/UX perspective and I don't want to go back to using lackluster solutions.
@alextran1502 commented on GitHub (Feb 28, 2024):
@Wave6677 We have yet to solve this issue since it currently affects a small portion of the user base. However, this is something that we are planning to fix eventually.
Right now, we are fetching the assets in a bucket grouped by month. We can modify this to give it a better logic to group it by date when the monthly asset is too many.
We have a solution, just not have the resource to get around to implementing this yet
@Handrail9 commented on GitHub (Feb 28, 2024):
@alextran1502
This might sound like a silly question, as I am in no way an experienced developer, but would that work for a large amount of images coming from a zip/tar archive without exif data? For instance, how I have 43k photos all dated "June 1st 2023"
@alextran1502 commented on GitHub (Feb 28, 2024):
@Wave6677 It means the photos don't have proper EXIF information. I could have been removed when you ingest/extract from a different software.
@mertalev commented on GitHub (Feb 28, 2024):
I agree we should handle assets in the same month or even day gracefully. Missing EXIF data should ideally just mean the date comes up wrong on the timeline. It's unexpected for it to make the web app crawl.
@freekk commented on GitHub (Apr 2, 2024):
Hello, just a quick note to say that this is indeed a real issue for many people around me.
It's as simple as gathering photos from friends on a special event like a wedding. I easily end up with thousands of photos on the same hour!
It happens all the time in my timeline, rendering immich totally unusable for now.
I really hope this will be fixed one day as I really like your app and you did a wonderful job on all other aspects!
@rezzorix commented on GitHub (Apr 7, 2024):
Just got v1.101.0 up and running yesterday.
I added a library of ~50.000 photos which immich seems not be able to handle with its gui.
Especially the web interface (tested in different browsers FF, Chromium based, Safari) has major performance problems.
I have setup immich on a VM with 8 cores and 16 GB RAM.
When opening the web interface I am simultaneously checking usage of the VM and my local machine (CPU, RAM, Disk, Network).
All looks OK, resources are sufficiently available, nothing seems to hit the limit.
However, immich performance is very poor, to the point of browser being unusable despite having enough resources.
How could this be resolved?
@mertalev commented on GitHub (Apr 7, 2024):
Do all these images appear as being on the same month? If so, the current solution would be to fix their metadata with exiftool and run metadata extraction on all assets.
@curtwagner1984 commented on GitHub (Apr 21, 2024):
As previously discussed, this isn't always feasible in various scenarios.
For instance: There might be nothing wrong with the metadata, and the user maybe be just taking/creating a lot of images in a given month. Also, the metadata might not be available.
This issue renders immich completely unusable for people who encounter this issue. The only workaround I see on the user side is to try to artificially split the images from the same month into groups of different months by manufactuing a fictitious metadata.
Can't this issue be resloved with virtualization? What I mean is, only fetch the count of images for each month and calculate the side bar (where the months appear) based on the count. And then only load a subset of that count in the viewport.
At the very bare minimum, just have a maximum number of images that can be loaded in a single month so instead of crushing the UI if you have too many images in a certain month, it will show the first X of that amount. In the very least the webui would still function with this option.
@mertalev commented on GitHub (Apr 21, 2024):
Yes, that suggestion is just for the current state of things. We plan to make improvements in this area. It can dynamically split buckets into days if there are too many assets in a month and truncate the number shown for a day if there are still too many to display.
@curtwagner1984 commented on GitHub (Apr 21, 2024):
I'm really happy to hear that, because spliting into days won't solve the issue for images that just don't have metadata, they would be counted as being form the same month, same day, same hour, same second.
Does this problem exist also with search results? As in, if your query returns too many images, then the UI crashes?
@mertalev commented on GitHub (Apr 21, 2024):
No, search doesn't use the timeline so it's unaffected. It's paginated, so it'll load 100, then scrolling further will load another 100, and so on.
@curtwagner1984 commented on GitHub (May 3, 2024):
I fail to see what this has to do with the issue currently discussed.
@sutusa commented on GitHub (May 8, 2024):
I don't mean to dog pile on this issue (sorry), but I'm seeing considerable slowness as well with my library. Specifically with the Albums page than the Timeline... but I feel my situation is mostly applicable here.
v.1.103.1via docker-compose28,665photos1,528albumsWhen I take pictures, the albums I create are per that day. So, in a given year I could have up to 365 albums. I seem to average about ~200 albums each year. I don't take pictures every day, with maybe 15-30 photos with my Fuji camera as an average for the day. Maybe a lot more for special events like a kid's birthday or something. So, I don't think my use-case is out of the ordinary or unreasonable.
Not sure if my library counts as a "large library", but from my perspective, the purpose of any image database is to potentially store a lifetime of photos for easy reference. I would argue that large libraries shouldn't take a back seat and/or be 2nd-class citizens.
With that said, when navigating to the Albums page, there is a long 3-5sec pause/hang as the page downloads 4Mb JSON responses from the API (
GETrequests against/api/album):(firefox calls)

(firefox headers)

Not surprisingly, I have seen the same wait times when doing manual curl requests against the same
/api/albumpath when writing some quality-of-life scripts for myself.Looking at the request's response ... it seems, too verbose?
(curl response, pipe'd through jq for prettifying)

That is all the data that is returned per album (and I have ~1500 albums which will grow over time). Seems like a lot of data. Especially when the page's goal seems to be only leveraging each album's Title, Thumbnail, Date, Asset count, and User Permissions.
Not sure what a good approach might be to resolve. My first thought goes towards something like using GraphQL to help reduce unwanted information in the response, as only what is requested is what is returned. That might be a heavy lift though. Maybe custom/partial REST responses? Pagination? Dunno.
I have experience in architecture, devops and operational things... and I code for fun occasionally... but I do NOT view myself as a professional web developer or anything. So, feel free to correct me if I'm wrong here on any of this! I just wanted to give my perspective based on what I have seen.
Also, thank you for your time and effort on such a wonderful project! I dropped Flickr like a hot potato when I found your project earlier this year! I love it!
@alextran1502 commented on GitHub (May 8, 2024):
@sutusa No problem. The album page hasn't been tuned for that many albums' use cases yet, so it doesn't have lazy loading or chunk fetching. It explains the issue that you are seeing.
@curtwagner1984 commented on GitHub (Jun 24, 2024):
Hi, I just wanted to inquire about whether or not there is any progress on this issue?
I looked at the release notes of the last few releases and didn't find it mentioned. Also, the issue remains open. So it doesn't bode well.
I'm sure this isn't that high up on the to-do list, but I just want to reiterate that this is a significant issue where the application just freezes and crushes if it receives unexpected input. Fixing this to the point where the application wouldn't crush would be also appreciated if implementing the wanted behavior is too time-consuming.
Maybe this doesn't affect a lot of people, but for those affected the timeline is unusable. (I understand that the albums and people tab is unaffected by this, but maybe this is wrong as the people table also has a timeline, if that's the case it's unusable too if you have too many images of a specific person from a specific time)
@alextran1502 commented on GitHub (Jun 24, 2024):
@curtwagner1984 I started some work on this #9935, but that approach wasn't the best, so I will implement it a different way. However, this issue will be resolved before the stable release
@curtwagner1984 commented on GitHub (Jun 24, 2024):
Hey, thank you so much for replying so quickly and for all the hard work you're putting into this!
I saw the PR you linked, and I agree with you that this might not be the best approach, loading by intervals of days instead of months won't solve the underlying issue because a large batch of images can still come in a smaller time frame. , I believe the core issue is that the application can't handle a large specific input of assets at one time.
In my opinion, decoupling the amount of images loaded from the time span of when the image was taken is crucial for the resolution of this issue.
Not sure if this is technically feasible, but one idea is to implement virtualization for the timeline, arranging images in buckets based on image count rather than time periods. Each bucket would have start and end times to ensure correct placement in the timeline.
I have limited knowledge about how the current implementation works, but from what I gather, all the images in the library a divided to monthly buckets based on the time the image was taken, and then the timeline loads those buckets from the most recent to the last. The arrangement to months helps the sidescroller be in the correct position.
I think maybe this still be the case if images are loaded in buckets of X max images per bucket instead of time period and still arranged from the latest to the oldest, when you load a month, you can keep track of which bucket you're currently are for that month.
By loading a maximum of one bucket of X images at a time, even if there are 100*X images from the same date, only one bucket would be loaded at once. The start and end times would ensure the buckets are correctly ordered in the timeline.
I understand this might be something you've already considered and found to be technically challenging. I'm just sharing my thoughts in the hope they might be useful.
Thank you again for your dedication to resolving this issue. Your work is highly appreciated!
EDIT: Loading buckets of X images instead of dates will also improve overall performance because you still will have months with more images than others, even if this amount won't crush the UI, it will slow it down and the UX will be worse If you load 100 images at the time, the scrolling experience would be smooth as butter, no matter the months
@alextran1502 commented on GitHub (Jun 24, 2024):
@curtwagner1984 No worries. It is good to check in often to keep us informed as well. The timeline is already lazily loaded based on the bucket, my plan is to return the asset count of each bucket, and depending on the number of the bucket, it will use the corresponding grouping strategy i.e. day vs month
@curtwagner1984 commented on GitHub (Jun 25, 2024):
But aren't the current buckets based on months? As in, it loads one month at a time?
I unsure I understand though. Let's say the app crushes if you load more than 1000 asssts at once. And the scenario is that the user has 2000 images timestamped with the same date. So first you'll try the month buckets, and you see the asset count is 2000, the. You'll try the daily buckets, and you'll still get a count of 2000. So in this case increasing the resolution form months to days doesn't resolve the issue.
On the other hand, if you say that each unit of lazy load can be max of 500 assets, then you'll never run into the issue, regardless of their date of creation.
Unless Im missing something or don't understand something.
@jrasm91 commented on GitHub (Jun 25, 2024):
I think that is correct. How would something like that work with the virtual scrollbar? Can you still jump to a specific point in time? It is going to be a different implementation to just use pagination, which the timeline buckets don't even support right now. I think the point is moving to per day is easy enough with the current implementation vs moving to pagination is not. So this is an easy win and we can tackle an alternative implementation later
@curtwagner1984 commented on GitHub (Jun 26, 2024):
Depends what you mean by "works". Currently, the virtual scrollbar is time-dependent. So let's take an edge case scenario, where all your assets are timestamped with the same date and time. The virtual scrollbar will show that time, and and if there 's a tie in the timestamp (like it is for all images in this case) then the app would just display them in the order they were added to immich database. So in this case you won't be able to jump to a time, because there is only one date and time.
In case, only a portion of your images are timestamped at the same time, I don't see why you wouldn't be able to jump to a certain date. It's just that the date at which there are many images will be 'fatter' relative to the others.
I think pagination will increase overall stability and smoothness of the app, regardless of this issue. Because all new batches of images that would be loaded will be the same size. (maybe even user-configurable based on their hardware.) So you have control over the amount of images loaded each time. Yet when you are based on dates, you just hope that the user's images are distributed far enough in dates that it won't slow down or crush the webui.
@jrasm91 commented on GitHub (Jun 27, 2024):
If you are talking about pagination within date buckets that might work. If you are saying replace time buckets with pagination that is basically a rewrite of the rendering process and likely to be more involved to implement.
@curtwagner1984 commented on GitHub (Jul 3, 2024):
This is exactly what I'm talking about, And IMHO this is a good idea regardless of this issue because you want the amount of images loaded by each request to be deterministic.
Also, this solution effectively be a replacement of buckets with pagination if all a user has is a single bucket.
@The-Real-Thisas commented on GitHub (Aug 13, 2024):
@curtwagner1984 does the current scrollbar not paginate already ?
I'd assume the logical setup for this is only loading a set amount of images, say 30-40, and then loading more on scroll that way it feels relatively seamless with fast load times kinda like how tiktok scroll works. I think this can be pretty easily added on the web client with a scroll trigger.
In this situation since we don't have the timestamp for when the image was taken we can paginate based on the timestamp added to immich which I'm assuming we have, if not when maybe we can do some basic sort on the bucket and return based on that with like something arbitrary but repeatable, this will have to be properly considered.
Should be pretty easy to modify the backend to add the pagination behaviour without fully removing the bucket system which might be a pain and instead just have another endpoint with the pagination which can take in a parameter like, 30-40 which will filter the bucket and return that range.
Also, I'm new to this project but have worked with sveltekit for a while and have experience with the tech stack so I'm willing to contribute.