[PR #11881] fix(ml): tokenization for webli models #13398

New Issue

OVERLORD · 2026-02-05T15:14:01+03:00

OVERLORD commented

2026-02-05 15:14:01 +03:00

Original Pull Request: https://github.com/immich-app/immich/pull/11881

State: closed
Merged: Yes

Description

The new round of models assume the text is canonicalized. This PR checks the model config file to see if this needs to be done for a model. If so, it removes punctuation and converts the text to lower case.

Tested that search results are relevant regardless of casing.

Fixes #11865

**Original Pull Request:** https://github.com/immich-app/immich/pull/11881 **State:** closed **Merged:** Yes --- ### Description The new round of models assume the text is canonicalized. This PR checks the model config file to see if this needs to be done for a model. If so, it removes punctuation and converts the text to lower case. Tested that search results are relevant regardless of casing. Fixes #11865

OVERLORD added the pull-request label 2026-02-05 15:14:01 +03:00

OVERLORD closed this issue

2026-02-05 15:14:02 +03:00

Sign in to join this conversation.

Branches Tags

main

chore/translations

fix/partner-ownership-contention

renovate/machine-learning

renovate/pypi-pillow-vulnerability

renovate/pypi-pydantic-settings-vulnerability

renovate/java-25.x

renovate/mobile

chore/harden-mobile-openapi-gen

renovate/java-21.x

fix/min-faces

refactor/asset-update

feat/merge-people-mobile

chore/face-cluster-abstraction

refactor/search-v3/pr-2a

fcast

fix/android-local-album-date

fix/24545-sync-retry-cx

fix/android-locked-folder-delete

make-sure-memory-widget-render

chore/freezed-example

fix/clamp-local-date-time-to-valid-value

fix/datetime-max-limit

fix/viewer-long-image-aspect

feat/native-core

feat/yucca-integration

refactor/mobile-actions

fix/ws-reconnect-battery

renovate/sharp

renovate/pigeon-27.x

refactor/auto-trash-sync

feat/release-source-branch

refactor/session-repository

fix/library-map-live-bounds-updates

renovate/permission_handler-12.x

renovate/node-24.x

fix/mobile-scroll-velocity-placeholders

fix/library-map-loading-flash

agg23/appbar-experiments

renovate/pep621-lock-file-maintenance-machine-learning

refactor/undo-actions

refactor/actions

feat/restore-action

feat/delete-action

fix/29853-thumbnail-decode-size

bugfix/live-photo-stuck

feature/load-previews

chore/pub-upgrade

custom-app-logo

fix/failed-sync-blocks-resume

feat/ml-hwaccel-testing

feat/workflow-logs

feat/server-chunked-uploads

feat/upload-action

fix-29727

feat/download-tag-action

feat/share-action

feat/album-actions

fix/android-cr3-save

feat/browser-profile-action

feat/cast-slideshow-action

feat/locked-folder-action

fix/unauthorized-memory-creation

feat/context-actions

refactor/action-class

feat/memories-view

refactor/asset-action-filter

feat/archive-action

feat/stack-action

feat/mobile-hls-player

refactor/stack-actions

feat/workflow-asset-tags

fix/timeline-drag-select-stops

fix/featured-photo-thumbnail-cache

fix/album-timeline-date-ordering

chore/album-remove-assets

feat/partner-table-permissions

fix/ocr-after-edit

fix/re-enable-pruning

refactor/delete-restore-actions

apk-build-deployment

edit-photo-stacking

fix/mise-lockfile

feat/web-hls-hover

refactor/store-cleanup

fix/hash-mismatch

refactor/app-metadata

refactor/advanced-settings

refactor/current-user

fix/dedupe-integrity-missing-paths

chore/albums-select

fix/fdroid-meta

refactor/per-asset-backup-flag

fix/av1-threads

fix/pass-secrets

fix/download-progress-stuck

fix/negative-backup-remainder

feat/map-asset-number

test-bump-prerelease

fix/handle-unlinking-motion-photos

fix/date-range-formatting

fix/mobile-album-action-msg

asset-add-to-album-trigger

refactor/face-editor-decoupling

feat/zoom-aware-face-editing

refactor/adaptive-image-dimensions

fix-face-editor-video-preview

fix-stack-face-tag-selection

fix/face-boxes-edit-panel

workflow-webhook-step

feat/hero_view_transitions

fix/map-sidepanel-queries

feat/dart-openapi-generator

claude/share-quality-5d7Gn

drag-drop-ghost-target

refactor/workflow-page

refactor/mobile-upload

fix-scroll-flicker-2

update-mise-lock-file

feat/kysely-0.29

release-candidate-flow-1

fix/mobile-22522-sqlite-2067

fix/job-lock

release-candidate-flow-3

release-candidate-flow-2

show-in-timeline-toggle

fix/mise-windows

plan-local-image-display

fix/remote-thumbnail-fallback

chore/riverpod-update

album-order-per-user

fix/web-timeline-dom-node-retention

chore/svelte-enable-state-referenced-locally

feat/favorite-albums

feat/patch-release-from-branch

fix/oauth-linking

feat/timeline_scroll_buffer

panorama-face-overlay

push-txxyuusptoru

push-zpwsovysllvn

chore/queue-endpoint-migration

fix/remove-from-album

chore/thumbnail-histogram

fix/video-thumbnail-bindthis-regression

fix/scrubber-drag-edge-cases

fix/timeline_layout_row_height

refactor-cancellable-task

uhthomas/fix-mobile-use-timer

uhthomas/fix-mobile-video-controls-key

uhthomas/chore-mobile-search-declarative

owner-role-album-user

feat/livephoto-editing

favorite-albums

uhthomas/fix-mobile-isolate-auth

fix/library-watcher-excludes

chore/library-e2e-to-medium

push-ynllunsvrpxy

fix/dont-delete-offline-library-files

push-zvwwxmvkrspv

push-kywsnvvkwzts

push-xxnxokokzrou

push-voozlrlkxsnu

view-in-timeline-transition

push-uymvmlknxpmx

push-qslzsplnusoz

fix/mobile-video-stall

uhthomas/draft-mobile-image-cache-cancel

push-yquostpqnkxp

feat/vitest-4

claude/offload-icloud-hashing-bI3GZ

midzelis/wip

uhthomas/fix-server-fsync-all

feat/mobile-widget-shared-client

push-lnyuzsutzkqq

push-vqqqxxvkunru

push-wvnmqoswptww

push-nwxlpmyzkyrl

fix/map-webgl-error

fix/mobile-player-fit

feat/panorama-tiles

csp-policy

push-rsywxvptwxuv

refactor/restores-file-interceptor

postgres-socketio

claude/auto-screenshot-web-changes-Y7efI

visual-review/pr-26535

push-lvyturrtwkrq

feat/library-offline-count

fix/mobile-video-aspect-ratio

uhthomas/fix-mobile-search-results

uhthomas/feat-sort-smart-search

uhthomas/chore-mobile-maplibre

uhthomas/mobile-fix-asset-details-album-pop

feat/crawl-wrapper

push-skvzqoozqkpl

feat/edit-filters

feat/pg-queue

refactor/asset-upload

better-project-structure

uhthomas/mobile-feat-asset-viewer-details

fix/ml-rocm-build

feat/asset-file-apis

feature/bottom-buttons-order

sqlite_thumbs

fix-keep-correct-ios-shared-album-asset

push-vpxwmwwxwnvw

fix-migration-width-height

shared-deep-link-handler

feat/thumbnail-native-clients

feat/platform-clients

fix/foreground-cloud-sync

filter-by-person

refactor/sidebar

fix/merged-edited-assets

feat/create-job-with-dto

feat/ios-fastlane-match

match-signing

fix-update-time-update-timeline

feat/modal-routes

feature/mobile-view-asset-owner

feat/system-settings

feature/show-activity-count

feat/location-favorites

feature/rearrange-buttons-2

fix/download-storage-template

chore/originals-in-asset-files

ben/tree-a11y

new-search-filter-ui

refactor/expectSelectedReadonly

refactor/mobile-grdb

feat/mobile-native-local-sync

refactor/timeline_ops

fix/scrubber_end

refactor/virtualsegment

refactor/rename_daymonth_groups

fix-remote-sync-clean-up

debug/cf-chunked-uploads

feat/search-filter-album/web

feat/session-permissions

feat/mobile-dynamic-thumbnails

refactor/extract_photostream

refactor/rename_load_api

refactor/timeline2

refactor/timeline3

feat-no-thumbhash-cache

feat/mobile-hdr-images

feat/beta-background-upload

fix/beta-timeline-memories-setting

fix/failed-uploads-not-removed

feat/groups

drift-map-page

feat/add-to-album-action

track-livephotos

feat/maintenance-worker

refactor/server-side-dedupe

feat/integrity-checks

dev/recognition-eval

lighter_buckets_test

perf/postgres-queue

tmp/demo-snapshot-preview

fix/server-migration-file-extension

rknn-toolkit-lite2

feature/Add-rocm-support-for-machine-learning

chore/async-hash-file

feat/shared-link-view-count

feat/graphql

no-video-player

fix/server-qsv-output-format

chore/server-geodata-tweaks

mobile/native-video-player-no-hero

feat/local-tileserver

feat/ml-armnn-conversion

chore/handle-output_dims

feat/capacitor-mobile-app-poc

feat/server-nvenc-hw-decoding

web/automation-ui

object-storage

ml/tflite

feat/ml-export-cli

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: immich-app/immich#13398