immich-app/immich
1
0
Fork 0
mirror of https://github.com/immich-app/immich.git synced 2026-07-25 14:00:45 +03:00
Code Issues 783 Packages Projects Releases 22 Wiki Activity

[PR #6785] [MERGED] feat(server)!: pgvecto.rs 0.2 and pgvector compatibility #11352

New Issue
Closed
opened 2026-02-05 14:37:57 +03:00 by OVERLORD · 0 comments
OVERLORD commented 2026-02-05 14:37:57 +03:00
Owner
Copy Link

📋 Pull Request Information

Original PR: https://github.com/immich-app/immich/pull/6785
Author: @mertalev
Created: 1/31/2024
Status: ✅ Merged
Merged: 2/7/2024
Merged by: @mertalev

Base: main ← Head: chore/server-pgvecto-rs-0.2

📝 Commits (10+)

📊 Changes

25 files changed (+640 additions, -236 deletions)

View changed files

📝 .github/workflows/test.yml (+1 -1)
📝 cli/test/e2e/setup.ts (+1 -1)
📝 docker/docker-compose.dev.yml (+1 -1)
📝 docker/docker-compose.prod.yml (+3 -2)
📝 docker/docker-compose.yml (+1 -1)
📝 server/e2e/api/setup.ts (+1 -1)
📝 server/e2e/docker-compose.server-e2e.yml (+1 -1)
📝 server/e2e/jobs/setup.ts (+1 -1)
📝 server/src/domain/database/database.service.spec.ts (+151 -77)
📝 server/src/domain/database/database.service.ts (+124 -47)
📝 server/src/domain/domain.config.ts (+1 -0)
📝 server/src/domain/domain.constant.spec.ts (+27 -35)
📝 server/src/domain/domain.constant.ts (+24 -12)
📝 server/src/domain/repositories/database.repository.ts (+26 -0)
📝 server/src/domain/repositories/smart-info.repository.ts (+1 -1)
📝 server/src/infra/database.config.ts (+4 -0)
📝 server/src/infra/logger.ts (+1 -1)
📝 server/src/infra/migrations/1700713871511-UsePgVectors.ts (+3 -1)
📝 server/src/infra/migrations/1700713994428-AddCLIPEmbeddingIndex.ts (+9 -5)
📝 server/src/infra/migrations/1700714033632-AddFaceEmbeddingIndex.ts (+9 -5)

...and 5 more files

📄 Description

Description

This PR upgrades the codebase to be compatible with pgvecto.rs 0.2 and future patch releases. This change is beneficial in several respects:

  • It allows for searching with VBASE, a novel vector search algorithm that allows fetching an arbitrary number of rows instead of a fixed number upfront
  • There have been many bug fixes as well as stability and performance improvements since the 0.1.11 release
  • It makes adding and maintaining compatibility with pgvector easier
  • Updating is simpler for admins as they can use any 0.2.* release, not a specific patch release
  • Maintaining Immich is easier for third-party platforms as they have more freedom in the pgvecto.rs version they use
  • It fixes a kernel issue with some Synology servers

Additionally, it now allows pgvector to be used instead. This is added for compatibility reasons - there are environments where pgvecto.rs simply cannot be used right now.

Summary of changes:

  • Check for available extension upgrades at startup and update if possible
  • Generalize vector extension code to work with both pgvecto.rs and pgvector
    • Use pgvecto.rs's pgvector compatibility mode for more code reusability between the two
  • Add an env that determines which extension is used
    • Migrations are applied based on this env, so this choice cannot be changed after a successful startup
  • Make parts of database startup run in a lock to prevent both server and microservices from making changes at the same time
  • Add vectors to the database search path
    • pgvecto.rs now uses this schema to be able to coexist with pgvector
  • Update version constraints to allow pinning to the patch/minor/major part of the minimum version

To do:

  • Add more unit tests

Follow-up work:

  • Refactor the vector DB code - there's some duplication and the logic is spread apart across the database and smart info repos as well as migrations
    • It might be good to move the existing dim size update and migration code all to the database repo
  • Update e2e to run with both extensions to make sure we don't break pgvector
  • Prepare documentation
    • The instructions in the 1.91.0 release notes will be outdated - should we consider making a page for this in the docs?
  • Do more manual testing to discover possible edge-cases

Fixes #5813
Fixes #5880
Fixes #5893
Addresses discussion #5954

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/immich-app/immich/pull/6785 **Author:** [@mertalev](https://github.com/mertalev) **Created:** 1/31/2024 **Status:** ✅ Merged **Merged:** 2/7/2024 **Merged by:** [@mertalev](https://github.com/mertalev) **Base:** `main` ← **Head:** `chore/server-pgvecto-rs-0.2` --- ### 📝 Commits (10+) - [`da4434e`](https://github.com/immich-app/immich/commit/da4434e7362bbd5b6a5c0bbd01f6ac8599791c6b) basic changes - [`fe9a36b`](https://github.com/immich-app/immich/commit/fe9a36bd930152a9c4b6496cd8734485fa11f829) pgvector compatibility - [`718f096`](https://github.com/immich-app/immich/commit/718f0962b80c2dafe01ed5f51d385ca0879ebdfd) update extension at startup - [`006dc5c`](https://github.com/immich-app/immich/commit/006dc5cd636c6c4169799893014cffe55ee9b059) wording - [`a6d3049`](https://github.com/immich-app/immich/commit/a6d30498228c79089940479381db1589be61d7fb) nightly docker - [`9209315`](https://github.com/immich-app/immich/commit/920931586d1964f34904a0bce869239a13c2b9bf) fixed version checks - [`176c637`](https://github.com/immich-app/immich/commit/176c6377aec71678dff24fc11ac7d8bdadf33093) update tests - [`83e2878`](https://github.com/immich-app/immich/commit/83e2878faa8373d8100eb524a467a64d4537c445) simplify `getRuntimeConfig` - [`d0a6665`](https://github.com/immich-app/immich/commit/d0a6665cdd66b79160fe1a41507bffd53b52fa69) wording - [`266a952`](https://github.com/immich-app/immich/commit/266a95284804eefe5b485b411ffbd9c4f42c3fdb) reindex on minor version update ### 📊 Changes **25 files changed** (+640 additions, -236 deletions) <details> <summary>View changed files</summary> 📝 `.github/workflows/test.yml` (+1 -1) 📝 `cli/test/e2e/setup.ts` (+1 -1) 📝 `docker/docker-compose.dev.yml` (+1 -1) 📝 `docker/docker-compose.prod.yml` (+3 -2) 📝 `docker/docker-compose.yml` (+1 -1) 📝 `server/e2e/api/setup.ts` (+1 -1) 📝 `server/e2e/docker-compose.server-e2e.yml` (+1 -1) 📝 `server/e2e/jobs/setup.ts` (+1 -1) 📝 `server/src/domain/database/database.service.spec.ts` (+151 -77) 📝 `server/src/domain/database/database.service.ts` (+124 -47) 📝 `server/src/domain/domain.config.ts` (+1 -0) 📝 `server/src/domain/domain.constant.spec.ts` (+27 -35) 📝 `server/src/domain/domain.constant.ts` (+24 -12) 📝 `server/src/domain/repositories/database.repository.ts` (+26 -0) 📝 `server/src/domain/repositories/smart-info.repository.ts` (+1 -1) 📝 `server/src/infra/database.config.ts` (+4 -0) 📝 `server/src/infra/logger.ts` (+1 -1) 📝 `server/src/infra/migrations/1700713871511-UsePgVectors.ts` (+3 -1) 📝 `server/src/infra/migrations/1700713994428-AddCLIPEmbeddingIndex.ts` (+9 -5) 📝 `server/src/infra/migrations/1700714033632-AddFaceEmbeddingIndex.ts` (+9 -5) _...and 5 more files_ </details> ### 📄 Description ## Description This PR upgrades the codebase to be compatible with pgvecto.rs 0.2 and future patch releases. This change is beneficial in several respects: * It allows for searching with VBASE, a novel vector search algorithm that allows fetching an arbitrary number of rows instead of a fixed number upfront * There have been many bug fixes as well as stability and performance improvements since the 0.1.11 release * It makes adding and maintaining compatibility with pgvector easier * Updating is simpler for admins as they can use any 0.2.* release, not a specific patch release * Maintaining Immich is easier for third-party platforms as they have more freedom in the pgvecto.rs version they use * It fixes a kernel issue with some Synology servers Additionally, it now allows pgvector to be used instead. This is added for compatibility reasons - there are environments where pgvecto.rs simply cannot be used right now. Summary of changes: * Check for available extension upgrades at startup and update if possible * Generalize vector extension code to work with both pgvecto.rs and pgvector * Use pgvecto.rs's pgvector compatibility mode for more code reusability between the two * Add an env that determines which extension is used * Migrations are applied based on this env, so this choice cannot be changed after a successful startup * Make parts of database startup run in a lock to prevent both server and microservices from making changes at the same time * Add `vectors` to the database search path * pgvecto.rs now uses this schema to be able to coexist with pgvector * Update version constraints to allow pinning to the patch/minor/major part of the minimum version To do: * [x] Add more unit tests Follow-up work: * [ ] Refactor the vector DB code - there's some duplication and the logic is spread apart across the database and smart info repos as well as migrations * It might be good to move the existing dim size update and migration code all to the database repo * [ ] Update e2e to run with both extensions to make sure we don't break pgvector * [x] Prepare documentation * The instructions in the 1.91.0 release notes will be outdated - should we consider making a page for this in the docs? * [x] Do more manual testing to discover possible edge-cases Fixes #5813 Fixes #5880 Fixes #5893 Addresses discussion #5954 --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
OVERLORD added the pull-request label 2026-02-05 14:37:57 +03:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
accessibility
changelog:enhancement
changelog:security
changelog:skip
changelog:translation
cli
date-time
dependencies
documentation
external-library
format
good first issue
mobile-beta
mobile-beta
mobile-beta
needs-answer
nice to have
pull-request
Mirrored from GitHub Pull Request
 sharing
tech-debt
📱mobile
🖥️web
🗄️server
🧠machine-learning
No Label pull-request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
OVERLORD
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: immich-app/immich#11352
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?