Commit Graph

28 Commits

Author SHA1 Message Date
Mert
e334443919 feat(server, web): smart search filtering and pagination (#6525)
* initial pagination impl

* use limit + offset instead of take + skip

* wip web pagination

* working infinite scroll

* update api

* formatting

* fix rebase

* search refactor

* re-add runtime config for vector search

* fix rebase

* fixes

* useless omitBy

* unnecessary handling

* add sql decorator for `searchAssets`

* fixed search builder

* fixed sql

* remove mock method

* linting

* fixed pagination

* fixed unit tests

* formatting

* fix e2e tests

* re-flatten search builder

* refactor endpoints

* clean up dto

* refinements

* don't break everything just yet

* update openapi spec & sql

* update api

* linting

* update sql

* fixes

* optimize web code

* fix typing

* add page limit

* make limit based on asset count

* increase limit

* simpler import
2024-02-12 20:50:47 -05:00
Mert
56b0643890 feat(server)!: pgvecto.rs 0.2 and pgvector compatibility (#6785)
* basic changes

update version check

set ef_search for clip

* pgvector compatibility

Revert "pgvector compatibility"

This reverts commit 2b66a52aa4097dd27da58138c5288fd87cb9b24a.

pgvector compatibility: minimal edition

pgvector startup check

* update extension at startup

* wording

shortened vector extension variable name

* nightly docker

* fixed version checks

* update tests

add tests for updating extension

remove unnecessary check

* simplify `getRuntimeConfig`

* wording

* reindex on minor version update

* 0.2 upgrade testing

update prod compose

* acquire lock for init

* wip vector down on shutdown

* use upgrade helper

* update image tag

* refine restart check

check error message

* test reindex

testing

upstream fix

formatting

fixed reindexing

* use enum in signature

* fix tests

remove unused code

* add reindexing tests

* update to official 0.2

remove alpha from version name

* add warning test if restart required

* update test image to 0.2.0

* linting and test cleanup

* formatting

* update sql

* wording

* handle setting search path for new and existing databases

* handle new db in reindex check

* fix post-update reindexing

* get dim size

* formatting

* use vbase

* handle different db name

* update sql

* linting

* fix suggested env
2024-02-06 21:46:38 -05:00
renovate[bot]
d88ca5fb2a chore(deps): update server (#6930)
* chore(deps): update server

* chore: generate sql

---------

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Daniel Dietzler <mail@ddietzler.dev>
2024-02-06 22:06:02 +00:00
martin
fa0913120d feat(web,server): search people (#5703)
* feat: search peoples

* fix: responsive design

* use existing count

* generate sql file

* fix: tests

* remove visible people

* fix: merge, hide...

* use component

* fix: linter

* chore: regenerate api

* fix: change name when searching for a face

* save search

* remove duplicate

* use enums for query parameters

* fix: increase to 20 for the local search

* use constants

* simplify

* fix: number of people more visible

* fix: merge

* fix: search

* fix: loading spinner position

* pr feedback
2024-01-27 19:54:31 -05:00
Zack Pollard
25cad79657 refactor: move asset stacks to their own entity (#6353)
* feat: auto-stack burst photos

* feat: move stacks to asset stack entity

* chore: pin node version with volta in server

* chore: update e2e cases

* chore: cleanup

* feat: migrate existing stacks

---------

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
2024-01-27 18:52:14 +00:00
Jason Rasmussen
7fc4abba72 feat(server): sql access checks (#6635) 2024-01-25 10:14:38 -05:00
Daniel Dietzler
607fd39130 fix(server): only calculate quota usage for internal assets (#6556)
only calculate usage for internal assets
2024-01-21 15:48:29 -06:00
Mert
68f52818ae feat(server): separate face clustering job (#5598)
* separate facial clustering job

* update api

* fixed some tests

* invert clustering

* hdbscan

* update api

* remove commented code

* wip dbscan

* cleanup

removed cluster endpoint

remove commented code

* fixes

updated tests

minor fixes and formatting

fixed queuing

refinements

* scale search range based on library size

* defer non-core faces

* optimizations

removed unused query option

* assign faces individually for correctness

fixed unit tests

remove unused method

* don't select face embedding

update sql

linting

fixed ml typing

* updated job mock

* paginate people query

* select face embeddings because typeorm

* fix setting face detection concurrency

* update sql

formatting

linting

* simplify logic

remove unused imports

* more specific delete signature

* more accurate typing for face stubs

* add migration

formatting

* chore: better typing

* don't select embedding by default

remove unused import

* updated sql

* use normal try/catch

* stricter concurrency typing and enforcement

* update api

* update job concurrency panel to show disabled queues

formatting

* check jobId in queueAll

fix tests

* remove outdated comment

* better facial recognition icon

* wording

wording

formatting

* fixed tests

* fix

* formatting & sql

* try to fix sql check

* more detailed description

* update sql

* formatting

* wording

* update `minFaces` description

---------

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
2024-01-18 00:08:48 -05:00
Jason Rasmussen
6f291006e4 fix(server): handle 5 digit years (#6457) 2024-01-17 16:08:38 -05:00
renovate[bot]
0350058689 fix(deps): update server (#6415)
* fix(deps): update server

* chore: fix tests

---------

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
2024-01-17 18:24:51 +00:00
Sushain Cherivirala
7fc1954e2a fix(server): add filename search (#6394)
Fixes https://github.com/immich-app/immich/issues/5982.

There are basically three options:

1. Search `originalFileName` by dropping a file extension from the query
(if present). Lower fidelity but very easy - just a standard index &
equality.
2. Search `originalPath` by adding an index on `reverse(originalPath)`
and using `starts_with(reverse(query) + "/", reverse(originalPath)`. A
weird index & query but high fidelity.
3. Add a new generated column called `originalFileNameWithExtension` or
something. More storage, kinda jank.

TBH, I think (1) is good enough and easy to make better in the future.
For example, if I search "DSC_4242.jpg", I don't really think it matters
if "DSC_4242.mov" also shows up.

edit: There's a fourth approach that we discussed a bit in Discord and
decided we could switch to it in the future: using a GIN. The minor
issue is that Postgres doesn't tokenize paths in a useful (they're a
single token and it won't match against partial components). We can
solve that by tokenizing it ourselves. For example:

```
immich=# with vecs as (select to_tsvector('simple', array_to_string(string_to_array('upload/library/sushain/2015/2015-08-09/IMG_275.JPG', '/'), ' ')) as vec)  select * from vecs where vec @@ phraseto_tsquery('simple', array_to_string(string_to_array('library/sushain', '/'), ' '));
                                      vec
-------------------------------------------------------------------------------
 '-08':6 '-09':7 '2015':4,5 'img_275.jpg':8 'library':2 'sushain':3 'upload':1
(1 row)
```

The query is also tokenized with the 'split-by-slash-join-with-space'
strategy. This strategy results in `IMG_275.JPG`, `2015`, `sushain` and
`library/sushain` matching. But, `08` and `IMG_275` do not match. The
former is because the token is `-08` and the latter because the
`img_275.jpg` token is matched against exactly.
2024-01-15 14:40:28 -06:00
Alex
d096caccac chore(web): quota enhancement (#6371)
* chore(web): quota enhancement

* show quota in user table

* update quota for single user ioption

* Add a note how to set unlimited storage

* fixed deletion doesn't update quota

* refactor relation

* fixed test

* re-refactor

* update sql

* fix e2e test

* Update server/src/domain/user/user.service.ts

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>

* revert e2e test

---------

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
2024-01-15 09:04:29 -06:00
cfitzw
deb1f970a8 feat(server, web): quotas (#4471)
* feat: quotas

* chore: open api

* chore: update status box and upload error message

---------

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
2024-01-12 19:43:36 -05:00
Michael Manganiello
8921278447 fix(server): Execute query in AlbumRepository.removeAsset method (#6216)
The current `removeAsset` implementation just builds the query but does
not execute it. That also seems to be the reason the `@GenerateSql`
decorator was commented out.
2024-01-06 11:56:08 -06:00
Emanuel Bennici
d1e16025cf fix(server): Fix incorrect sorting with search (#6031)
The search results are not sorted by date causing confision.

Fixes #5916
2024-01-04 14:47:09 -06:00
Jason Rasmussen
7b40c20ea5 chore(server): time bucket sql (#6156) 2024-01-03 22:32:52 -06:00
Fynn Petersen-Frey
cc7ba3c21a feat(server): search across own+partner assets (#5966)
* feat(server): search across own+partner assets

* generate sql

* fix sql parameter
2024-01-01 17:25:22 -05:00
martin
0ed89e61ec fix: remove archived people from explore (#6091) 2024-01-01 11:07:42 -05:00
Mert
cc2dc12f6c fix(server): run migrations after database checks (#5832)
* run migrations after checks

* optional migrations

* only run checks in server and e2e

* re-add migrations for microservices

* refactor

* move e2e init

* remove assert from migration

* update providers

* update microservices app service

* fixed logging

* refactored version check, added unit tests

* more version tests

* don't use mocks for sut

* refactor tests

* suggest image only if postgres is 14, 15 or 16

* review suggestions

* fixed regexp escape

* fix typing

* update migration
2023-12-21 10:06:26 -06:00
Mert
b520955d0e fix(server): add more conditions to smart search (#5806)
* add more asset conditions

* udpate sql
2023-12-17 20:17:30 -06:00
Mert
6e7b3d6f24 fix(server): fix metadata search not working (#5800)
* don't require ml

* update e2e

* fixes

* fix e2e

* add additional conditions

* select all exif columns

* more fixes

* update sql
2023-12-17 20:16:08 -06:00
Mert
e3fa32ad23 fix(server): fix inconsistent explore queries (#5774)
* remove limits

* update sql
2023-12-17 11:04:35 -06:00
Alex
f206cb9403 chore(server): simplify search face query and better clustering (#5573)
* chore(server): simplify search face query and better clustering

* update sql

* Use correct syntax for utilizing the index

* Update sql
2023-12-08 16:26:17 -06:00
Mert
2553c54b26 fix(server): select asset face columns explicitly (#5564)
* select columns explicitly

* updated sql

* formatting
2023-12-08 12:43:35 -06:00
Jason Rasmussen
1e99ba8167 feat: use pgvecto.rs (#3605) 2023-12-08 11:15:46 -05:00
martin
7702560b12 feat(web): re-assign person faces (2) (#4949)
* feat: unassign person faces

* multiple improvements

* chore: regenerate api

* feat: improve face interactions in photos

* fix: tests

* fix: tests

* optimize

* fix: wrong assignment on complex-multiple re-assignments

* fix: thumbnails with large photos

* fix: complex reassign

* fix: don't send people with faces

* fix: person thumbnail generation

* chore: regenerate api

* add tess

* feat: face box even when zoomed

* fix: change feature photo

* feat: make the blue icon hoverable

* chore: regenerate api

* feat: use websocket

* fix: loading spinner when clicking on the done button

* fix: use the svelte way

* fix: tests

* simplify

* fix: unused vars

* fix: remove unused code

* fix: add migration

* chore: regenerate api

* ci: add unit tests

* chore: regenerate api

* feat: if a new person is created for a face and the server takes more than 15 seconds to generate the person thumbnail, don't wait for it

* reorganize

* chore: regenerate api

* feat: global edit

* pr feedback

* pr feedback

* simplify

* revert test

* fix: face generation

* fix: tests

* fix: face generation

* fix merge

* feat: search names in unmerge face selector modal

* fix: merge face selector

* simplify feature photo generation

* fix: change endpoint

* pr feedback

* chore: fix merge

* chore: fix merge

* fix: tests

* fix: edit & hide buttons

* fix: tests

* feat: show if person is hidden

* feat: rename face to person

* feat: split in new panel

* copy-paste-error

* pr feedback

* fix: feature photo

* do not leak faces

* fix: unmerge modal

* fix: merge modal event

* feat(server): remove duplicates

* fix: title for image thumbnails

* fix: disable side panel when there's no face until next PR

---------

Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
2023-12-05 09:43:15 -06:00
PyKen
1a63d3837e fix(server): Return correct asset count in album (#5465)
* fix(server): Return correct asset count in album

* Update album.repository.sql

Add generated sql
2023-12-04 10:22:00 -06:00
Jason Rasmussen
5e55a17b2a chore(server): sql versioning (#5346)
* chore(server): sql versioning

* chore: always add newline to end of file

* refactor: generator

* chore: pr feedback

* chore: pr feedback
2023-11-30 10:10:30 -05:00