[PR #5552] [CLOSED] Vector Search & LLM Integration #6517

New Issue

OVERLORD · 2026-02-05T10:34:28+03:00

OVERLORD commented

2026-02-05 10:34:28 +03:00

📋 Pull Request Information

Original PR: https://github.com/BookStackApp/BookStack/pull/5552
Author: @ssddanbrown
Created: 3/24/2025
Status: ❌ Closed

Base: development ← Head: vectors

📝 Commits (9)

2d55482 Vectors: Built content vector indexing system
b9ecf55 Vectors: Got basic LLM querying working using vector search context
e611b32 Vectors: Added command to regenerate for all
54f883e Improved vector text chunking
2c3100e Vectors: Started front-end work, moved to own controller
88ccd9e Vectors: Split out vector search and llm query runs
8eef5a1 Vectors: Updated query response to use server-side-events
bb08f62 Vectors: Finished core fetch & display functionality
8e0edb6 Merge branch 'development' into vectors

📊 Changes

25 files changed (+873 additions, -5 deletions)

View changed files

📝 app/Config/services.php (+12 -0)
➕ app/Console/Commands/RegenerateVectorsCommand.php (+46 -0)
➕ app/Search/Queries/EntityVectorGenerator.php (+89 -0)
➕ app/Search/Queries/LlmQueryRunner.php (+26 -0)
➕ app/Search/Queries/QueryController.php (+61 -0)
➕ app/Search/Queries/SearchVector.php (+26 -0)
➕ app/Search/Queries/Services/OpenAiVectorQueryService.php (+66 -0)
➕ app/Search/Queries/Services/VectorQueryService.php (+21 -0)
➕ app/Search/Queries/StoreEntityVectorsJob.php (+30 -0)
➕ app/Search/Queries/TextChunker.php (+79 -0)
➕ app/Search/Queries/VectorQueryServiceProvider.php (+38 -0)
➕ app/Search/Queries/VectorSearchResult.php (+17 -0)
➕ app/Search/Queries/VectorSearchRunner.php (+54 -0)
📝 app/Search/SearchController.php (+1 -0)
📝 app/Search/SearchIndex.php (+13 -1)
➕ database/migrations/2025_03_24_155748_create_search_vectors_table.php (+37 -0)
📝 package-lock.json (+22 -0)
📝 package.json (+1 -0)
📝 resources/js/components/index.ts (+1 -0)
➕ resources/js/components/query-manager.ts (+77 -0)

...and 5 more files

📄 Description

This is a proof of concept for some level of somewhat native LLM integration.
This manages vector indexing, searching and LLM querying, with vectors stored & queried in the local BookStack database, and an external LLM service (OpenAI, Ollama etc...) queried for embeddings & query execution using locally found vector-based results for context.

The result is a LLM generated response to user query, with a list of relevant BookStack content used for the query as reference.

Issues & Questionables

MySQL lacks proper support. MySQL 9 supports vector columns and indexes, but you can't query distance in the community versions. MariaDB is good from 11.7.
- Not sure how I feel about MariaDB specific features/functionality. Current implementation uses MariaDB specific functions.
No idea how non-en/mixed language content works for these systems yet.
Do we set official targets of support? How common is OpenAI API compatibility across other providers?
Does embedding size vary much across difference services/models?
- Seems to: all-minilm=384, openai:text-embedding-3-small=1536, openai:text-embedding-3-large=3072
- Inserting 384 length embeddings into vector(1536) mariadb column seemed to result in 0 hex values. Seems like the column size has to be correct, can't insert under?
  - Response from Sergei at MariaDB on this:
    - "all vectors in a column are strictly the same size, because the distance between two vectors of different sizes is not defined. if you store 384-dim vectors in vector(1536) you can pad them with zeros and the result will be exactly the same as storing them in vector(384) except that it'll use about 5x more storage and memory"
  - Will need to provide controls/mechanism to change this.
How is those accessed/shown in the UI? Option in the search bar maybe to toggle search type? Action/reference from the normal search page?
What needs to be configurable? Looks like a lot of things that could be tweaked, but don't want to be maintaining loads of specific options. Need to assess what's most important.

Considerations

This will really require a queue worker so indexing can be done in the background.
- This does mean that search index may be behind content.
Would need migration handling due to limited DB support, so maybe have the migration have widespread support by default but add vector parts via a command?
For a nice UX, would probably need to have results queried via JS rather than results on page load. Not a big deal, but I generally try to have read/view operations be non-JS friendly.
Lots of context-window/vector-size considerations to be made that have not been implemented yet.
We'd probably want to cap the context documents (at least those shown to a user) at a certain score to avoid showing completely irrelevant content.

Todo

Resolve above issues/considerations where needed.
Need to ensure vectors handled on content delete/update.
Resolve code TODOs
Cover functionality with testing
Show warning for LLM/AI-based output (inaccurate)
Extract text from views
Check against dark mode
Check accessibility
Add JS server-side-event lib to readme attribution list?

Implementation Notes

In some scenarios extra web-server config may be needed to prevent server-side buffering from messing with server-side-event handling. Not crucial, as won't break things, but can delay the response until finished on server, providing a worse experience. For example, for nginx proxies: proxy_buffering off; proxy_cache off;.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/BookStackApp/BookStack/pull/5552 **Author:** [@ssddanbrown](https://github.com/ssddanbrown) **Created:** 3/24/2025 **Status:** ❌ Closed **Base:** `development` ← **Head:** `vectors` --- ### 📝 Commits (9) - [`2d55482`](https://github.com/BookStackApp/BookStack/commit/2d5548240aa027c0e197df2eff9c9ae6794115cb) Vectors: Built content vector indexing system - [`b9ecf55`](https://github.com/BookStackApp/BookStack/commit/b9ecf55e1fb773eba2820b31ab91f132770532a6) Vectors: Got basic LLM querying working using vector search context - [`e611b32`](https://github.com/BookStackApp/BookStack/commit/e611b3239e94fe2e60f7d9159961998bce829b74) Vectors: Added command to regenerate for all - [`54f883e`](https://github.com/BookStackApp/BookStack/commit/54f883e815e6640564006aa5f5ceb82b79f8f8b8) Improved vector text chunking - [`2c3100e`](https://github.com/BookStackApp/BookStack/commit/2c3100e40180289c52c88250e63eefd4a6c52204) Vectors: Started front-end work, moved to own controller - [`88ccd9e`](https://github.com/BookStackApp/BookStack/commit/88ccd9e5b9f45964d616f2fa7d6fa18abb768a33) Vectors: Split out vector search and llm query runs - [`8eef5a1`](https://github.com/BookStackApp/BookStack/commit/8eef5a1ee7b423aa07aac83ad0b044616486a3ba) Vectors: Updated query response to use server-side-events - [`bb08f62`](https://github.com/BookStackApp/BookStack/commit/bb08f62327a7c7feab901bd89b832da79b832a3f) Vectors: Finished core fetch & display functionality - [`8e0edb6`](https://github.com/BookStackApp/BookStack/commit/8e0edb63c7aa821f7106577a39cbf5804bc77185) Merge branch 'development' into vectors ### 📊 Changes **25 files changed** (+873 additions, -5 deletions) <details> <summary>View changed files</summary> 📝 `app/Config/services.php` (+12 -0) ➕ `app/Console/Commands/RegenerateVectorsCommand.php` (+46 -0) ➕ `app/Search/Queries/EntityVectorGenerator.php` (+89 -0) ➕ `app/Search/Queries/LlmQueryRunner.php` (+26 -0) ➕ `app/Search/Queries/QueryController.php` (+61 -0) ➕ `app/Search/Queries/SearchVector.php` (+26 -0) ➕ `app/Search/Queries/Services/OpenAiVectorQueryService.php` (+66 -0) ➕ `app/Search/Queries/Services/VectorQueryService.php` (+21 -0) ➕ `app/Search/Queries/StoreEntityVectorsJob.php` (+30 -0) ➕ `app/Search/Queries/TextChunker.php` (+79 -0) ➕ `app/Search/Queries/VectorQueryServiceProvider.php` (+38 -0) ➕ `app/Search/Queries/VectorSearchResult.php` (+17 -0) ➕ `app/Search/Queries/VectorSearchRunner.php` (+54 -0) 📝 `app/Search/SearchController.php` (+1 -0) 📝 `app/Search/SearchIndex.php` (+13 -1) ➕ `database/migrations/2025_03_24_155748_create_search_vectors_table.php` (+37 -0) 📝 `package-lock.json` (+22 -0) 📝 `package.json` (+1 -0) 📝 `resources/js/components/index.ts` (+1 -0) ➕ `resources/js/components/query-manager.ts` (+77 -0) _...and 5 more files_ </details> ### 📄 Description This is a proof of concept for some level of somewhat native LLM integration. This manages vector indexing, searching and LLM querying, with vectors stored & queried in the local BookStack database, and an external LLM service (OpenAI, Ollama etc...) queried for embeddings & query execution using locally found vector-based results for context. The result is a LLM generated response to user query, with a list of relevant BookStack content used for the query as reference. ### Issues & Questionables - MySQL lacks proper support. MySQL 9 supports vector columns and indexes, but you can't query distance in the community versions. MariaDB is good from 11.7. - Not sure how I feel about MariaDB specific features/functionality. Current implementation uses MariaDB specific functions. - No idea how non-en/mixed language content works for these systems yet. - Do we set official targets of support? How common is OpenAI API compatibility across other providers? - Does embedding size vary much across difference services/models? - Seems to: `all-minilm`=384, `openai:text-embedding-3-small`=1536, `openai:text-embedding-3-large`=3072 - Inserting 384 length embeddings into `vector(1536)` mariadb column seemed to result in 0 hex values. Seems like the column size has to be correct, can't insert under? - Response from Sergei at MariaDB on this: - "all vectors in a column are strictly the same size, because the distance between two vectors of different sizes is not defined. if you store 384-dim vectors in vector(1536) you can pad them with zeros and the result will be exactly the same as storing them in vector(384) except that it'll use about 5x more storage and memory" - Will need to provide controls/mechanism to change this. - How is those accessed/shown in the UI? Option in the search bar maybe to toggle search type? Action/reference from the normal search page? - What *needs* to be configurable? Looks like a lot of things that could be tweaked, but don't want to be maintaining loads of specific options. Need to assess what's most important. ### Considerations - This will really require a queue worker so indexing can be done in the background. - This does mean that search index may be behind content. - Would need migration handling due to limited DB support, so maybe have the migration have widespread support by default but add vector parts via a command? - For a nice UX, would probably need to have results queried via JS rather than results on page load. Not a big deal, but I generally try to have read/view operations be non-JS friendly. - Lots of context-window/vector-size considerations to be made that have not been implemented yet. - We'd probably want to cap the context documents (at least those shown to a user) at a certain score to avoid showing completely irrelevant content. ### Todo - [ ] Resolve above issues/considerations where needed. - [ ] Need to ensure vectors handled on content delete/update. - [ ] Resolve code TODOs - [ ] Cover functionality with testing - [ ] Show warning for LLM/AI-based output (inaccurate) - [ ] Extract text from views - [ ] Check against dark mode - [ ] Check accessibility - [ ] Add JS server-side-event lib to readme attribution list? ### Implementation Notes - In some scenarios extra web-server config may be needed to prevent server-side buffering from messing with server-side-event handling. Not crucial, as won't break things, but can delay the response until finished on server, providing a worse experience. For example, for nginx proxies: `proxy_buffering off; proxy_cache off;`. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

OVERLORD added the pull-request label 2026-02-05 10:34:28 +03:00

OVERLORD closed this issue

2026-02-05 10:34:29 +03:00

Sign in to join this conversation.

Branches Tags

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: starred/BookStack#6517