starred/BookStack
1
0
Fork 0
mirror of https://github.com/BookStackApp/BookStack.git synced 2026-02-06 19:06:02 +03:00
Code Issues 784 Packages Projects Releases 10 Wiki Activity

[PR #5800] [MERGED] Big scary entity table change #6574

New Issue
Closed
opened 2026-02-05 10:35:53 +03:00 by OVERLORD · 0 comments
OVERLORD commented 2026-02-05 10:35:53 +03:00
Owner
Copy Link

📋 Pull Request Information

Original PR: https://github.com/BookStackApp/BookStack/pull/5800
Author: @ssddanbrown
Created: 9/15/2025
Status: ✅ Merged
Merged: 10/18/2025
Merged by: @ssddanbrown

Base: development ← Head: db_restructure

📝 Commits (10+)

  • 50b147c DB: Planned out new entity table format via migrations
  • 4e935f7 DB: Created entity migration logic
  • bc49db7 DB: Added change of entity relation columns to suit new entities table
  • c767dde DB: Got most view queries working for new stucture
  • c60052b Entities: Started logic change to new structure
  • bf09b42 Entities: Been through repos to update for new format
  • b866dee Entities: Updated repos to act on refreshed clones
  • 5c2908e Entities: Updated model classes & relations for changes
  • 06ec914 Entities: Changed from *Data to a common "contents" system
  • 4f21e54 Entities: Moved entity description/covers to own non-model classes

📊 Changes

120 files changed (+1598 additions, -595 deletions)

View changed files

📝 app/Access/Mfa/MfaSession.php (+0 -1)
📝 app/Console/Commands/UpdateUrlCommand.php (+2 -4)
📝 app/Entities/Controllers/BookApiController.php (+4 -3)
📝 app/Entities/Controllers/BookshelfApiController.php (+4 -3)
📝 app/Entities/Controllers/BookshelfController.php (+1 -0)
📝 app/Entities/Controllers/ChapterApiController.php (+2 -2)
📝 app/Entities/Controllers/ChapterController.php (+1 -1)
📝 app/Entities/Controllers/PageController.php (+1 -0)
➕ app/Entities/EntityExistsRule.php (+20 -0)
📝 app/Entities/Models/Book.php (+33 -58)
📝 app/Entities/Models/BookChild.php (+2 -3)
📝 app/Entities/Models/Bookshelf.php (+20 -54)
📝 app/Entities/Models/Chapter.php (+12 -17)
➕ app/Entities/Models/ContainerTrait.php (+26 -0)
➖ app/Entities/Models/CoverImageInterface.php (+0 -18)
📝 app/Entities/Models/Entity.php (+102 -9)
➕ app/Entities/Models/EntityContainerData.php (+52 -0)
➕ app/Entities/Models/EntityPageData.php (+25 -0)
➕ app/Entities/Models/EntityQueryBuilder.php (+38 -0)
➕ app/Entities/Models/EntityScope.php (+27 -0)

...and 80 more files

📄 Description

This PR aims to merge the core entity types (books, chapters, pages, shelves) into one core table, with sub-tables for added data.

This moves things into three new tables:

  • entities - All items, containing most shared metadata.
  • entity_container_data - The added data for container (non-page) items.
  • entity_page_data - The page specific data.

This is something I've wanted to do for a while.
These core structural items are often queried and used in the same locations, and joint with many of the same relations.
This alignment allows querying & fetching & joining across all items via one indexed table, instead of needing to run multiple queries.

This should bring a lot of query efficiency, while allowing things like properly pageable and ordered search results (compared to the sketchy non-predictable-length handling we have now due to separate query results being combined together).

This also takes the opportunity to perform some data cleaning, like changing 0 relation Ids to nulls, switching to unsigned bigint for entity ids.

Code logic approach

  • We'll handle joins automatically at query level, so usage in almost all read-based cases can stay the same.

Considerations

  • As part of this, the use of numeric ids will be changing. Existing item ids should remain the same, but for new items, the id values will be incrementing across all entity items (although is not assured to be unique across all, to remain back compatible).
  • This new table relies strongly on multi-column primary keys, which have before caused issues in some environments, but I think should be fine to have here since they're already required for the joint_permissions table.
  • Migrations will take a while, will need to note this to ensure it's left. May cause trouble with watchtower instances. As a reference, my dev instance took 30s for the change in relation column types.

Todo

  • Complete data conversion migration.
    • Core migration of main data
    • Fix of created_by, updated_by, owned_by & chater_id columns to convert 0 to null, and remove where no relation exists
  • Complete entity relation column conversion migration.
  • Ensure we're handling deleting of books/shelves relations on destroy since they are no longer foreign key constrained.
  • Add query handling logic to app.
  • Update logical use in app.
    • Create new multi-column-key entity object for entities.
      • Really need this for working with base methods like refresh, find, delete etc...
      • Delete filtering at global scope level if possible? (Check single, query, and relation usage since scopes are used for all).
        • Done this at a more direct way instead by overriding the base queries to set the global scope.
    • Update entity model hydration to move extra data loaded (via joins) into relation rather than stuck in model.
    • Update entity model relations.
    • Update usage in repos.
    • Update usage in related services.
    • Update usage in views.
  • Address remaining code todo notes.
  • Check API responses against samples to ensure no changes.

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/BookStackApp/BookStack/pull/5800 **Author:** [@ssddanbrown](https://github.com/ssddanbrown) **Created:** 9/15/2025 **Status:** ✅ Merged **Merged:** 10/18/2025 **Merged by:** [@ssddanbrown](https://github.com/ssddanbrown) **Base:** `development` ← **Head:** `db_restructure` --- ### 📝 Commits (10+) - [`50b147c`](https://github.com/BookStackApp/BookStack/commit/50b147c632926700fcedbc721f15f78a776693f5) DB: Planned out new entity table format via migrations - [`4e935f7`](https://github.com/BookStackApp/BookStack/commit/4e935f7de001e6204e32e251756089719ed1043c) DB: Created entity migration logic - [`bc49db7`](https://github.com/BookStackApp/BookStack/commit/bc49db75093feb53edc7c2ae80ebefdfea0c0929) DB: Added change of entity relation columns to suit new entities table - [`c767dde`](https://github.com/BookStackApp/BookStack/commit/c767ddedb8fa688ed4cf91c65a4fa2abf58b1f5c) DB: Got most view queries working for new stucture - [`c60052b`](https://github.com/BookStackApp/BookStack/commit/c60052b5e195563589c69e75ebbafe2697ca8762) Entities: Started logic change to new structure - [`bf09b42`](https://github.com/BookStackApp/BookStack/commit/bf09b42dcfc83502818ba0ae114b097ab37545a7) Entities: Been through repos to update for new format - [`b866dee`](https://github.com/BookStackApp/BookStack/commit/b866dee0cf01a8576e5fce5bb2bde54107aa9b62) Entities: Updated repos to act on refreshed clones - [`5c2908e`](https://github.com/BookStackApp/BookStack/commit/5c2908ef4dafab7c6ba88714bb4f3639b07c8a1c) Entities: Updated model classes & relations for changes - [`06ec914`](https://github.com/BookStackApp/BookStack/commit/06ec914a6e39f60bcf4c565741afa89213c86eec) Entities: Changed from *Data to a common "contents" system - [`4f21e54`](https://github.com/BookStackApp/BookStack/commit/4f21e5472675706fd6911be373f27b82ce1fa561) Entities: Moved entity description/covers to own non-model classes ### 📊 Changes **120 files changed** (+1598 additions, -595 deletions) <details> <summary>View changed files</summary> 📝 `app/Access/Mfa/MfaSession.php` (+0 -1) 📝 `app/Console/Commands/UpdateUrlCommand.php` (+2 -4) 📝 `app/Entities/Controllers/BookApiController.php` (+4 -3) 📝 `app/Entities/Controllers/BookshelfApiController.php` (+4 -3) 📝 `app/Entities/Controllers/BookshelfController.php` (+1 -0) 📝 `app/Entities/Controllers/ChapterApiController.php` (+2 -2) 📝 `app/Entities/Controllers/ChapterController.php` (+1 -1) 📝 `app/Entities/Controllers/PageController.php` (+1 -0) ➕ `app/Entities/EntityExistsRule.php` (+20 -0) 📝 `app/Entities/Models/Book.php` (+33 -58) 📝 `app/Entities/Models/BookChild.php` (+2 -3) 📝 `app/Entities/Models/Bookshelf.php` (+20 -54) 📝 `app/Entities/Models/Chapter.php` (+12 -17) ➕ `app/Entities/Models/ContainerTrait.php` (+26 -0) ➖ `app/Entities/Models/CoverImageInterface.php` (+0 -18) 📝 `app/Entities/Models/Entity.php` (+102 -9) ➕ `app/Entities/Models/EntityContainerData.php` (+52 -0) ➕ `app/Entities/Models/EntityPageData.php` (+25 -0) ➕ `app/Entities/Models/EntityQueryBuilder.php` (+38 -0) ➕ `app/Entities/Models/EntityScope.php` (+27 -0) _...and 80 more files_ </details> ### 📄 Description This PR aims to merge the core entity types (books, chapters, pages, shelves) into one core table, with sub-tables for added data. This moves things into three new tables: - `entities` - All items, containing most shared metadata. - `entity_container_data` - The added data for container (non-page) items. - `entity_page_data` - The page specific data. This is something I've wanted to do for a while. These core structural items are often queried and used in the same locations, and joint with many of the same relations. This alignment allows querying & fetching & joining across all items via one indexed table, instead of needing to run multiple queries. This should bring a lot of query efficiency, while allowing things like properly pageable and ordered search results (compared to the sketchy non-predictable-length handling we have now due to separate query results being combined together). This also takes the opportunity to perform some data cleaning, like changing `0` relation Ids to `nulls`, switching to `unsigned bigint` for entity ids. ### Code logic approach - We'll handle joins automatically at query level, so usage in almost all read-based cases can stay the same. ### Considerations - As part of this, the use of numeric ids will be changing. Existing item ids should remain the same, but for new items, the id values will be incrementing across all entity items (although is not assured to be unique across all, to remain back compatible). - This new table relies strongly on multi-column primary keys, which have before caused issues in some environments, but I think should be fine to have here since they're already required for the `joint_permissions` table. - Migrations will take a while, will need to note this to ensure it's left. May cause trouble with watchtower instances. As a reference, my dev instance took 30s for the change in relation column types. ### Todo - [x] Complete data conversion migration. - [x] Core migration of main data - [x] Fix of created_by, updated_by, owned_by & chater_id columns to convert 0 to null, and remove where no relation exists - [x] Complete entity relation column conversion migration. - [x] Ensure we're handling deleting of books/shelves relations on destroy since they are no longer foreign key constrained. - [x] Add query handling logic to app. - [x] Update logical use in app. - [x] Create new multi-column-key entity object for entities. - Really need this for working with base methods like refresh, find, delete etc... - [ ] ~~Delete filtering at global scope level if possible? (Check single, query, and relation usage since scopes are used for all)~~. - Done this at a more direct way instead by overriding the base queries to set the global scope. - [x] Update entity model hydration to move extra data loaded (via joins) into relation rather than stuck in model. - [x] Update entity model relations. - [x] Update usage in repos. - [x] Update usage in related services. - [x] Update usage in views. - [x] Address remaining code todo notes. - [x] Check API responses against samples to ensure no changes. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
OVERLORD added the pull-request label 2026-02-05 10:35:53 +03:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
🎨 Design
📖 Docs Update
🐛 Bug
🐛 Bug
:cat2:🐈 Possible duplicate
💿 Database
☕ Open to discussion
💻 Front-End
🐕 Support
🚪 Authentication
🌍 Translations
🔌 API Task
🏭 Back-End
⛲ Upstream
🔨 Feature Request
🛠️ Enhancement
🛠️ Enhancement
🛠️ Enhancement
❤️ Happy feedback
🔒 Security
🔍 Pending Validation
💆 UX
📝 WYSIWYG Editor
🌔 Out of scope
🔩 API Request
:octocat: Admin/Meta
🖌️ View Customization
❓ Question
🚀 Priority
🛡️ Blocked
🚚 Export System
♿ A11y
🔧 Maintenance
> Markdown Editor
pull-request
Mirrored from GitHub Pull Request
No Label pull-request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
OVERLORD
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/BookStack#6574
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?