[PR #388] [MERGED] fix: use WAL for SQLite by default and set busy_timeout #831

Open
opened 2025-10-07 00:23:08 +03:00 by OVERLORD · 0 comments
Owner

📋 Pull Request Information

Original PR: https://github.com/pocket-id/pocket-id/pull/388
Author: @ItalyPaleAle
Created: 3/27/2025
Status: Merged
Merged: 3/30/2025
Merged by: @stonith404

Base: mainHead: sqlite-connstring


📝 Commits (7)

  • fc16a75 fix: use WAL for SQLite by default and set busy_timeout
  • 6ceff7b Updated as requested
  • 565a52a Merge branch 'main' into sqlite-connstring
  • 85684c2 Merge branch 'main' into sqlite-connstring
  • 725e678 Merge branch 'main' into sqlite-connstring
  • 912b1db Use a separate migration method
  • 734acfe Merge branch 'main' into sqlite-connstring

📊 Changes

5 files changed (+64 additions, -15 deletions)

View changed files

📝 .github/workflows/e2e-tests.yml (+1 -1)
📝 backend/internal/bootstrap/bootstrap.go (+2 -0)
backend/internal/bootstrap/config_migration.go (+34 -0)
📝 backend/internal/bootstrap/db_bootstrap.go (+18 -7)
📝 backend/internal/common/env_config.go (+9 -7)

📄 Description

When using SQLite in a web application, it's common to enable WAL which makes it possible to get better performance especially for concurrent access. WAL is supported by Gorm since under the hood it does use the C sqlite library, so it should be enabled by default.

Additionally, this PR makes sure to set the explicit _txlock=immediate in the connection string unless the user set another value (not recommended). It also sets a busy_timeout to 2500ms (unless another value is set by the user) to make SQLite wait if there's already a transaction/query that has locked the database - otherwise, SQLite would just error immediately, showing a failure to the user (not a good experimence). (Note that when WAL is enabled, SQLite creates 2 additional files next to the database called pocket-id.db-shm and pocket-id.db-wal)

One thing to note is that Pocket ID's configuration uses the terminology "database path" for SQLite, assuming that only paths to files can be passed. However, the underlying driver can accept either a path, or a connection string (starting with file:), which allows passing additional options as query string args. This is what we rely on.

Most users will not need to change a thing in how they use Pocket ID.

The only situation where one would want to change anything is to disable the WAL if the SQLite DB is running on a networked filesystem, such as NFS or SMB. However, even with the WAL disabled, running SQLite over a networked file system is strongly discouraged by the SQLite developers themselves, as it's risky in all cases.


🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/pocket-id/pocket-id/pull/388 **Author:** [@ItalyPaleAle](https://github.com/ItalyPaleAle) **Created:** 3/27/2025 **Status:** ✅ Merged **Merged:** 3/30/2025 **Merged by:** [@stonith404](https://github.com/stonith404) **Base:** `main` ← **Head:** `sqlite-connstring` --- ### 📝 Commits (7) - [`fc16a75`](https://github.com/pocket-id/pocket-id/commit/fc16a75cf4f001295842f83906ea881424aa5d56) fix: use WAL for SQLite by default and set busy_timeout - [`6ceff7b`](https://github.com/pocket-id/pocket-id/commit/6ceff7b603139a8820e0d7c6727ce0f8345a9ee5) Updated as requested - [`565a52a`](https://github.com/pocket-id/pocket-id/commit/565a52a247882ea7216873c56c8fdc790cc79fd5) Merge branch 'main' into sqlite-connstring - [`85684c2`](https://github.com/pocket-id/pocket-id/commit/85684c2bb4c89ca3d5c35f4581e716b6a630a315) Merge branch 'main' into sqlite-connstring - [`725e678`](https://github.com/pocket-id/pocket-id/commit/725e678d6d2c7cd4d765f54478653a6d60173c5d) Merge branch 'main' into sqlite-connstring - [`912b1db`](https://github.com/pocket-id/pocket-id/commit/912b1dbb877d6da00680c4f8197fcb931c7e7fe0) Use a separate migration method - [`734acfe`](https://github.com/pocket-id/pocket-id/commit/734acfecd1ffa5ad117240014202ee197f09585e) Merge branch 'main' into sqlite-connstring ### 📊 Changes **5 files changed** (+64 additions, -15 deletions) <details> <summary>View changed files</summary> 📝 `.github/workflows/e2e-tests.yml` (+1 -1) 📝 `backend/internal/bootstrap/bootstrap.go` (+2 -0) ➕ `backend/internal/bootstrap/config_migration.go` (+34 -0) 📝 `backend/internal/bootstrap/db_bootstrap.go` (+18 -7) 📝 `backend/internal/common/env_config.go` (+9 -7) </details> ### 📄 Description When using SQLite in a web application, it's common to [enable WAL](https://www.sqlite.org/wal.html) which makes it possible to get better performance especially for concurrent access. WAL is supported by Gorm since under the hood it does use the C sqlite library, so it should be enabled by default. Additionally, this PR makes sure to set the explicit `_txlock=immediate` in the connection string unless the user set another value (not recommended). It also sets a `busy_timeout` to 2500ms (unless another value is set by the user) to make SQLite wait if there's already a transaction/query that has locked the database - otherwise, SQLite would just error immediately, showing a failure to the user (not a good experimence). (Note that when WAL is enabled, SQLite creates 2 additional files next to the database called `pocket-id.db-shm` and `pocket-id.db-wal`) One thing to note is that Pocket ID's configuration uses the terminology "database path" for SQLite, assuming that only paths to files can be passed. However, the underlying driver can accept either a path, or a connection string (starting with `file:`), which allows passing additional options as query string args. This is what we rely on. Most users will not need to change a thing in how they use Pocket ID. The only situation where one would want to change anything is to disable the WAL if the SQLite DB is running on a networked filesystem, such as NFS or SMB. However, even with the WAL disabled, running SQLite over a networked file system is [strongly discouraged](https://www.sqlite.org/faq.html#q5) by the SQLite developers themselves, as it's risky in all cases. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
OVERLORD added the pull-request label 2025-10-07 00:23:08 +03:00
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/pocket-id#831