starred/pocket-id-pocket-id-2
1
0
Fork 0
mirror of https://github.com/pocket-id/pocket-id.git synced 2025-12-11 07:32:57 +03:00
Code Issues 116 Packages Projects Releases 10 Wiki Activity

[PR #372] [MERGED] refactor: use atomic renames for uploaded files #853

New Issue
Closed
opened 2025-10-09 16:58:28 +03:00 by OVERLORD · 0 comments
OVERLORD commented 2025-10-09 16:58:28 +03:00
Owner
Copy Link

📋 Pull Request Information

Original PR: https://github.com/pocket-id/pocket-id/pull/372
Author: @ItalyPaleAle
Created: 3/21/2025
Status: ✅ Merged
Merged: 3/23/2025
Merged by: @stonith404

Base: main ← Head: fix-file-upload

📝 Commits (8)

  • 66eec1d fix: use atomic renames for uploaded files
  • 0f93c7c Avoid buffering encoded images in-memory
  • 8c51ac4 Small lints
  • c5b7177 Update backend/internal/service/user_service.go
  • 7864ec4 Use crc64
  • 76be054 Merge branch 'main' into fix-file-upload
  • a3e7ad7 Merge branch 'main' into fix-file-upload
  • 7376481 Merge branch 'main' into fix-file-upload

📊 Changes

4 files changed (+106 additions, -35 deletions)

View changed files

📝 backend/internal/model/app_config.go (+9 -0)
📝 backend/internal/service/user_service.go (+12 -17)
📝 backend/internal/utils/file_util.go (+66 -5)
📝 backend/internal/utils/image/profile_picture.go (+19 -13)

📄 Description

This PR fixes a bug in how uploaded files are handled, by using temporary files that are then atomically renamed to the final destination.

This ensures that in case of errors, there is no corrupted file saved on disk. For example, imagine scenarios such as pocket id crashing while writing the file, or a power failure, or more simply interrupted uploads... Because the stream was opened to the original file, only partial data would be written to the file.

The typical solution to this problem is to write to a temporary file first, and then perform a rename. In most OS's, renaming is an atomic operation.

Note that renaming must be done in the same filesystem, so the temporary file needs to be in the same directory. We can't use /tmp because that's often on a separate filesystem

PS: Also updates CreateProfilePicture to avoid buffering the entire encoded image in-memory

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/pocket-id/pocket-id/pull/372 **Author:** [@ItalyPaleAle](https://github.com/ItalyPaleAle) **Created:** 3/21/2025 **Status:** ✅ Merged **Merged:** 3/23/2025 **Merged by:** [@stonith404](https://github.com/stonith404) **Base:** `main` ← **Head:** `fix-file-upload` --- ### 📝 Commits (8) - [`66eec1d`](https://github.com/pocket-id/pocket-id/commit/66eec1da477dda0afc1a25296f55b9eeb564a8a4) fix: use atomic renames for uploaded files - [`0f93c7c`](https://github.com/pocket-id/pocket-id/commit/0f93c7c24d707eab90d1f3468d06d28a1dfa0a66) Avoid buffering encoded images in-memory - [`8c51ac4`](https://github.com/pocket-id/pocket-id/commit/8c51ac4287b8f870904de947a60e2c9d9355b34d) Small lints - [`c5b7177`](https://github.com/pocket-id/pocket-id/commit/c5b71770880b21cfe8aa8983ce72b0326d432cc2) Update backend/internal/service/user_service.go - [`7864ec4`](https://github.com/pocket-id/pocket-id/commit/7864ec4de1a14b5f25482d2265e96e47822d560b) Use crc64 - [`76be054`](https://github.com/pocket-id/pocket-id/commit/76be054d722602f2a62da9d9e0fcad2ff37a6efe) Merge branch 'main' into fix-file-upload - [`a3e7ad7`](https://github.com/pocket-id/pocket-id/commit/a3e7ad720b484dcd9a77d7c7af8e7912f81a5617) Merge branch 'main' into fix-file-upload - [`7376481`](https://github.com/pocket-id/pocket-id/commit/7376481ff5ea6a25f5ded5cfea2f30965a03f8a8) Merge branch 'main' into fix-file-upload ### 📊 Changes **4 files changed** (+106 additions, -35 deletions) <details> <summary>View changed files</summary> 📝 `backend/internal/model/app_config.go` (+9 -0) 📝 `backend/internal/service/user_service.go` (+12 -17) 📝 `backend/internal/utils/file_util.go` (+66 -5) 📝 `backend/internal/utils/image/profile_picture.go` (+19 -13) </details> ### 📄 Description This PR fixes a bug in how uploaded files are handled, by using temporary files that are then atomically renamed to the final destination. This ensures that in case of errors, there is no corrupted file saved on disk. For example, imagine scenarios such as pocket id crashing while writing the file, or a power failure, or more simply interrupted uploads... Because the stream was opened to the original file, only partial data would be written to the file. The typical solution to this problem is to write to a temporary file first, and then perform a rename. In most OS's, renaming is an atomic operation. Note that renaming must be done in the same filesystem, so the temporary file needs to be in the same directory. We can't use `/tmp` because that's often on a separate filesystem PS: Also updates `CreateProfilePicture` to avoid buffering the entire encoded image in-memory --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
OVERLORD added the pull-request label 2025-10-09 16:58:28 +03:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
breaking
bug
documentation
feature
needs more upvotes
open to pull requests
pull-request
Mirrored from GitHub Pull Request
No Label pull-request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
OVERLORD
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/pocket-id-pocket-id-2#853
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?