immich-app/immich
1
0
Fork 0
mirror of https://github.com/immich-app/immich.git synced 2026-02-05 00:30:57 +03:00
Code Issues 14 Packages Projects Releases 10 Wiki Activity

[Feature]: Support more storage backend #197

New Issue
Closed
opened 2026-02-04 18:39:37 +03:00 by OVERLORD · 11 comments
OVERLORD commented 2026-02-04 18:39:37 +03:00
Owner
Copy Link

Originally created by @gromain on GitHub (Aug 10, 2022).

Feature detail

Right now, only a local folder is supported natively. Even though more can be supported through fuse mounted filesystems, but this method has drawbacks (performance hit mainly), so it would be nice to have support for more back-ends eventually.

S3 is an obvious choice that comes to mind and that would support a lot of providers, but there could be others (like WebDAV perhaps).

Platform

Server

Originally created by @gromain on GitHub (Aug 10, 2022). ### Feature detail Right now, only a local folder is supported natively. Even though more can be supported through fuse mounted filesystems, but this method has drawbacks (performance hit mainly), so it would be nice to have support for more back-ends eventually. S3 is an obvious choice that comes to mind and that would support a lot of providers, but there could be others (like WebDAV perhaps). ### Platform Server
OVERLORD commented 2026-02-04 18:39:57 +03:00
Author
Owner
Copy Link

@Nonobis commented on GitHub (Aug 10, 2022):

Minio would be great !

@Nonobis commented on GitHub (Aug 10, 2022): Minio would be great !
OVERLORD commented 2026-02-04 18:40:00 +03:00
Author
Owner
Copy Link

@gromain commented on GitHub (Aug 10, 2022):

Minio is compatible with S3!

@gromain commented on GitHub (Aug 10, 2022): Minio is compatible with S3!
OVERLORD commented 2026-02-04 18:40:16 +03:00
Author
Owner
Copy Link

@Nonobis commented on GitHub (Aug 12, 2022):

@gromain : i know :)

@Nonobis commented on GitHub (Aug 12, 2022): @gromain : i know :)
OVERLORD commented 2026-02-04 18:40:27 +03:00
Author
Owner
Copy Link

@palitu commented on GitHub (Aug 14, 2022):

Would this be local storage for thumbnails, and remote storage for files? That would maintain speed for scrolling.

Then you would need to pass a Uri to the app to retrieve the full file for viewing?

@palitu commented on GitHub (Aug 14, 2022): Would this be local storage for thumbnails, and remote storage for files? That would maintain speed for scrolling. Then you would need to pass a Uri to the app to retrieve the full file for viewing?
OVERLORD commented 2026-02-04 18:40:32 +03:00
Author
Owner
Copy Link

@akashkj commented on GitHub (Aug 14, 2022):

If it allows to have a backup also for the media, that would be awesome.
Like 2 storage backends - s3 and G-drive. If one of them is not reachable, switch to other. This could be defined in a priority list.

@akashkj commented on GitHub (Aug 14, 2022): If it allows to have a backup also for the media, that would be awesome. Like 2 storage backends - s3 and G-drive. If one of them is not reachable, switch to other. This could be defined in a priority list.
OVERLORD commented 2026-02-04 18:40:43 +03:00
Author
Owner
Copy Link

@gromain commented on GitHub (Aug 17, 2022):

In my mind it could have been for all objects, but @palitu you're right that thumbnails could still be local.
And yes, @akashkj, that's how I felt about it. Having two or more storage back-ends possible, that can either work as RAID0 or RAID1 (not exactly RAID, but either mirror or concatenate the available storage).

@gromain commented on GitHub (Aug 17, 2022): In my mind it could have been for all objects, but @palitu you're right that thumbnails could still be local. And yes, @akashkj, that's how I felt about it. Having two or more storage back-ends possible, that can either work as RAID0 or RAID1 (not exactly RAID, but either mirror or concatenate the available storage).
OVERLORD commented 2026-02-04 18:40:48 +03:00
Author
Owner
Copy Link

@zackpollard commented on GitHub (Aug 17, 2022):

In my mind, supporting a well defined platform such as S3-like APIs is a good idea and is something we can look into after the first major release is completed. As for the other idea of having multiple backends that we switch between, in my mind that is the job of the storage layer, not something that immich should natively support. We'll leave this here and probably get around to it at some point in the future!

@zackpollard commented on GitHub (Aug 17, 2022): In my mind, supporting a well defined platform such as S3-like APIs is a good idea and is something we can look into after the first major release is completed. As for the other idea of having multiple backends that we switch between, in my mind that is the job of the storage layer, not something that immich should natively support. We'll leave this here and probably get around to it at some point in the future!
OVERLORD commented 2026-02-04 18:40:50 +03:00
Author
Owner
Copy Link

@palitu commented on GitHub (Oct 3, 2022):

Really great to hear your desire for an S3 type backend. FYI, i use the local filesystem for most of immich, then a mounted NFS share for the main photo storage. My server doesnt have the local storage for the bulk data (especially with all the video's), but thumbs is more than appropriate for local.

I do think that there are some delays with video playing as a result of this, but i assume in the future it will be improved with this S3 feature.

ta

@palitu commented on GitHub (Oct 3, 2022): > Really great to hear your desire for an S3 type backend. FYI, i use the local filesystem for most of immich, then a mounted NFS share for the main photo storage. My server doesnt have the local storage for the bulk data (especially with all the video's), but thumbs is more than appropriate for local. I do think that there are some delays with video playing as a result of this, but i assume in the future it will be improved with this S3 feature. ta
OVERLORD commented 2026-02-04 18:41:03 +03:00
Author
Owner
Copy Link

@Cellivar commented on GitHub (Oct 11, 2022):

I just started writing this up as a new feature request before discovering this one. I would love to be able to use S3 storage for all blob objects and make a raw disk volume optional. Big +1 to this feature request!

At home I run a Nomad cluster, like a Kubernetes cluster but with less yaml involved. We use CSI-managed iSCSI storage volumes that are mounted directly into running docker images (or other tasks). The preference here is a single writer job and maybe multiple read-only mounts for other volumes. This means that sharing a directory is hard while communicating over the network is easy. Right now the docker-compose files seem to suggest mounting one volume to multiple hosts that all have write access which is a little scary (and hard to do with iSCSI). The performance hit of switching to NFS is very visible on our network.

There's an official TS SDK for all the bells and whistles MinIO provides, but I'd strongly suggest looking at just using the S3 SDK which would allow folks to use any S3-compatible storage backend.

This is of course asking a lot, adding a new storage backend like S3 is completely different than writing to disk 😄 I'm perfectly happy to be patient for this sort of feature.

@Cellivar commented on GitHub (Oct 11, 2022): I just started writing this up as a new feature request before discovering this one. I would love to be able to use S3 storage _for all blob objects_ and make _a raw disk volume optional_. Big +1 to this feature request! At home I run a Nomad cluster, like a Kubernetes cluster but with less yaml involved. We use CSI-managed iSCSI storage volumes that are mounted directly into running docker images (or other tasks). The preference here is a single writer job and _maybe_ multiple read-only mounts for other volumes. This means that _sharing a directory is hard_ while _communicating over the network is easy_. Right now the docker-compose files seem to suggest mounting one volume to multiple hosts that all have write access which is a little scary (and hard to do with iSCSI). The performance hit of switching to NFS is very visible on our network. There's an [official TS SDK](https://github.com/minio/minio-js) for all the bells and whistles MinIO provides, but I'd strongly suggest looking at just [using the S3 SDK which would allow folks to use any S3-compatible storage backend](https://northflank.com/guides/connect-nodejs-to-minio-with-tls-using-aws-s3). This is of course asking a lot, adding a new storage backend like S3 is completely different than writing to disk 😄 I'm perfectly happy to be patient for this sort of feature.
OVERLORD commented 2026-02-04 18:41:15 +03:00
Author
Owner
Copy Link

@athornfam2 commented on GitHub (Oct 31, 2022):

Would like to that I am interested in S3 storage access as well to offload.

@athornfam2 commented on GitHub (Oct 31, 2022): Would like to that I am interested in S3 storage access as well to offload.
OVERLORD commented 2026-02-04 18:41:25 +03:00
Author
Owner
Copy Link

@Zen3515 commented on GitHub (Dec 27, 2022):

I try mounting S3 with rclone and found that immich uses softlink which is not allowed in S3.

I guess this is one of the things to consider when implement this in the future.

Error: ENOSYS: function not implemented, link 'upload/04f8f839-9ee4-47fc-89b1-fb3883cf3008/original/WEB/33809e6c-1f1f-4bec-a8cc-7a0367493e8d.png' -> 'upload/04f8f839-9ee4-47fc-89b1-fb3883cf3008/2022/2022-12-11/hahaha.png'
@Zen3515 commented on GitHub (Dec 27, 2022): I try mounting S3 with rclone and found that immich uses softlink which is not allowed in S3. I guess this is one of the things to consider when implement this in the future. ```shell Error: ENOSYS: function not implemented, link 'upload/04f8f839-9ee4-47fc-89b1-fb3883cf3008/original/WEB/33809e6c-1f1f-4bec-a8cc-7a0367493e8d.png' -> 'upload/04f8f839-9ee4-47fc-89b1-fb3883cf3008/2022/2022-12-11/hahaha.png' ```
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
accessibility
changelog:enhancement
changelog:security
changelog:translation
cli
date-time
documentation
external-library
format
good first issue
needs-answer
nice to have
sharing
tech-debt
📱mobile
🖥️web
🗄️server
🧠machine-learning
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
OVERLORD
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: immich-app/immich#197
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?