immich-app/immich
1
0
Fork 0
mirror of https://github.com/immich-app/immich.git synced 2026-02-05 00:30:57 +03:00
Code Issues 11 Packages Projects Releases 10 Wiki Activity

[BUG] Rclone not supported because of one move command #962

New Issue
Closed
opened 2026-02-04 23:43:16 +03:00 by OVERLORD · 6 comments
OVERLORD commented 2026-02-04 23:43:16 +03:00
Owner
Copy Link

Originally created by @cycneuramus on GitHub (Jun 17, 2023).

The bug

If I'm reading the source code correctly, the reason Immich currently doesn't support rclone mounted backends (see #1899, #1683) is simply because:

  • its way of moving files is by relying on the mv library, which:
    • works by calling fs.link (at least on clobber: false, which Immich is currently using), which:
      • creates a hard link of the source file at the destination path, at which:
        • mv will unlink the source file, thus having "moved" the file from source to destination.

Now, the problem is, rclone doesn't support hard links. Hence the move operation fails halfway with the ENOSYS: function not implemented, link ... error mentioned in the issues linked above.

I don't have a thorough understanding of the way Immich is supposed to work here, but if the point is to move the files and nothing more, it seems to me we could achieve full rclone compatibility—and with it support for countless storage backends—simply by reworking the move command (say, by using fs.rename(sourcePath, desPath); or something).

The OS that Immich Server is running on

Docker

Version of Immich Server

latest

Version of Immich Mobile App

latest

Platform with the issue

  • Server
  • Web
  • Mobile

Your docker-compose.yml content

not relevant

Your .env content

not relevant

Reproduction steps

Use an `rclone` remote as a storage backed for Immich and try e.g. to run the storage template migration job.

Additional information

No response

Originally created by @cycneuramus on GitHub (Jun 17, 2023). ### The bug If I'm reading the source code correctly, the reason Immich currently doesn't support `rclone` mounted backends (see #1899, #1683) is simply because: + its [way of moving files ](https://github.com/immich-app/immich/blob/main/server/src/infra/repositories/filesystem.provider.ts#L8) is by relying on the [mv library](https://github.com/andrewrk/node-mv), which: + works by calling [fs.link](https://github.com/andrewrk/node-mv/blob/master/index.js#L39) (at least on `clobber: false`, which Immich is currently using), which: + creates a hard link of the source file at the destination path, at which: + `mv` will unlink the source file, thus having "moved" the file from source to destination. Now, the problem is, `rclone` doesn't support hard links. Hence the move operation fails halfway with the `ENOSYS: function not implemented, link ...` error mentioned in the issues linked above. I don't have a thorough understanding of the way Immich is supposed to work here, but if the point is to move the files and nothing more, it seems to me we could achieve full `rclone` compatibility—and with it support for countless storage backends—simply by reworking the move command (say, by using `fs.rename(sourcePath, desPath);` or something). ### The OS that Immich Server is running on Docker ### Version of Immich Server latest ### Version of Immich Mobile App latest ### Platform with the issue - [X] Server - [ ] Web - [ ] Mobile ### Your docker-compose.yml content ```YAML not relevant ``` ### Your .env content ```Shell not relevant ``` ### Reproduction steps ```bash Use an `rclone` remote as a storage backed for Immich and try e.g. to run the storage template migration job. ``` ### Additional information _No response_
OVERLORD added the 🗄️servernice to have labels 2026-02-04 23:43:16 +03:00
OVERLORD commented 2026-02-04 23:43:22 +03:00
Author
Owner
Copy Link

@uhthomas commented on GitHub (Jun 17, 2023):

Hmm, so I believe Immich uses clobber to avoid overwriting existing files. The underlying package seems to assume that linking the file will implicitly not overwrite the destination, and is backed up by the documentation on link.

If newpath exists, it will not be overwritten.

Immich should probably just check if the file exists manually instead. This does introduce a potential race condition, though it may be behaviour which should just be accepted.

@uhthomas commented on GitHub (Jun 17, 2023): Hmm, so I believe Immich uses clobber to avoid overwriting existing files. The underlying package seems to assume that linking the file will implicitly not overwrite the destination, and is backed up by the documentation on [link](https://man7.org/linux/man-pages/man2/link.2.html). > If newpath exists, it will not be overwritten. Immich should probably just check if the file exists manually instead. This does introduce a potential race condition, though it may be behaviour which should just be accepted.
OVERLORD commented 2026-02-04 23:43:26 +03:00
Author
Owner
Copy Link

@cycneuramus commented on GitHub (Jun 18, 2023):

I wonder then what is the desired behavior. The clobber: false option seems to involve deleting the source file, so in moving away from hard links, would the same behavior therefore be achieved by checking manually for the destination file and, if found, simply delete the source...?

@cycneuramus commented on GitHub (Jun 18, 2023): I wonder then what is the desired behavior. The `clobber: false` option seems to involve [deleting the source file](https://github.com/andrewrk/node-mv/blob/master/index.js#L52), so in moving away from hard links, would the same behavior therefore be achieved by checking manually for the destination file and, if found, simply delete the source...?
OVERLORD commented 2026-02-04 23:43:32 +03:00
Author
Owner
Copy Link

@uhthomas commented on GitHub (Jun 18, 2023):

I believe link will error if the destination exists.

EEXIST newpath already exists.
@uhthomas commented on GitHub (Jun 18, 2023): I believe `link` will error if the destination exists. ``` EEXIST newpath already exists. ```
OVERLORD commented 2026-02-04 23:43:36 +03:00
Author
Owner
Copy Link

@cycneuramus commented on GitHub (Jun 18, 2023):

Are we talking about the same thing here? The mv library (to which I was referring) calls the fs.unlink() method.

@cycneuramus commented on GitHub (Jun 18, 2023): Are we talking about the same thing here? The `mv` library (to which I was referring) calls the [`fs.unlink()`](https://www.geeksforgeeks.org/node-js-fs-unlink-method) method.
OVERLORD commented 2026-02-04 23:43:39 +03:00
Author
Owner
Copy Link

@uhthomas commented on GitHub (Jun 18, 2023):

I think so. If you read the source for doRename, it will only unlink if there was no error during linking. Therefore, if the destination exists then the source remains untouched and an error is returned.

@uhthomas commented on GitHub (Jun 18, 2023): I think so. If you read the source for `doRename`, it will only unlink if there was no error during linking. Therefore, if the destination exists then the source remains untouched and an error is returned.
OVERLORD commented 2026-02-04 23:43:47 +03:00
Author
Owner
Copy Link

@cycneuramus commented on GitHub (Jun 18, 2023):

Ah, I misunderstood the reference to link. In any case, I'll see if I can cook up a PR one of these days and get the ball rolling.

@cycneuramus commented on GitHub (Jun 18, 2023): Ah, I misunderstood the reference to `link`. In any case, I'll see if I can cook up a PR one of these days and get the ball rolling.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
accessibility
changelog:enhancement
changelog:security
changelog:translation
cli
date-time
documentation
external-library
format
good first issue
needs-answer
nice to have
sharing
tech-debt
📱mobile
🖥️web
🗄️server
🧠machine-learning
No Label nice to have 🗄️server
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
OVERLORD
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: immich-app/immich#962
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?