feat(server): Automatic watching of library folders (#6192)

* feat: initial watch support

* allow offline files

* chore: ignore query errors when resetting e2e db

* revert db query

* add savepoint

* guard the user query

* chore: openapi and db migration

* wip

* support multiple libraries

* fix tests

* wip

* can now cleanup chokidar watchers

* fix unit tests

* add library watch queue

* add missing init from merge

* wip

* can now filter file extensions

* remove watch api from non job client

* Fix e2e test

* watch library with updated import path and exclusion pattern

* add library watch frontend ui

* case sensitive watching extensions

* can auto watch libraries

* move watcher e2e tests to separate file

* don't watch libraries from a queue

* use event emitters

* shorten e2e test timeout

* refactor chokidar code to filesystem provider

* expose chokidar parameters to config file

* fix storage mock

* set default config for library watching

* add fs provider mocks

* cleanup

* add more unit tests for watcher

* chore: fix format + sql

* add more tests

* move unwatch feature back to library service

* add file event unit tests

* chore: formatting

* add documentation

* fix e2e tests

* chore: fix e2e tests

* fix library updating

* test cleanup

* fix typo

* cleanup

* fixing as per pr comments

* reduce library watch config file

* update storage config and mocks

* move negative event tests to unit tests

* fix library watcher e2e

* make watch configuration global

* remove the feature flag

* refactor watcher teardown

* fix microservices init

* centralize asset scan job queue

* improve docs

* add more tests

* chore: open api

* initialize app service

* fix docs

* fix library watch feature flag

* Update docs/docs/features/libraries.md

Co-authored-by: Daniel Dietzler <36593685+danieldietzler@users.noreply.github.com>

* fix: import right app service

* don't be truthy

* fix test speling

* stricter library update tests

* move fs watcher mock to external file

* subscribe to config changes

* docker does not need polling

* make library watch() private

* feat: add configuration ui

---------

Co-authored-by: Daniel Dietzler <36593685+danieldietzler@users.noreply.github.com>
Co-authored-by: Alex Tran <alex.tran1502@gmail.com>

mobile/openapi/lib/model/system_config_library_dto.dart

@@ -14,25 +14,31 @@ class SystemConfigLibraryDto {
   /// Returns a new [SystemConfigLibraryDto] instance.
   SystemConfigLibraryDto({
     required this.scan,
+    required this.watch,
   });
 
   SystemConfigLibraryScanDto scan;
 
+  SystemConfigLibraryWatchDto watch;
+
   @override
   bool operator ==(Object other) => identical(this, other) || other is SystemConfigLibraryDto &&
-    other.scan == scan;
+    other.scan == scan &&
+    other.watch == watch;
 
   @override
   int get hashCode =>
     // ignore: unnecessary_parenthesis
-    (scan.hashCode);
+    (scan.hashCode) +
+    (watch.hashCode);
 
   @override
-  String toString() => 'SystemConfigLibraryDto[scan=$scan]';
+  String toString() => 'SystemConfigLibraryDto[scan=$scan, watch=$watch]';
 
   Map<String, dynamic> toJson() {
     final json = <String, dynamic>{};
       json[r'scan'] = this.scan;
+      json[r'watch'] = this.watch;
     return json;
   }
 
@@ -45,6 +51,7 @@ class SystemConfigLibraryDto {
 
       return SystemConfigLibraryDto(
         scan: SystemConfigLibraryScanDto.fromJson(json[r'scan'])!,
+        watch: SystemConfigLibraryWatchDto.fromJson(json[r'watch'])!,
       );
     }
     return null;
@@ -93,6 +100,7 @@ class SystemConfigLibraryDto {
   /// The list of required keys that must be present in a JSON.
   static const requiredKeys = <String>{
     'scan',
+    'watch',
   };
 }