Unable to login after one week #2182

Closed
opened 2025-10-09 17:49:27 +03:00 by OVERLORD · 5 comments
Owner

Originally created by @Gilneus on GitHub.

Hello,

I deployed bitwarden on AWS ECS about two weeks ago. First everything ran fine, I could use bitwarden without any problems. Since yesterday I have the problem that I can't log in anymore. Unfortunately I don't get a clear error message via the browser console. The error is "Error saving device". The whole call stack looks like this:

POST https://MYURL/identity/connect/token 400
(anonymous) @ polyfills.c8f35891102feaad9f9e.js:16
n.fetch @ main.c8f35891102feaad9f9e.js:1
(anonymous) @ main.c8f35891102feaad9f9e.js:1
(anonymous) @ main.c8f35891102feaad9f9e.js:1
(anonymous) @ main.c8f35891102feaad9f9e.js:1
(anonymous) @ main.c8f35891102feaad9f9e.js:1
t @ polyfills.c8f35891102feaad9f9e.js:16
t @ main.c8f35891102feaad9f9e.js:1
n.postIdentityToken @ main.c8f35891102feaad9f9e.js:1
(anonymous) @ main.c8f35891102feaad9f9e.js:1
(anonymous) @ main.c8f35891102feaad9f9e.js:1
(anonymous) @ main.c8f35891102feaad9f9e.js:1
r @ main.c8f35891102feaad9f9e.js:1
t.invoke @ polyfills.c8f35891102feaad9f9e.js:16
onInvoke @ vendor.c8f35891102feaad9f9e.js:200
t.invoke @ polyfills.c8f35891102feaad9f9e.js:16
e.run @ polyfills.c8f35891102feaad9f9e.js:16
(anonymous) @ polyfills.c8f35891102feaad9f9e.js:16
t.invokeTask @ polyfills.c8f35891102feaad9f9e.js:16
onInvokeTask @ vendor.c8f35891102feaad9f9e.js:200
t.invokeTask @ polyfills.c8f35891102feaad9f9e.js:16
e.runTask @ polyfills.c8f35891102feaad9f9e.js:16
d @ polyfills.c8f35891102feaad9f9e.js:16
e.invokeTask @ polyfills.c8f35891102feaad9f9e.js:16
m @ polyfills.c8f35891102feaad9f9e.js:39
b @ polyfills.c8f35891102feaad9f9e.js:39

Do any of you have any idea how I can fix this?

Originally created by @Gilneus on GitHub. Hello, I deployed bitwarden on AWS ECS about two weeks ago. First everything ran fine, I could use bitwarden without any problems. Since yesterday I have the problem that I can't log in anymore. Unfortunately I don't get a clear error message via the browser console. The error is "Error saving device". The whole call stack looks like this: ``` POST https://MYURL/identity/connect/token 400 (anonymous) @ polyfills.c8f35891102feaad9f9e.js:16 n.fetch @ main.c8f35891102feaad9f9e.js:1 (anonymous) @ main.c8f35891102feaad9f9e.js:1 (anonymous) @ main.c8f35891102feaad9f9e.js:1 (anonymous) @ main.c8f35891102feaad9f9e.js:1 (anonymous) @ main.c8f35891102feaad9f9e.js:1 t @ polyfills.c8f35891102feaad9f9e.js:16 t @ main.c8f35891102feaad9f9e.js:1 n.postIdentityToken @ main.c8f35891102feaad9f9e.js:1 (anonymous) @ main.c8f35891102feaad9f9e.js:1 (anonymous) @ main.c8f35891102feaad9f9e.js:1 (anonymous) @ main.c8f35891102feaad9f9e.js:1 r @ main.c8f35891102feaad9f9e.js:1 t.invoke @ polyfills.c8f35891102feaad9f9e.js:16 onInvoke @ vendor.c8f35891102feaad9f9e.js:200 t.invoke @ polyfills.c8f35891102feaad9f9e.js:16 e.run @ polyfills.c8f35891102feaad9f9e.js:16 (anonymous) @ polyfills.c8f35891102feaad9f9e.js:16 t.invokeTask @ polyfills.c8f35891102feaad9f9e.js:16 onInvokeTask @ vendor.c8f35891102feaad9f9e.js:200 t.invokeTask @ polyfills.c8f35891102feaad9f9e.js:16 e.runTask @ polyfills.c8f35891102feaad9f9e.js:16 d @ polyfills.c8f35891102feaad9f9e.js:16 e.invokeTask @ polyfills.c8f35891102feaad9f9e.js:16 m @ polyfills.c8f35891102feaad9f9e.js:39 b @ polyfills.c8f35891102feaad9f9e.js:39 ``` Do any of you have any idea how I can fix this?
Author
Owner

@mprasil commented on GitHub:

Ah, yes, this was probably it. We use WAL, which does not work with networked filesystem. We have some info in the wiki regarding this. Generally speaking sqlite doesn't work well on NFS, I've seen multiple projects completely breaking with sqlite DB on NFS.

As for restoring the corrupted DB I personally think that you should just restore a backup if you have it. If not, try googling something about restoring corrupted sqlite DB. From my quick search it seems like the common approach is to dump the DB:

mv db.sqlite3 corrupted_db.sqlite3
echo ".dump" | sqlite corrupted_db.sqlite3 | sqlite db.sqlite3

Make sure to create backup of the corrupted DB before trying to fix it.

As for running on NFS in the future, I wouldn't recommend it, but if you do want that consider disabling WAL or make sure it never scales to more than one instance running. (the whole thing is going to be bottlenecked by the DB operations anyway, so there's no benefit in running more instances)

@mprasil commented on GitHub: Ah, yes, this was probably it. We use WAL, which does not work with networked filesystem. We have [some info in the wiki](https://github.com/dani-garcia/bitwarden_rs/wiki/Running-without-WAL-enabled) regarding this. Generally speaking sqlite doesn't work well on NFS, I've seen multiple projects completely breaking with sqlite DB on NFS. As for restoring the corrupted DB I personally think that you should just [restore a backup](https://github.com/dani-garcia/bitwarden_rs/wiki/Backing-up-your-vault) if you have it. If not, try googling something about restoring corrupted sqlite DB. From my quick search it seems like the common approach is to dump the DB: ```bash mv db.sqlite3 corrupted_db.sqlite3 echo ".dump" | sqlite corrupted_db.sqlite3 | sqlite db.sqlite3 ``` Make sure to create backup of the corrupted DB before trying to fix it. As for running on NFS in the future, I wouldn't recommend it, but if you do want that consider disabling WAL or make sure it never scales to more than one instance running. (the whole thing is going to be bottlenecked by the DB operations anyway, so there's no benefit in running more instances)
Author
Owner

@mprasil commented on GitHub:

Hi @Gilneus, I'm going to close this as I don't think there's much we can do here. If you think we could improve something or add some documentation to avoid this, feel free to reopen.

@mprasil commented on GitHub: Hi @Gilneus, I'm going to close this as I don't think there's much we can do here. If you think we could improve something or add some documentation to avoid this, feel free to reopen.
Author
Owner

@Gilneus commented on GitHub:

I took a short view in the docker logs and found this:

[2019-02-21 08:40:54][rocket::rocket][INFO] POST /api/accounts/prelogin application/json; charset=utf-8:
[2019-02-21 08:40:54][_][INFO] Matched: POST /api/accounts/prelogin (prelogin)
[2019-02-21 08:40:54][_][INFO] Outcome: Success
[2019-02-21 08:40:54][_][INFO] Response succeeded.
[2019-02-21 08:40:54][rocket::rocket][INFO] POST /identity/connect/token application/x-www-form-urlencoded; charset=utf-8:
[2019-02-21 08:40:54][_][INFO] Matched: POST /identity/connect/token (login)
[2019-02-21 08:40:55][bitwarden_rs::error][ERROR] JsonError. {"TwoFactorProviders":[0],"TwoFactorProviders2":{"0":null},"error":"invalid_grant","error_description":"Two factor required."}
[2019-02-21 08:40:55][_][INFO] Outcome: Success
[2019-02-21 08:40:55][_][INFO] Response succeeded.
[2019-02-21 08:41:03][rocket::rocket][INFO] POST /identity/connect/token application/x-www-form-urlencoded; charset=utf-8:
[2019-02-21 08:41:03][_][INFO] Matched: POST /identity/connect/token (login)
[2019-02-21 08:41:08][bitwarden_rs::error][ERROR] Error saving device.
[CAUSE] DatabaseError(
    __Unknown,
    "database disk image is malformed"
)

@Gilneus commented on GitHub: I took a short view in the docker logs and found this: ``` [2019-02-21 08:40:54][rocket::rocket][INFO] POST /api/accounts/prelogin application/json; charset=utf-8: [2019-02-21 08:40:54][_][INFO] Matched: POST /api/accounts/prelogin (prelogin) [2019-02-21 08:40:54][_][INFO] Outcome: Success [2019-02-21 08:40:54][_][INFO] Response succeeded. [2019-02-21 08:40:54][rocket::rocket][INFO] POST /identity/connect/token application/x-www-form-urlencoded; charset=utf-8: [2019-02-21 08:40:54][_][INFO] Matched: POST /identity/connect/token (login) [2019-02-21 08:40:55][bitwarden_rs::error][ERROR] JsonError. {"TwoFactorProviders":[0],"TwoFactorProviders2":{"0":null},"error":"invalid_grant","error_description":"Two factor required."} [2019-02-21 08:40:55][_][INFO] Outcome: Success [2019-02-21 08:40:55][_][INFO] Response succeeded. [2019-02-21 08:41:03][rocket::rocket][INFO] POST /identity/connect/token application/x-www-form-urlencoded; charset=utf-8: [2019-02-21 08:41:03][_][INFO] Matched: POST /identity/connect/token (login) [2019-02-21 08:41:08][bitwarden_rs::error][ERROR] Error saving device. [CAUSE] DatabaseError( __Unknown, "database disk image is malformed" ) ```
Author
Owner

@mprasil commented on GitHub:

That sounds as if the db file got corrupted. This is quite weird as sqlite tends to be quite resilient. Do you have any filesystem issues?

@mprasil commented on GitHub: That sounds as if the db file got corrupted. This is quite weird as sqlite tends to be quite resilient. Do you have any filesystem issues?
Author
Owner

@Gilneus commented on GitHub:

I use an efs (nfs) under AWS to persit the file. I can imagine that access by two servers (autoscaling 2 instances of bitwarden) damaged the database file. Is there a way to recover the sqlite database

@Gilneus commented on GitHub: I use an efs (nfs) under AWS to persit the file. I can imagine that access by two servers (autoscaling 2 instances of bitwarden) damaged the database file. Is there a way to recover the sqlite database
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/vaultwarden#2182