starred/vaultwarden
1
0
Fork 0
mirror of https://github.com/dani-garcia/vaultwarden.git synced 2025-12-10 01:10:09 +03:00
Code Issues 429 Packages Projects Releases 10 Wiki Activity

OpenSSL-related crashes on Docker Alpine image #1176

New Issue
Closed
opened 2025-10-09 17:06:35 +03:00 by OVERLORD · 14 comments
OVERLORD commented 2025-10-09 17:06:35 +03:00
Owner
Copy Link

Originally created by @JeDaYoshi on GitHub.

Subject of the issue

Looking at my vaultwarden logs, I noticed something peculiar in them. Apparently, vaultwarden has had occasional crashes due to OpenSSL errors, which seem pretty random.

[2022-03-19 17:55:36.236][response][INFO] (login) POST /identity/connect/token => 200 OK
[2022-03-19 17:55:36.398][request][INFO] GET /api/sync
[2022-03-19 17:55:36.425][response][INFO] (sync) GET /api/sync?<data..> => 200 OK
crypto/evp/evp_enc.c:510: OpenSSL internal error: assertion failed: b <= sizeof(ctx->final)
/--------------------------------------------------------------------\
|                        Starting Vaultwarden                        |
|                      Version 1.24.0-8d06d9c1                       |
|--------------------------------------------------------------------|

[2022-03-19 18:15:37.503][parity_ws][INFO] Listening for new connections on 0.0.0.0:3012.
[2022-03-19 18:15:37.506][start][INFO] Rocket has launched from http://0.0.0.0:8080

[2022-03-20 16:14:15.683][response][INFO] (login) POST /identity/connect/token => 200 OK
[2022-03-20 16:14:16.022][request][INFO] GET /api/sync
[2022-03-20 16:14:16.048][response][INFO] (sync) GET /api/sync?<data..> => 200 OK
crypto/evp/evp_enc.c:156: OpenSSL internal error: assertion failed: ctx->cipher->block_size == 1 || ctx->cipher->block_size == 8 || ctx->cipher->block_size == 16
/--------------------------------------------------------------------\
|                        Starting Vaultwarden                        |
|                      Version 1.24.0-8d06d9c1                       |
|--------------------------------------------------------------------|

[2022-03-20 16:15:42.751][parity_ws][INFO] Listening for new connections on 0.0.0.0:3012.
[2022-03-20 16:15:42.753][start][INFO] Rocket has launched from http://0.0.0.0:8080

I have not noticed any disruption while using it, but it raises an eye.

Deployment environment

  • Vaultwarden version: v1.24.0-8d06d9c1 (latest vaultwarden/server:testing-alpine, amd64)
  • Web-vault version: v2.26.1
  • Running within Docker: true (Base: Alpine)
  • Environment settings overridden: false
  • Uses a reverse proxy: true
  • IP Header check: true (X-Real-IP)
  • Internet access: true
  • Internet access via a proxy: false
  • DNS Check: true
  • Time Check: true
  • Domain Configuration Check: true
  • HTTPS Check: true
  • Database type: PostgreSQL
  • Database version: PostgreSQL 14.2 (Debian 14.2-1.pgdg110+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 10.2.1-6) 10.2.1 20210110, 64-bit
  • Clients used: Multiple of them, including Web Vault, Desktop and Android apps.
  • Reverse proxy and version: Caddy v2.4.6

Steps to reproduce

Unsure. I have pretty much a "regular" setup, with WEBSOCKET_ENABLED and that's it.

It seems to have happened during a while after regular sync intervals, no entries added or anything.

Originally created by @JeDaYoshi on GitHub. ### Subject of the issue Looking at my vaultwarden logs, I noticed something peculiar in them. Apparently, vaultwarden has had occasional crashes due to OpenSSL errors, which seem pretty random. ``` [2022-03-19 17:55:36.236][response][INFO] (login) POST /identity/connect/token => 200 OK [2022-03-19 17:55:36.398][request][INFO] GET /api/sync [2022-03-19 17:55:36.425][response][INFO] (sync) GET /api/sync?<data..> => 200 OK crypto/evp/evp_enc.c:510: OpenSSL internal error: assertion failed: b <= sizeof(ctx->final) /--------------------------------------------------------------------\ | Starting Vaultwarden | | Version 1.24.0-8d06d9c1 | |--------------------------------------------------------------------| [2022-03-19 18:15:37.503][parity_ws][INFO] Listening for new connections on 0.0.0.0:3012. [2022-03-19 18:15:37.506][start][INFO] Rocket has launched from http://0.0.0.0:8080 ``` ``` [2022-03-20 16:14:15.683][response][INFO] (login) POST /identity/connect/token => 200 OK [2022-03-20 16:14:16.022][request][INFO] GET /api/sync [2022-03-20 16:14:16.048][response][INFO] (sync) GET /api/sync?<data..> => 200 OK crypto/evp/evp_enc.c:156: OpenSSL internal error: assertion failed: ctx->cipher->block_size == 1 || ctx->cipher->block_size == 8 || ctx->cipher->block_size == 16 /--------------------------------------------------------------------\ | Starting Vaultwarden | | Version 1.24.0-8d06d9c1 | |--------------------------------------------------------------------| [2022-03-20 16:15:42.751][parity_ws][INFO] Listening for new connections on 0.0.0.0:3012. [2022-03-20 16:15:42.753][start][INFO] Rocket has launched from http://0.0.0.0:8080 ``` I have not noticed any disruption while using it, but it raises an eye. ### Deployment environment * Vaultwarden version: v1.24.0-8d06d9c1 (latest `vaultwarden/server:testing-alpine`, `amd64`) * Web-vault version: v2.26.1 * Running within Docker: true (Base: Alpine) * Environment settings overridden: false * Uses a reverse proxy: true * IP Header check: true (X-Real-IP) * Internet access: true * Internet access via a proxy: false * DNS Check: true * Time Check: true * Domain Configuration Check: true * HTTPS Check: true * Database type: PostgreSQL * Database version: PostgreSQL 14.2 (Debian 14.2-1.pgdg110+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 10.2.1-6) 10.2.1 20210110, 64-bit * Clients used: Multiple of them, including Web Vault, Desktop and Android apps. * Reverse proxy and version: Caddy v2.4.6 ### Steps to reproduce Unsure. I have pretty much a "regular" setup, with `WEBSOCKET_ENABLED` and that's it. It seems to have happened ~~during~~ a while after regular sync intervals, no entries added or anything.
OVERLORD commented 2025-10-09 17:06:37 +03:00
Author
Owner
Copy Link

@JeDaYoshi commented on GitHub:

I don't have TLS enabled on Vaultwarden. It's served via HTTP to Caddy.

I am using testing-alpine already, as pointed out in the original issue. I don't recall seeing this issue in the latest stable version, but I can't give out any confirmations right now.

@JeDaYoshi commented on GitHub: I don't have TLS enabled on Vaultwarden. It's served via HTTP to Caddy. I am using `testing-alpine` already, as pointed out in the original issue. I don't recall seeing this issue in the latest stable version, but I can't give out any confirmations right now.
OVERLORD commented 2025-10-09 17:06:37 +03:00
Author
Owner
Copy Link

@BlackDex commented on GitHub:

It almost looks like you also have Vaultwarden configured to serve HTTPS to Caddy.

The SSL implementation for the Rocket version used is a bit unstable sometimes. That is why we suggest to use a reverse proxy and let it offload the SSL also.

I'm not sure if you have it configured like that, but if that is the case i would suggest two things to try

  1. Disable SSL on the Vaultwarden side and let it be handled by Caddy.
  2. Try the testing-alpine version and see if that helps. That version has a newer library.
@BlackDex commented on GitHub: It almost looks like you also have Vaultwarden configured to serve HTTPS to Caddy. The SSL implementation for the Rocket version used is a bit unstable sometimes. That is why we suggest to use a reverse proxy and let it offload the SSL also. I'm not sure if you have it configured like that, but if that is the case i would suggest two things to try 1. Disable SSL on the Vaultwarden side and let it be handled by Caddy. 2. Try the `testing-alpine` version and see if that helps. That version has a newer library.
OVERLORD commented 2025-10-09 17:06:37 +03:00
Author
Owner
Copy Link

@JeDaYoshi commented on GitHub:

Yep.

postgres=# SELECT * FROM pg_stat_ssl;
  pid   | ssl | version |         cipher         | bits | client_dn | client_serial | issuer_dn 
--------+-----+---------+------------------------+------+-----------+---------------+-----------
 186528 | t   | TLSv1.3 | TLS_AES_256_GCM_SHA384 |  256 |           |               | 
 186526 | t   | TLSv1.3 | TLS_AES_256_GCM_SHA384 |  256 |           |               | 
 186527 | t   | TLSv1.3 | TLS_AES_256_GCM_SHA384 |  256 |           |               | 
 186529 | t   | TLSv1.3 | TLS_AES_256_GCM_SHA384 |  256 |           |               | 
 186530 | t   | TLSv1.3 | TLS_AES_256_GCM_SHA384 |  256 |           |               | 
 186531 | t   | TLSv1.3 | TLS_AES_256_GCM_SHA384 |  256 |           |               | 
 186532 | t   | TLSv1.3 | TLS_AES_256_GCM_SHA384 |  256 |           |               | 
 186533 | t   | TLSv1.3 | TLS_AES_256_GCM_SHA384 |  256 |           |               | 
 186534 | t   | TLSv1.3 | TLS_AES_256_GCM_SHA384 |  256 |           |               | 
 186535 | t   | TLSv1.3 | TLS_AES_256_GCM_SHA384 |  256 |           |               | 

postgres  186526  0.0  0.3 357448 16032 ?        Ss   19:15   0:00 postgres: 14/main: vaultwarden vaultwarden 172.18.0.2(39474) idle
postgres  186527  0.0  0.3 357448 16032 ?        Ss   19:15   0:00 postgres: 14/main: vaultwarden vaultwarden 172.18.0.2(39476) idle
postgres  186528  0.0  0.3 357448 16036 ?        Ss   19:15   0:00 postgres: 14/main: vaultwarden vaultwarden 172.18.0.2(39478) idle
postgres  186529  0.0  0.3 357448 16032 ?        Ss   19:15   0:00 postgres: 14/main: vaultwarden vaultwarden 172.18.0.2(39480) idle
postgres  186530  0.0  0.3 357448 16036 ?        Ss   19:15   0:00 postgres: 14/main: vaultwarden vaultwarden 172.18.0.2(39482) idle
postgres  186531  0.0  0.3 357448 16032 ?        Ss   19:15   0:00 postgres: 14/main: vaultwarden vaultwarden 172.18.0.2(39484) idle
postgres  186532  0.0  0.3 357448 16036 ?        Ss   19:15   0:00 postgres: 14/main: vaultwarden vaultwarden 172.18.0.2(39486) idle
postgres  186533  0.0  0.3 357448 16032 ?        Ss   19:15   0:00 postgres: 14/main: vaultwarden vaultwarden 172.18.0.2(39488) idle
postgres  186534  0.0  0.5 359044 21256 ?        Ss   19:15   0:00 postgres: 14/main: vaultwarden vaultwarden 172.18.0.2(39490) idle
postgres  186535  0.0  0.5 359376 22384 ?        Ss   19:15   0:00 postgres: 14/main: vaultwarden vaultwarden 172.18.0.2(39492) idle
@JeDaYoshi commented on GitHub: Yep. ``` postgres=# SELECT * FROM pg_stat_ssl; pid | ssl | version | cipher | bits | client_dn | client_serial | issuer_dn --------+-----+---------+------------------------+------+-----------+---------------+----------- 186528 | t | TLSv1.3 | TLS_AES_256_GCM_SHA384 | 256 | | | 186526 | t | TLSv1.3 | TLS_AES_256_GCM_SHA384 | 256 | | | 186527 | t | TLSv1.3 | TLS_AES_256_GCM_SHA384 | 256 | | | 186529 | t | TLSv1.3 | TLS_AES_256_GCM_SHA384 | 256 | | | 186530 | t | TLSv1.3 | TLS_AES_256_GCM_SHA384 | 256 | | | 186531 | t | TLSv1.3 | TLS_AES_256_GCM_SHA384 | 256 | | | 186532 | t | TLSv1.3 | TLS_AES_256_GCM_SHA384 | 256 | | | 186533 | t | TLSv1.3 | TLS_AES_256_GCM_SHA384 | 256 | | | 186534 | t | TLSv1.3 | TLS_AES_256_GCM_SHA384 | 256 | | | 186535 | t | TLSv1.3 | TLS_AES_256_GCM_SHA384 | 256 | | | ``` ``` postgres 186526 0.0 0.3 357448 16032 ? Ss 19:15 0:00 postgres: 14/main: vaultwarden vaultwarden 172.18.0.2(39474) idle postgres 186527 0.0 0.3 357448 16032 ? Ss 19:15 0:00 postgres: 14/main: vaultwarden vaultwarden 172.18.0.2(39476) idle postgres 186528 0.0 0.3 357448 16036 ? Ss 19:15 0:00 postgres: 14/main: vaultwarden vaultwarden 172.18.0.2(39478) idle postgres 186529 0.0 0.3 357448 16032 ? Ss 19:15 0:00 postgres: 14/main: vaultwarden vaultwarden 172.18.0.2(39480) idle postgres 186530 0.0 0.3 357448 16036 ? Ss 19:15 0:00 postgres: 14/main: vaultwarden vaultwarden 172.18.0.2(39482) idle postgres 186531 0.0 0.3 357448 16032 ? Ss 19:15 0:00 postgres: 14/main: vaultwarden vaultwarden 172.18.0.2(39484) idle postgres 186532 0.0 0.3 357448 16036 ? Ss 19:15 0:00 postgres: 14/main: vaultwarden vaultwarden 172.18.0.2(39486) idle postgres 186533 0.0 0.3 357448 16032 ? Ss 19:15 0:00 postgres: 14/main: vaultwarden vaultwarden 172.18.0.2(39488) idle postgres 186534 0.0 0.5 359044 21256 ? Ss 19:15 0:00 postgres: 14/main: vaultwarden vaultwarden 172.18.0.2(39490) idle postgres 186535 0.0 0.5 359376 22384 ? Ss 19:15 0:00 postgres: 14/main: vaultwarden vaultwarden 172.18.0.2(39492) idle ```
OVERLORD commented 2025-10-09 17:06:38 +03:00
Author
Owner
Copy Link

@BlackDex commented on GitHub:

Ah indeed. Sorry late here already hehe.
Is the postgresql connection secured with ssl?
That would be the only item left which i can think of.

@BlackDex commented on GitHub: Ah indeed. Sorry late here already hehe. Is the postgresql connection secured with ssl? That would be the only item left which i can think of.
OVERLORD commented 2025-10-09 17:06:38 +03:00
Author
Owner
Copy Link

@JeDaYoshi commented on GitHub:

Actually, good point. Isn't PostgreSQL secured with TLS by default?
That could possibly be it. Would also explain why it's not exactly when a HTTP request is received.

@JeDaYoshi commented on GitHub: Actually, good point. Isn't PostgreSQL secured with TLS by default? That could possibly be it. Would also explain why it's not exactly when a HTTP request is received.
OVERLORD commented 2025-10-09 17:06:39 +03:00
Author
Owner
Copy Link

@JeDaYoshi commented on GitHub:

If needed, i could build a version which uses PostgreSQL v14 btw instead of PQ11

Let's see if it crashes again, and if so, I'll ask for it. You can build it anyway if it's no issue.

Looking good so far, though.

@JeDaYoshi commented on GitHub: >If needed, i could build a version which uses PostgreSQL v14 btw instead of PQ11 Let's see if it crashes again, and if so, I'll ask for it. You can build it anyway if it's no issue. Looking good so far, though.
OVERLORD commented 2025-10-09 17:06:39 +03:00
Author
Owner
Copy Link

@JeDaYoshi commented on GitHub:

I'm gonna try adding ?sslmode=disable to the PostgreSQL URI and see if it still crashes. Shouldn't matter that much since they're in the same server.

Will report back with results.

@JeDaYoshi commented on GitHub: I'm gonna try adding `?sslmode=disable` to the PostgreSQL URI and see if it still crashes. Shouldn't matter that much since they're in the same server. Will report back with results.
OVERLORD commented 2025-10-09 17:06:40 +03:00
Author
Owner
Copy Link

@BlackDex commented on GitHub:

Its either that, or there is a weird issue with the client which is still using a PostgreSQL v11 library.

The library does indeed first tries ssl, if that fails it tries without.

@BlackDex commented on GitHub: Its either that, or there is a weird issue with the client which is still using a PostgreSQL v11 library. The library does indeed first tries ssl, if that fails it tries without.
OVERLORD commented 2025-10-09 17:06:40 +03:00
Author
Owner
Copy Link

@BlackDex commented on GitHub:

I tried this my self using a postgre docker container with a self-signed ssl-cert.
I'm not seeing any issues so far.

I created a personal vault with 9000+ items in it and a org vault with 2200+ items.
It seems to keep working.

Looking the the version string of your PostgreSQL Instance, you are probably using the exact same container.
I'm not sure which parameters you used to start it, but i used the following:

docker run --rm --name vw-postgres \
  -e POSTGRES_PASSWORD=MySecretPassword \
  -e POSTGRES_USER=vaultwarden_user \
  -e POSTGRES_DB=vaultwarden_db \
  -v "$(pwd)/server.crt:/server.crt:ro" \
  -v "$(pwd)/server.key:/server.key:ro" \
  -p5432:5432 \
  postgres:14.2-bullseye \
  -c ssl=on -c ssl_cert_file=/server.crt -c ssl_key_file=/server.key

And i started the Vaultwarden container like this:

docker run --rm --name vaultwarden \
  -e RUST_BACKTRACE=full \
  -e LOG_LEVEL=debug \
  -e DISABLE_ADMIN_TOKEN=true \
  -e DATABASE_URL="postgresql://vaultwarden_user:MySecretPassword@x.x.x.x:5432/vaultwarden_db?sslmode=require" \
  -p8181:80 \
  vaultwarden/server:testing-alpine

I then used the Bitwarden Desktop, CLI and Browser extension to login to the instance and add/sync items.
I left it running for a long time and did some more testing and no issues as of yet.

Not sure what could be the issue on your side then though.
Have you tried to restart the PostgreSQL instance/server?

@BlackDex commented on GitHub: I tried this my self using a postgre docker container with a self-signed ssl-cert. I'm not seeing any issues so far. I created a personal vault with 9000+ items in it and a org vault with 2200+ items. It seems to keep working. Looking the the version string of your PostgreSQL Instance, you are probably using the exact same container. I'm not sure which parameters you used to start it, but i used the following: ```bash docker run --rm --name vw-postgres \ -e POSTGRES_PASSWORD=MySecretPassword \ -e POSTGRES_USER=vaultwarden_user \ -e POSTGRES_DB=vaultwarden_db \ -v "$(pwd)/server.crt:/server.crt:ro" \ -v "$(pwd)/server.key:/server.key:ro" \ -p5432:5432 \ postgres:14.2-bullseye \ -c ssl=on -c ssl_cert_file=/server.crt -c ssl_key_file=/server.key ``` And i started the Vaultwarden container like this: ```bash docker run --rm --name vaultwarden \ -e RUST_BACKTRACE=full \ -e LOG_LEVEL=debug \ -e DISABLE_ADMIN_TOKEN=true \ -e DATABASE_URL="postgresql://vaultwarden_user:MySecretPassword@x.x.x.x:5432/vaultwarden_db?sslmode=require" \ -p8181:80 \ vaultwarden/server:testing-alpine ``` I then used the Bitwarden Desktop, CLI and Browser extension to login to the instance and add/sync items. I left it running for a long time and did some more testing and no issues as of yet. Not sure what could be the issue on your side then though. Have you tried to restart the PostgreSQL instance/server?
OVERLORD commented 2025-10-09 17:06:40 +03:00
Author
Owner
Copy Link

@BlackDex commented on GitHub:

When a newer version of the testing will be released it will be using a newer version of OpenSSL. Maybe that helps, but i doubt it actually. But who knows.

But please keep us posted.

Edit:
If needed, i could build a version which uses PostgreSQL v14 btw instead of PQ11

@BlackDex commented on GitHub: When a newer version of the testing will be released it will be using a newer version of OpenSSL. Maybe that helps, but i doubt it actually. But who knows. But please keep us posted. Edit: If needed, i could build a version which uses PostgreSQL v14 btw instead of PQ11
OVERLORD commented 2025-10-09 17:06:41 +03:00
Author
Owner
Copy Link

@JeDaYoshi commented on GitHub:

I restarted it before creating this issue.

So far, it has still been working fine. No crashes or any odd errors. I'll try switching back to TLS.

@JeDaYoshi commented on GitHub: I restarted it before creating this issue. So far, it has still been working fine. No crashes or any odd errors. I'll try switching back to TLS.
OVERLORD commented 2025-10-09 17:06:42 +03:00
Author
Owner
Copy Link

@JeDaYoshi commented on GitHub:

...And nothing.
It's just.. working.

Software.

@JeDaYoshi commented on GitHub: ...And nothing. It's just.. working. Software.
OVERLORD commented 2025-10-09 17:06:42 +03:00
Author
Owner
Copy Link

@BlackDex commented on GitHub:

Still strange.
I can understand that you would prefer a secure db connection.

I dived a bit into the specific errors, the only thing i can see is that it could be some weird OpenSSL version difference somewhere. The Alpine image has a statically linked OpenSSL which could differ a bit from the OpenSSL installed by Alpine, but i'm not seeing how those two would even interact, since the vaultwarden application shouldn't be using the OS libraries at all.

Maybe a newer build of the Alpine image would solve this, since they then probably both use OpenSSL v1.1.1n on all levels.

@BlackDex commented on GitHub: Still strange. I can understand that you would prefer a secure db connection. I dived a bit into the specific errors, the only thing i can see is that it could be some weird OpenSSL version difference somewhere. The Alpine image has a statically linked OpenSSL which could differ a bit from the OpenSSL installed by Alpine, but i'm not seeing how those two would even interact, since the vaultwarden application shouldn't be using the OS libraries at all. Maybe a newer build of the Alpine image would solve this, since they then probably both use OpenSSL v1.1.1n on all levels.
OVERLORD commented 2025-10-09 17:06:42 +03:00
Author
Owner
Copy Link

@JeDaYoshi commented on GitHub:

I have no idea what's happening in here either, but yeah, it's functional..

Probably a newer build would be fine. The crashes are indeed strange.

@JeDaYoshi commented on GitHub: I have no idea what's happening in here either, but yeah, it's functional.. Probably a newer build would be fine. The crashes are indeed strange.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
better for forum
bug
bug
bug
documentation
duplicate
enhancement
future Vault
good first issue
help wanted
low priority
notes
pull-request
Mirrored from GitHub Pull Request
 question
SSO
Third party
troubleshooting
troubleshooting
wontfix
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
OVERLORD
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/vaultwarden#1176
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?