API register route doesn't seems to work - Docker Swarm && Traefik #1009

Closed
opened 2025-10-09 17:00:42 +03:00 by OVERLORD · 1 comment
Owner

Originally created by @babson4 on GitHub.

Subject of the issue

We are using Vaultwarden in university project. The following docker-compose file is running on a docker swarm stack composed by 3 nodes : public, admin and prod.
I'm writing this issue because we have a problem with the stack. In fact, we are not able to create account, when the api is requested on register, the request does not succeed. But the login API route does.

Deployment environment

  • docker with swarm mode

  • behind treaefik reverse proxy

  • Server Server Installed : 1.26.0

  • Server Latest : 1.26.0

  • Web Installed : 2022.10.0

  • Database : SQLite: 3.35.4

  • Clients used: web vault, desktop, Android

  • Reverse proxy and version: Traefik latest

docker-compose.yml for docker stack

services:
  traefik:
    image: traefik:latest
    networks:
      - proxy
    ports:
    # expose port to the host, skip the swarm mesh
      - target: 80
        published: 80
        protocol: tcp
        mode: host
      - target: 443
        published: 443
        protocol: tcp
        mode: host

    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
      - /home/user/swarm/certificates:/certificates
      - /home/user/swarm/logs:/logs
    command:
      # Docker conf
      - --providers.docker
      - --providers.docker.exposedbydefault=false
      - --providers.docker.swarmmode=true
      - --providers.docker.network=proxy
      # entrypoints 
      - --entrypoints.web.address=:80
      - --entrypoints.websecure.address=:443
      - --entrypoints.mysql.address=:3306
      # Acme https
      - --certificatesresolvers.le_resolver.acme.email=web@domain.fr
      - --certificatesresolvers.le_resolver.acme.storage=/certificates/acme.json
      - --certificatesresolvers.le_resolver.acme.tlschallenge=true
      # global HTTP to HTTPS
      - --entrypoints.web.http.redirections.entrypoint.to=websecure
      - --entrypoints.web.http.redirections.entrypoint.scheme=https
      - --entrypoints.web.http.redirections.entrypoint.permanent=true
      # traefik accesslog
      - --accesslog
      - --accesslog.filepath=/logs/traefik_access.log
      - --accesslog.filters.minduration=1ms
      # traefik log
      - --log.filePath=/logs/traefik.log
      - --log.format=json
      - --log.level=DEBUG
      # API && dashboard
      #- --api
      - --api.dashboard=true
    deploy:
      placement:
        constraints:
          # Make the traefik service run only on the node with this label
          - node.labels.env == public
      labels:
        - traefik.enable=true
        - traefik.http.middlewares.need_auth.basicauth.users=user:passEncoded
        # HTTPS router
        - traefik.http.routers.rtr-traefik.rule=Host(`traefik.domain.fr`)
        - traefik.http.routers.rtr-traefik.entrypoints=websecure
        - traefik.http.routers.rtr-traefik.tls=true
        - traefik.http.routers.rtr-traefik.tls.certresolver=le_resolver
        # Use the special Traefik service api@internal with the web UI/Dashboard
        - traefik.http.routers.rtr-traefik.service=api@internal
        # Enable HTTP Basic auth from middleware
        - traefik.http.routers.rtr-traefik.middlewares=need_auth
        # Define the port inside of the Docker service to use
        - traefik.http.services.traefik.loadbalancer.server.port=8080  

  bitwarden:
    image: vaultwarden/server:latest
    networks:
      - proxy
    volumes:
      - /docker/apps/bw-data:/data
    environment:
        - ADMIN_TOKEN=AdminPass
        - WEBSOCKET_ENABLED=true
        - DOMAIN=https://bw.domain.fr
        - EXTENDED_LOGGING=true
        - LOG_FILE=/data/vaultwarden.log
        - LOG_LEVEL=TRACE
        - ADMIN_RATELIMIT_MAX_BURST=3
        - SIGNUPS_ALLOWED=true
    deploy:
      placement:
        constraints:
          - node.labels.env == prod
      
      labels:
      - traefik.enable=true

      # # bitwarden-ui
      - traefik.http.routers.bitwarden-ui-https.rule=Host(`bw.domain.fr`)
      - traefik.http.routers.bitwarden-ui-https.entrypoints=websecure
      - traefik.http.routers.bitwarden-ui-https.tls=true
      - traefik.http.routers.bitwarden-ui-https.service=bitwarden-ui
      - traefik.http.routers.bitwarden-ui-https.tls.certresolver=leresolver

      # WebSocket      
      - traefik.http.routers.bitwarden-websocket-https.rule=Host(`bw.domain.fr`) && Path(`/notifications/hub/`)
      - traefik.http.routers.bitwarden-websocket-https.entrypoints=websecure
      - traefik.http.routers.bitwarden-websocket-https.tls=true
      - traefik.http.routers.bitwarden-websocket-https.tls.certresolver=leresolver
      - traefik.http.routers.bitwarden-websocket-https.service=bitwarden-websocket
      
      # balancers
      - traefik.http.services.bitwarden-ui.loadbalancer.server.port=80      
      - traefik.http.services.bitwarden-websocket.loadbalancer.server.port=3012

Steps to reproduce

  • deploy the stack with lastest version behind traefik with docker on swarm mode.
  • ¯_(ツ)_/¯ cry.

Expected behaviour / Actual behaviour

We cannot create account, the api seems to not work on the register route.
We tried:

  • Change database for Mysql --> changed nothing
  • Invite user with mail --> changed nothing
  • Use older images --> changed nothing
  • Tried to add Host(api.domain.fr) in traefik for the service --> changed nothing

If we tried requesting the API route bw.domain.fr/api/accounts/register with Postman we got 400 (malformed request) with no payload and NO response with a payload.

Troubleshooting data

Trace of the bitwarden service from the swarm cluster in case of connection

Note that everything is OK for the connection API and respond that the user does not exist. Because we can't create one.

traefik_bitwarden.1.e5dxgnraolhb@vm0    | [2022-11-14 11:18:36.929][request][INFO] POST /api/accounts/prelogin
traefik_bitwarden.1.e5dxgnraolhb@vm0    | [2022-11-14 11:18:36.966][response][INFO] (prelogin) POST /api/accounts/prelogin => 200 OK
traefik_bitwarden.1.e5dxgnraolhb@vm0    | [2022-11-14 11:18:36.967][tracing::span][TRACE] encode_headers;
traefik_bitwarden.1.e5dxgnraolhb@vm0    | [2022-11-14 11:18:36.967][tracing::span::active][TRACE] -> encode_headers;
traefik_bitwarden.1.e5dxgnraolhb@vm0    | [2022-11-14 11:18:36.967][tracing::span::active][TRACE] <- encode_headers;
traefik_bitwarden.1.e5dxgnraolhb@vm0    | [2022-11-14 11:18:36.967][tracing::span][TRACE] -- encode_headers;
traefik_bitwarden.1.e5dxgnraolhb@vm0    | [2022-11-14 11:18:37.354][tracing::span][TRACE] parse_headers;
traefik_bitwarden.1.e5dxgnraolhb@vm0    | [2022-11-14 11:18:37.355][tracing::span::active][TRACE] -> parse_headers;
traefik_bitwarden.1.e5dxgnraolhb@vm0    | [2022-11-14 11:18:37.355][tracing::span::active][TRACE] <- parse_headers;
traefik_bitwarden.1.e5dxgnraolhb@vm0    | [2022-11-14 11:18:37.355][tracing::span][TRACE] -- parse_headers;
traefik_bitwarden.1.e5dxgnraolhb@vm0    | [2022-11-14 11:18:37.356][request][INFO] POST /identity/connect/token
traefik_bitwarden.1.e5dxgnraolhb@vm0    | [2022-11-14 11:18:37.381][vaultwarden::api::identity][ERROR] Username or password is incorrect. Try again. IP: 192.168.224.1. Username: user@gmail.com.
traefik_bitwarden.1.e5dxgnraolhb@vm0    | [2022-11-14 11:18:37.382][response][INFO] (login) POST /identity/connect/token => 400 Bad Request
traefik_bitwarden.1.e5dxgnraolhb@vm0    | [2022-11-14 11:18:37.382][tracing::span][TRACE] encode_headers;
traefik_bitwarden.1.e5dxgnraolhb@vm0    | [2022-11-14 11:18:37.382][tracing::span::active][TRACE] -> encode_headers;
traefik_bitwarden.1.e5dxgnraolhb@vm0    | [2022-11-14 11:18:37.383][tracing::span::active][TRACE] <- encode_headers;
traefik_bitwarden.1.e5dxgnraolhb@vm0    | [2022-11-14 11:18:37.383][tracing::span][TRACE] -- encode_headers;
Originally created by @babson4 on GitHub. ## Subject of the issue We are using Vaultwarden in university project. The following docker-compose file is running on a docker swarm stack composed by 3 nodes : public, admin and prod. I'm writing this issue because we have a problem with the stack. In fact, we are not able to create account, when the api is requested on register, the request does not succeed. But the login API route does. ## Deployment environment * docker with swarm mode * behind treaefik reverse proxy * Server Server Installed : 1.26.0 * Server Latest : 1.26.0 * Web Installed : 2022.10.0 * Database : SQLite: 3.35.4 * Clients used: web vault, desktop, Android * Reverse proxy and version: Traefik latest ### docker-compose.yml for docker stack ```yml services: traefik: image: traefik:latest networks: - proxy ports: # expose port to the host, skip the swarm mesh - target: 80 published: 80 protocol: tcp mode: host - target: 443 published: 443 protocol: tcp mode: host volumes: - /var/run/docker.sock:/var/run/docker.sock:ro - /home/user/swarm/certificates:/certificates - /home/user/swarm/logs:/logs command: # Docker conf - --providers.docker - --providers.docker.exposedbydefault=false - --providers.docker.swarmmode=true - --providers.docker.network=proxy # entrypoints - --entrypoints.web.address=:80 - --entrypoints.websecure.address=:443 - --entrypoints.mysql.address=:3306 # Acme https - --certificatesresolvers.le_resolver.acme.email=web@domain.fr - --certificatesresolvers.le_resolver.acme.storage=/certificates/acme.json - --certificatesresolvers.le_resolver.acme.tlschallenge=true # global HTTP to HTTPS - --entrypoints.web.http.redirections.entrypoint.to=websecure - --entrypoints.web.http.redirections.entrypoint.scheme=https - --entrypoints.web.http.redirections.entrypoint.permanent=true # traefik accesslog - --accesslog - --accesslog.filepath=/logs/traefik_access.log - --accesslog.filters.minduration=1ms # traefik log - --log.filePath=/logs/traefik.log - --log.format=json - --log.level=DEBUG # API && dashboard #- --api - --api.dashboard=true deploy: placement: constraints: # Make the traefik service run only on the node with this label - node.labels.env == public labels: - traefik.enable=true - traefik.http.middlewares.need_auth.basicauth.users=user:passEncoded # HTTPS router - traefik.http.routers.rtr-traefik.rule=Host(`traefik.domain.fr`) - traefik.http.routers.rtr-traefik.entrypoints=websecure - traefik.http.routers.rtr-traefik.tls=true - traefik.http.routers.rtr-traefik.tls.certresolver=le_resolver # Use the special Traefik service api@internal with the web UI/Dashboard - traefik.http.routers.rtr-traefik.service=api@internal # Enable HTTP Basic auth from middleware - traefik.http.routers.rtr-traefik.middlewares=need_auth # Define the port inside of the Docker service to use - traefik.http.services.traefik.loadbalancer.server.port=8080 bitwarden: image: vaultwarden/server:latest networks: - proxy volumes: - /docker/apps/bw-data:/data environment: - ADMIN_TOKEN=AdminPass - WEBSOCKET_ENABLED=true - DOMAIN=https://bw.domain.fr - EXTENDED_LOGGING=true - LOG_FILE=/data/vaultwarden.log - LOG_LEVEL=TRACE - ADMIN_RATELIMIT_MAX_BURST=3 - SIGNUPS_ALLOWED=true deploy: placement: constraints: - node.labels.env == prod labels: - traefik.enable=true # # bitwarden-ui - traefik.http.routers.bitwarden-ui-https.rule=Host(`bw.domain.fr`) - traefik.http.routers.bitwarden-ui-https.entrypoints=websecure - traefik.http.routers.bitwarden-ui-https.tls=true - traefik.http.routers.bitwarden-ui-https.service=bitwarden-ui - traefik.http.routers.bitwarden-ui-https.tls.certresolver=leresolver # WebSocket - traefik.http.routers.bitwarden-websocket-https.rule=Host(`bw.domain.fr`) && Path(`/notifications/hub/`) - traefik.http.routers.bitwarden-websocket-https.entrypoints=websecure - traefik.http.routers.bitwarden-websocket-https.tls=true - traefik.http.routers.bitwarden-websocket-https.tls.certresolver=leresolver - traefik.http.routers.bitwarden-websocket-https.service=bitwarden-websocket # balancers - traefik.http.services.bitwarden-ui.loadbalancer.server.port=80 - traefik.http.services.bitwarden-websocket.loadbalancer.server.port=3012 ``` ## Steps to reproduce * deploy the stack with lastest version behind traefik with docker on swarm mode. * ¯\_(ツ)_/¯ cry. ## Expected behaviour / Actual behaviour We cannot create account, the api seems to not work on the register route. We tried: * Change database for Mysql --> changed nothing * Invite user with mail --> changed nothing * Use older images --> changed nothing * Tried to add Host(`api.domain.fr`) in traefik for the service --> changed nothing If we tried requesting the API route `bw.domain.fr/api/accounts/register` with Postman we got 400 (malformed request) with no payload and NO response with a payload. ## Troubleshooting data ### Trace of the bitwarden service from the swarm cluster in case of connection > Note that everything is OK for the connection API and respond that the user does not exist. Because we can't create one. ``` traefik_bitwarden.1.e5dxgnraolhb@vm0 | [2022-11-14 11:18:36.929][request][INFO] POST /api/accounts/prelogin traefik_bitwarden.1.e5dxgnraolhb@vm0 | [2022-11-14 11:18:36.966][response][INFO] (prelogin) POST /api/accounts/prelogin => 200 OK traefik_bitwarden.1.e5dxgnraolhb@vm0 | [2022-11-14 11:18:36.967][tracing::span][TRACE] encode_headers; traefik_bitwarden.1.e5dxgnraolhb@vm0 | [2022-11-14 11:18:36.967][tracing::span::active][TRACE] -> encode_headers; traefik_bitwarden.1.e5dxgnraolhb@vm0 | [2022-11-14 11:18:36.967][tracing::span::active][TRACE] <- encode_headers; traefik_bitwarden.1.e5dxgnraolhb@vm0 | [2022-11-14 11:18:36.967][tracing::span][TRACE] -- encode_headers; traefik_bitwarden.1.e5dxgnraolhb@vm0 | [2022-11-14 11:18:37.354][tracing::span][TRACE] parse_headers; traefik_bitwarden.1.e5dxgnraolhb@vm0 | [2022-11-14 11:18:37.355][tracing::span::active][TRACE] -> parse_headers; traefik_bitwarden.1.e5dxgnraolhb@vm0 | [2022-11-14 11:18:37.355][tracing::span::active][TRACE] <- parse_headers; traefik_bitwarden.1.e5dxgnraolhb@vm0 | [2022-11-14 11:18:37.355][tracing::span][TRACE] -- parse_headers; traefik_bitwarden.1.e5dxgnraolhb@vm0 | [2022-11-14 11:18:37.356][request][INFO] POST /identity/connect/token traefik_bitwarden.1.e5dxgnraolhb@vm0 | [2022-11-14 11:18:37.381][vaultwarden::api::identity][ERROR] Username or password is incorrect. Try again. IP: 192.168.224.1. Username: user@gmail.com. traefik_bitwarden.1.e5dxgnraolhb@vm0 | [2022-11-14 11:18:37.382][response][INFO] (login) POST /identity/connect/token => 400 Bad Request traefik_bitwarden.1.e5dxgnraolhb@vm0 | [2022-11-14 11:18:37.382][tracing::span][TRACE] encode_headers; traefik_bitwarden.1.e5dxgnraolhb@vm0 | [2022-11-14 11:18:37.382][tracing::span::active][TRACE] -> encode_headers; traefik_bitwarden.1.e5dxgnraolhb@vm0 | [2022-11-14 11:18:37.383][tracing::span::active][TRACE] <- encode_headers; traefik_bitwarden.1.e5dxgnraolhb@vm0 | [2022-11-14 11:18:37.383][tracing::span][TRACE] -- encode_headers; ```
Author
Owner

@babson4 commented on GitHub:

See the following schema of our infrastructure, may help

global_schema

@babson4 commented on GitHub: See the following schema of our infrastructure, may help ![global_schema](https://user-images.githubusercontent.com/33559514/201673761-438437b9-78f8-4110-8647-d90cffe56b14.png)
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/vaultwarden#1009