No Internet in Diagnostics and Outgoing connections fail #1724

Closed
opened 2026-02-05 01:35:12 +03:00 by OVERLORD · 5 comments
Owner

Originally created by @knightian on GitHub (Oct 1, 2023).

image

Websockets are working fine, everything seems to work except for anything that vaultwarden needs to reach out to like version checks, NTP and push notifications.

In the log there is error setting up push:

[2023-10-02 00:18:49.867][vaultwarden::api::push][ERROR] Error getting push token from bitwarden server: error sending request for url (https://identity.bitwarden.com/connect/token): error trying to connect: dns error: No connections available
[2023-10-02 00:18:49.868][vaultwarden::api::core::accounts][ERROR] An error occured while proceeding registration of a device: {"ErrorModel":{"Message":"Error getting push token from bitwarden server: error sending request for url (https://identity.bitwarden.com/connect/token): error trying to connect: dns error: No connections available","Object":"error"},"ExceptionMessage":null,"ExceptionStackTrace":null,"InnerExceptionMessage":null,"Message":"Error getting push token from bitwarden server: error sending request for url (https://identity.bitwarden.com/connect/token): error trying to connect: dns error: No connections available","Object":"error","ValidationErrors":{"":["Error getting push token from bitwarden server: error sending request for url (https://identity.bitwarden.com/connect/token): error trying to connect: dns error: No connections available"]},"error":"","error_description":""}
[2023-10-02 00:18:49.868][response][INFO] (put_device_token) PUT /api/devices/identifier/<uuid>/token => 400 Bad Request

I can curl https://identity.bitwarden.com/connect/token from the server and it reaches it without issue. This all used to work fine it is just the last few Bitwarden versions this seems to happen, never had any issues in the past.

SMTP reaches out no worries to send emails through Microsoft 365.

Using nginx as reverse proxy, was using my own config but I have switched to the nginx config provided by @BlackDex in the proxy examples.

FWIW I also see this in the vaultwarden log:
[2023-10-02 00:59:53.441][trust_dns_resolver::system_conf::unix][WARN] no nameservers found in config

But DNS is working fine on the server.

I suspect this is going to be a DNS issue within Vaultwarden, but why does it only pop up in current versions did something change?

Here is the debug string:

Your environment (Generated via diagnostics page)

  • Vaultwarden version: v1.29.2-bc26bfa5
  • Web-vault version: v2023.8.2
  • OS/Arch: linux/x86_64
  • Running within Docker: false (Base: Not applicable)
  • Environment settings overridden: false
  • Uses a reverse proxy: true
  • IP Header check: true (X-Real-IP)
  • Internet access: false
  • Internet access via a proxy: false
  • DNS Check: true
  • Browser/Server Time Check: true
  • Server/NTP Time Check: n/a
  • Domain Configuration Check: true
  • HTTPS Check: true
  • Database type: SQLite
  • Database version: 3.41.2
  • Clients used:
  • Reverse proxy and version:
  • Other relevant information:

Config (Generated via diagnostics page)

Show Running Config

Environment settings which are overridden:

{
  "_duo_akey": null,
  "_enable_duo": false,
  "_enable_email_2fa": true,
  "_enable_smtp": true,
  "_enable_yubico": true,
  "_icon_service_csp": "",
  "_icon_service_url": "",
  "_ip_header_enabled": true,
  "_smtp_img_src": "cid:",
  "admin_ratelimit_max_burst": 2,
  "admin_ratelimit_seconds": 6,
  "admin_session_lifetime": 20,
  "admin_token": "***",
  "allowed_iframe_ancestors": "",
  "attachments_folder": "data/attachments",
  "auth_request_purge_schedule": "30 * * * * *",
  "authenticator_disable_time_drift": false,
  "data_folder": "data",
  "database_conn_init": "",
  "database_max_conns": 256,
  "database_timeout": 30,
  "database_url": "***************",
  "db_connection_retries": 20,
  "disable_2fa_remember": false,
  "disable_admin_token": false,
  "disable_icon_download": false,
  "domain": "*****://********************",
  "domain_origin": "*****://********************",
  "domain_path": "",
  "domain_set": true,
  "duo_host": null,
  "duo_ikey": null,
  "duo_skey": null,
  "email_attempts_limit": 5,
  "email_expiration_time": 1800,
  "email_token_size": 6,
  "emergency_access_allowed": false,
  "emergency_notification_reminder_schedule": "0 3 * * * *",
  "emergency_request_timeout_schedule": "0 7 * * * *",
  "enable_db_wal": true,
  "event_cleanup_schedule": "0 10 0 * * *",
  "events_days_retain": null,
  "extended_logging": true,
  "helo_name": null,
  "hibp_api_key": null,
  "icon_blacklist_non_global_ips": true,
  "icon_blacklist_regex": null,
  "icon_cache_folder": "data/icon_cache",
  "icon_cache_negttl": 259200,
  "icon_cache_ttl": 2592000,
  "icon_download_timeout": 10,
  "icon_redirect_code": 302,
  "icon_service": "internal",
  "incomplete_2fa_schedule": "30 * * * * *",
  "incomplete_2fa_time_limit": 3,
  "invitation_expiration_hours": 120,
  "invitation_org_name": "Mine",
  "invitations_allowed": true,
  "ip_header": "X-Real-IP",
  "job_poll_interval_ms": 30000,
  "log_file": "/var/log/vaultwarden/vaultwarden.log",
  "log_level": "info",
  "log_timestamp_format": "%Y-%m-%d %H:%M:%S.%3f",
  "login_ratelimit_max_burst": 3,
  "login_ratelimit_seconds": 4,
  "org_attachment_limit": null,
  "org_creation_users": "**********************",
  "org_events_enabled": false,
  "org_groups_enabled": false,
  "password_hints_allowed": false,
  "password_iterations": 700442,
  "push_enabled": true,
  "push_installation_id": "***",
  "push_installation_key": "***",
  "push_relay_uri": "https://push.bitwarden.com",
  "reload_templates": false,
  "require_device_email": true,
  "rsa_key_filename": "data/rsa_key",
  "send_purge_schedule": "0 5 * * * *",
  "sendmail_command": null,
  "sends_allowed": true,
  "sends_folder": "data/sends",
  "show_password_hint": false,
  "signups_allowed": false,
  "signups_domains_whitelist": "***********,********************",
  "signups_verify": true,
  "signups_verify_resend_limit": 20,
  "signups_verify_resend_time": 300,
  "smtp_accept_invalid_certs": false,
  "smtp_accept_invalid_hostnames": false,
  "smtp_auth_mechanism": "\"Login\"",
  "smtp_debug": false,
  "smtp_embed_images": true,
  "smtp_explicit_tls": null,
  "smtp_from": "*****************************",
  "smtp_from_name": "Mine",
  "smtp_host": "******************",
  "smtp_password": "***",
  "smtp_port": 587,
  "smtp_security": "starttls",
  "smtp_ssl": null,
  "smtp_timeout": 15,
  "smtp_username": "***********************************",
  "templates_folder": "data/templates",
  "tmp_folder": "data/tmp",
  "trash_auto_delete_days": 30,
  "trash_purge_schedule": "0 5 0 * * *",
  "use_sendmail": false,
  "use_syslog": false,
  "user_attachment_limit": 102400,
  "web_vault_enabled": true,
  "web_vault_folder": "web-vault/",
  "websocket_address": "0.0.0.0",
  "websocket_enabled": false,
  "websocket_port": 3012,
  "yubico_client_id": null,
  "yubico_secret_key": null,
  "yubico_server": null
}
Originally created by @knightian on GitHub (Oct 1, 2023). ![image](https://github.com/dani-garcia/vaultwarden/assets/72685601/3341bc36-f7cb-457f-8d35-a02a343c6bfc) Websockets are working fine, everything seems to work except for anything that vaultwarden needs to reach out to like version checks, NTP and push notifications. In the log there is error setting up push: ``` [2023-10-02 00:18:49.867][vaultwarden::api::push][ERROR] Error getting push token from bitwarden server: error sending request for url (https://identity.bitwarden.com/connect/token): error trying to connect: dns error: No connections available [2023-10-02 00:18:49.868][vaultwarden::api::core::accounts][ERROR] An error occured while proceeding registration of a device: {"ErrorModel":{"Message":"Error getting push token from bitwarden server: error sending request for url (https://identity.bitwarden.com/connect/token): error trying to connect: dns error: No connections available","Object":"error"},"ExceptionMessage":null,"ExceptionStackTrace":null,"InnerExceptionMessage":null,"Message":"Error getting push token from bitwarden server: error sending request for url (https://identity.bitwarden.com/connect/token): error trying to connect: dns error: No connections available","Object":"error","ValidationErrors":{"":["Error getting push token from bitwarden server: error sending request for url (https://identity.bitwarden.com/connect/token): error trying to connect: dns error: No connections available"]},"error":"","error_description":""} [2023-10-02 00:18:49.868][response][INFO] (put_device_token) PUT /api/devices/identifier/<uuid>/token => 400 Bad Request ``` I can curl https://identity.bitwarden.com/connect/token from the server and it reaches it without issue. This all used to work fine it is just the last few Bitwarden versions this seems to happen, never had any issues in the past. SMTP reaches out no worries to send emails through Microsoft 365. Using nginx as reverse proxy, was using my own config but I have switched to the nginx config provided by @BlackDex in the proxy examples. FWIW I also see this in the vaultwarden log: ```[2023-10-02 00:59:53.441][trust_dns_resolver::system_conf::unix][WARN] no nameservers found in config``` But DNS is working fine on the server. I suspect this is going to be a DNS issue within Vaultwarden, but why does it only pop up in current versions did something change? Here is the debug string: ### Your environment (Generated via diagnostics page) * Vaultwarden version: v1.29.2-bc26bfa5 * Web-vault version: v2023.8.2 * OS/Arch: linux/x86_64 * Running within Docker: false (Base: Not applicable) * Environment settings overridden: false * Uses a reverse proxy: true * IP Header check: true (X-Real-IP) * Internet access: false * Internet access via a proxy: false * DNS Check: true * Browser/Server Time Check: true * Server/NTP Time Check: n/a * Domain Configuration Check: true * HTTPS Check: true * Database type: SQLite * Database version: 3.41.2 * Clients used: * Reverse proxy and version: * Other relevant information: ### Config (Generated via diagnostics page) <details><summary>Show Running Config</summary> **Environment settings which are overridden:** ```json { "_duo_akey": null, "_enable_duo": false, "_enable_email_2fa": true, "_enable_smtp": true, "_enable_yubico": true, "_icon_service_csp": "", "_icon_service_url": "", "_ip_header_enabled": true, "_smtp_img_src": "cid:", "admin_ratelimit_max_burst": 2, "admin_ratelimit_seconds": 6, "admin_session_lifetime": 20, "admin_token": "***", "allowed_iframe_ancestors": "", "attachments_folder": "data/attachments", "auth_request_purge_schedule": "30 * * * * *", "authenticator_disable_time_drift": false, "data_folder": "data", "database_conn_init": "", "database_max_conns": 256, "database_timeout": 30, "database_url": "***************", "db_connection_retries": 20, "disable_2fa_remember": false, "disable_admin_token": false, "disable_icon_download": false, "domain": "*****://********************", "domain_origin": "*****://********************", "domain_path": "", "domain_set": true, "duo_host": null, "duo_ikey": null, "duo_skey": null, "email_attempts_limit": 5, "email_expiration_time": 1800, "email_token_size": 6, "emergency_access_allowed": false, "emergency_notification_reminder_schedule": "0 3 * * * *", "emergency_request_timeout_schedule": "0 7 * * * *", "enable_db_wal": true, "event_cleanup_schedule": "0 10 0 * * *", "events_days_retain": null, "extended_logging": true, "helo_name": null, "hibp_api_key": null, "icon_blacklist_non_global_ips": true, "icon_blacklist_regex": null, "icon_cache_folder": "data/icon_cache", "icon_cache_negttl": 259200, "icon_cache_ttl": 2592000, "icon_download_timeout": 10, "icon_redirect_code": 302, "icon_service": "internal", "incomplete_2fa_schedule": "30 * * * * *", "incomplete_2fa_time_limit": 3, "invitation_expiration_hours": 120, "invitation_org_name": "Mine", "invitations_allowed": true, "ip_header": "X-Real-IP", "job_poll_interval_ms": 30000, "log_file": "/var/log/vaultwarden/vaultwarden.log", "log_level": "info", "log_timestamp_format": "%Y-%m-%d %H:%M:%S.%3f", "login_ratelimit_max_burst": 3, "login_ratelimit_seconds": 4, "org_attachment_limit": null, "org_creation_users": "**********************", "org_events_enabled": false, "org_groups_enabled": false, "password_hints_allowed": false, "password_iterations": 700442, "push_enabled": true, "push_installation_id": "***", "push_installation_key": "***", "push_relay_uri": "https://push.bitwarden.com", "reload_templates": false, "require_device_email": true, "rsa_key_filename": "data/rsa_key", "send_purge_schedule": "0 5 * * * *", "sendmail_command": null, "sends_allowed": true, "sends_folder": "data/sends", "show_password_hint": false, "signups_allowed": false, "signups_domains_whitelist": "***********,********************", "signups_verify": true, "signups_verify_resend_limit": 20, "signups_verify_resend_time": 300, "smtp_accept_invalid_certs": false, "smtp_accept_invalid_hostnames": false, "smtp_auth_mechanism": "\"Login\"", "smtp_debug": false, "smtp_embed_images": true, "smtp_explicit_tls": null, "smtp_from": "*****************************", "smtp_from_name": "Mine", "smtp_host": "******************", "smtp_password": "***", "smtp_port": 587, "smtp_security": "starttls", "smtp_ssl": null, "smtp_timeout": 15, "smtp_username": "***********************************", "templates_folder": "data/templates", "tmp_folder": "data/tmp", "trash_auto_delete_days": 30, "trash_purge_schedule": "0 5 0 * * *", "use_sendmail": false, "use_syslog": false, "user_attachment_limit": 102400, "web_vault_enabled": true, "web_vault_folder": "web-vault/", "websocket_address": "0.0.0.0", "websocket_enabled": false, "websocket_port": 3012, "yubico_client_id": null, "yubico_secret_key": null, "yubico_server": null } ``` </details>
Author
Owner

@knightian commented on GitHub (Oct 1, 2023):

I solved this problem.

I looked at the code and I see that vaultwarden is using a package called trust-dns-resolver

I found this article: https://crates.io/crates/trust-dns-resolver and I noticed that trust-dns-resolver package is parsing /etc/resolv.conf looking for nameserver entries. Because I am receiving my DNS via DNS options, I didn't have any nameserver entries in this file and so it was causing Bitwarden to not be able to do any DNS lookups.

I added:

nameserver 127.0.0.1 into /etc/resolv.conf because my server also runs a local unbound instance that can be used. Once I did this, it fixed the issues (but now push notifications with iOS are not working still).

So long story short

If you do not have any nameserver listed in /etc/resolv.conf it will break vaultwarden for some things (mostly outgoing requests)

@knightian commented on GitHub (Oct 1, 2023): I solved this problem. I looked at the code and I see that vaultwarden is using a package called `trust-dns-resolver` I found this article: https://crates.io/crates/trust-dns-resolver and I noticed that trust-dns-resolver package is parsing `/etc/resolv.conf` looking for nameserver entries. Because I am receiving my DNS via DNS options, I didn't have any nameserver entries in this file and so it was causing Bitwarden to not be able to do any DNS lookups. I added: `nameserver 127.0.0.1` into /etc/resolv.conf because my server also runs a local unbound instance that can be used. Once I did this, it fixed the issues (but now push notifications with iOS are not working still). **So long story short** If you do not have any nameserver listed in `/etc/resolv.conf` it will break vaultwarden for some things (mostly outgoing requests)
Author
Owner

@knightian commented on GitHub (Oct 1, 2023):

I would like to NOT have to have a nameserver entry in /etc/resolv.conf, is it possible to force trust-dns-resolver to use a particular DNS server instead? Maybe through an ENV var in Vaultwarden?

@knightian commented on GitHub (Oct 1, 2023): I would like to NOT have to have a nameserver entry in `/etc/resolv.conf`, is it possible to force trust-dns-resolver to use a particular DNS server instead? Maybe through an ENV var in Vaultwarden?
Author
Owner

@stefan0xC commented on GitHub (Oct 1, 2023):

@knightian I don't think so. This also seems rather esoteric to me. And I'm not sure if we want to add another configuration option. But you should probably ask in the trust-dns repository if this is possible or something they'd consider adding, if it's not.

@stefan0xC commented on GitHub (Oct 1, 2023): @knightian I don't think so. This also seems rather esoteric to me. And I'm not sure if we want to add another configuration option. But you should probably ask [in the trust-dns repository](https://github.com/bluejekyll/trust-dns) if this is possible or something they'd consider adding, if it's not.
Author
Owner

@knightian commented on GitHub (Oct 1, 2023):

This also seems rather esoteric to me.

I see, but if someone is using a VPS and the cloud provider is using Netplan to configure the network and DNS is coming from DHCP options and is not resident in resolv.conf then they are going to have the same bad time.

How to get around it? Or just allow people who install into VPS with these setups to fail?

@knightian commented on GitHub (Oct 1, 2023): > This also seems rather esoteric to me. I see, but if someone is using a VPS and the cloud provider is using Netplan to configure the network and DNS is coming from DHCP options and is not resident in resolv.conf then they are going to have the same bad time. How to get around it? Or just allow people who install into VPS with these setups to fail?
Author
Owner

@BlackDex commented on GitHub (Oct 1, 2023):

Well, by default docker takes the resolv.conf from the host, if that isn't ok, you should use docker run --dns=1.1.1.1 or it's equivalent for docker-compose.

@BlackDex commented on GitHub (Oct 1, 2023): Well, by default docker takes the `resolv.conf` from the host, if that isn't ok, you should use `docker run --dns=1.1.1.1` or it's equivalent for docker-compose.
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/vaultwarden#1724