Cannot import vault json into Vaultwarden #1147

Closed
opened 2026-02-05 00:08:09 +03:00 by OVERLORD · 4 comments
Owner

Originally created by @fbartels on GitHub (Nov 8, 2021).

Subject of the issue

I am trying to move one Vaultwarden Vault from one instance to another. Initially I wanted to do this with Portwarden (to also migrate attachments), but Portwarden was throwing the following error:

restoring folder folder1
restoring folder folder2
restoring item item1
An error occured:  exit status 1
(bytes.Buffer) 
(bytes.Buffer) Error saving cipher
restoring item item2
2021/11/08 07:21:48 unexpected end of JSON input

Trying to directly trying to import the vault data generated from the "Export Vault" function in the web ui does not give a visible error message, but in the browser tools I can see that the request produced an error 400 with the following response:

{"ErrorModel":{"Message":"Error saving cipher","Object":"error"},"ExceptionMessage":null,"ExceptionStackTrace":null,"InnerExceptionMessage":null,"Message":"Error saving cipher","Object":"error","ValidationErrors":{"":["Error saving cipher"]},"error":"","error_description":""}

Vaultwarden logs the following when this happens (in both cases):

Nov 08 08:21:48 [2021-11-08 07:21:48.697][request][INFO] POST /api/ciphers
Nov 08 08:21:48 [2021-11-08 07:21:48.701][error][ERROR] Error saving cipher.
Nov 08 08:21:48 [CAUSE] DatabaseError(
Nov 08 08:21:48 __Unknown,
Nov 08 08:21:48 "Malformed communication packet.",
Nov 08 08:21:48 )
Nov 08 08:21:48 [2021-11-08 07:21:48.701][response][INFO] POST /api/ciphers (post_ciphers) => 400 Bad Request
Nov 08 08:21:48 123.123.123.123 - - [08/Nov/2021:07:21:48 +0000] "POST /api/ciphers HTTP/1.1" 400 276 "-" "Bitwarden_CLI/[object Promise] (LINUX)"

I tried to import the same json file into the Bitwarden SaaS and got a more descriptive error there:

image

Deployment environment

Your environment (Generated via diagnostics page)

  • Vaultwarden version: v
  • Web-vault version: v2.23.0
  • Running within Docker: true (Base: Debian)
  • Environment settings overridden: false
  • Uses a reverse proxy: true
  • IP Header check: true (X-Forwarded-For)
  • Internet access: true
  • Internet access via a proxy: false
  • DNS Check: true
  • Time Check: true
  • Domain Configuration Check: true
  • HTTPS Check: true
  • Database type: MySQL
  • Database version: 8.0.23-0ubuntu0.20.04.1
  • Clients used:
  • Reverse proxy and version:
  • Other relevant information:

Config (Generated via diagnostics page)

Show Running Config

Environment settings which are overridden:

{
  "_duo_akey": null,
  "_enable_duo": false,
  "_enable_email_2fa": true,
  "_enable_smtp": true,
  "_enable_yubico": true,
  "_ip_header_enabled": true,
  "admin_token": "***",
  "allowed_iframe_ancestors": "",
  "attachments_folder": "/app/data/attachments",
  "authenticator_disable_time_drift": false,
  "data_folder": "/app/data",
  "database_max_conns": 10,
  "database_url": "*****://****************:************************************************@*****/****************",
  "db_connection_retries": 15,
  "disable_2fa_remember": false,
  "disable_admin_token": false,
  "disable_icon_download": false,
  "domain": "*****://********.***.**",
  "domain_origin": "*****://********.***.**",
  "domain_path": "",
  "domain_set": true,
  "duo_host": null,
  "duo_ikey": null,
  "duo_skey": null,
  "email_attempts_limit": 3,
  "email_expiration_time": 600,
  "email_token_size": 6,
  "emergency_access_allowed": true,
  "emergency_notification_reminder_schedule": "0 5 * * * *",
  "emergency_request_timeout_schedule": "0 5 * * * *",
  "enable_db_wal": false,
  "extended_logging": true,
  "helo_name": null,
  "hibp_api_key": null,
  "icon_blacklist_non_global_ips": true,
  "icon_blacklist_regex": null,
  "icon_cache_folder": "/app/data/icon_cache",
  "icon_cache_negttl": 259200,
  "icon_cache_ttl": 2592000,
  "icon_download_timeout": 10,
  "invitation_org_name": "Vaultwarden",
  "invitations_allowed": true,
  "ip_header": "X-Forwarded-For",
  "job_poll_interval_ms": 30000,
  "log_file": "/run/vaultwarden/vaultwarden.log",
  "log_level": "info",
  "log_timestamp_format": "%Y-%m-%d %H:%M:%S.%3f",
  "org_attachment_limit": null,
  "org_creation_users": "",
  "password_iterations": 100000,
  "reload_templates": false,
  "require_device_email": false,
  "rsa_key_filename": "/app/data/rsa_key",
  "send_purge_schedule": "0 5 * * * *",
  "sends_allowed": true,
  "sends_folder": "/app/data/sends",
  "show_password_hint": false,
  "signups_allowed": true,
  "signups_domains_whitelist": "",
  "signups_verify": false,
  "signups_verify_resend_limit": 6,
  "signups_verify_resend_time": 3600,
  "smtp_accept_invalid_certs": false,
  "smtp_accept_invalid_hostnames": false,
  "smtp_auth_mechanism": "Plain",
  "smtp_debug": false,
  "smtp_explicit_tls": false,
  "smtp_from": "********.***@***.**",
  "smtp_from_name": "password.app@domain.com",
  "smtp_host": "****",
  "smtp_password": "***",
  "smtp_port": 2525,
  "smtp_ssl": false,
  "smtp_timeout": 15,
  "smtp_username": "********.***@***.**",
  "templates_folder": "/app/data/templates",
  "trash_auto_delete_days": null,
  "trash_purge_schedule": "0 5 0 * * *",
  "use_syslog": false,
  "user_attachment_limit": null,
  "web_vault_enabled": true,
  "web_vault_folder": "web-vault/",
  "websocket_address": "0.0.0.0",
  "websocket_enabled": true,
  "websocket_port": 3012,
  "yubico_client_id": null,
  "yubico_secret_key": null,
  "yubico_server": null
}

Steps to reproduce

Import a json file with an entry that has more than 10000 characters in the notes field.

Expected behaviour

Import the json or give an error message similar to the Bitwarden SaaS.

Actual behaviour

Non descriptive error message.

Troubleshooting data

Originally created by @fbartels on GitHub (Nov 8, 2021). ### Subject of the issue I am trying to move one Vaultwarden Vault from one instance to another. Initially I wanted to do this with Portwarden (to also migrate attachments), but Portwarden was throwing the following error: ``` restoring folder folder1 restoring folder folder2 restoring item item1 An error occured: exit status 1 (bytes.Buffer) (bytes.Buffer) Error saving cipher restoring item item2 2021/11/08 07:21:48 unexpected end of JSON input ``` Trying to directly trying to import the vault data generated from the "Export Vault" function in the web ui does not give a visible error message, but in the browser tools I can see that the request produced an error 400 with the following response: ``` {"ErrorModel":{"Message":"Error saving cipher","Object":"error"},"ExceptionMessage":null,"ExceptionStackTrace":null,"InnerExceptionMessage":null,"Message":"Error saving cipher","Object":"error","ValidationErrors":{"":["Error saving cipher"]},"error":"","error_description":""} ``` Vaultwarden logs the following when this happens (in both cases): ``` Nov 08 08:21:48 [2021-11-08 07:21:48.697][request][INFO] POST /api/ciphers Nov 08 08:21:48 [2021-11-08 07:21:48.701][error][ERROR] Error saving cipher. Nov 08 08:21:48 [CAUSE] DatabaseError( Nov 08 08:21:48 __Unknown, Nov 08 08:21:48 "Malformed communication packet.", Nov 08 08:21:48 ) Nov 08 08:21:48 [2021-11-08 07:21:48.701][response][INFO] POST /api/ciphers (post_ciphers) => 400 Bad Request Nov 08 08:21:48 123.123.123.123 - - [08/Nov/2021:07:21:48 +0000] "POST /api/ciphers HTTP/1.1" 400 276 "-" "Bitwarden_CLI/[object Promise] (LINUX)" ``` I tried to import the same json file into the Bitwarden SaaS and got a more descriptive error there: ![image](https://user-images.githubusercontent.com/1257835/140704328-33a4fc9c-ee57-42d9-9ac6-8e88cbeee79c.png) ### Deployment environment ### Your environment (Generated via diagnostics page) * Vaultwarden version: v * Web-vault version: v2.23.0 * Running within Docker: true (Base: Debian) * Environment settings overridden: false * Uses a reverse proxy: true * IP Header check: true (X-Forwarded-For) * Internet access: true * Internet access via a proxy: false * DNS Check: true * Time Check: true * Domain Configuration Check: true * HTTPS Check: true * Database type: MySQL * Database version: 8.0.23-0ubuntu0.20.04.1 * Clients used: * Reverse proxy and version: * Other relevant information: ### Config (Generated via diagnostics page) <details><summary>Show Running Config</summary> **Environment settings which are overridden:** ```json { "_duo_akey": null, "_enable_duo": false, "_enable_email_2fa": true, "_enable_smtp": true, "_enable_yubico": true, "_ip_header_enabled": true, "admin_token": "***", "allowed_iframe_ancestors": "", "attachments_folder": "/app/data/attachments", "authenticator_disable_time_drift": false, "data_folder": "/app/data", "database_max_conns": 10, "database_url": "*****://****************:************************************************@*****/****************", "db_connection_retries": 15, "disable_2fa_remember": false, "disable_admin_token": false, "disable_icon_download": false, "domain": "*****://********.***.**", "domain_origin": "*****://********.***.**", "domain_path": "", "domain_set": true, "duo_host": null, "duo_ikey": null, "duo_skey": null, "email_attempts_limit": 3, "email_expiration_time": 600, "email_token_size": 6, "emergency_access_allowed": true, "emergency_notification_reminder_schedule": "0 5 * * * *", "emergency_request_timeout_schedule": "0 5 * * * *", "enable_db_wal": false, "extended_logging": true, "helo_name": null, "hibp_api_key": null, "icon_blacklist_non_global_ips": true, "icon_blacklist_regex": null, "icon_cache_folder": "/app/data/icon_cache", "icon_cache_negttl": 259200, "icon_cache_ttl": 2592000, "icon_download_timeout": 10, "invitation_org_name": "Vaultwarden", "invitations_allowed": true, "ip_header": "X-Forwarded-For", "job_poll_interval_ms": 30000, "log_file": "/run/vaultwarden/vaultwarden.log", "log_level": "info", "log_timestamp_format": "%Y-%m-%d %H:%M:%S.%3f", "org_attachment_limit": null, "org_creation_users": "", "password_iterations": 100000, "reload_templates": false, "require_device_email": false, "rsa_key_filename": "/app/data/rsa_key", "send_purge_schedule": "0 5 * * * *", "sends_allowed": true, "sends_folder": "/app/data/sends", "show_password_hint": false, "signups_allowed": true, "signups_domains_whitelist": "", "signups_verify": false, "signups_verify_resend_limit": 6, "signups_verify_resend_time": 3600, "smtp_accept_invalid_certs": false, "smtp_accept_invalid_hostnames": false, "smtp_auth_mechanism": "Plain", "smtp_debug": false, "smtp_explicit_tls": false, "smtp_from": "********.***@***.**", "smtp_from_name": "password.app@domain.com", "smtp_host": "****", "smtp_password": "***", "smtp_port": 2525, "smtp_ssl": false, "smtp_timeout": 15, "smtp_username": "********.***@***.**", "templates_folder": "/app/data/templates", "trash_auto_delete_days": null, "trash_purge_schedule": "0 5 0 * * *", "use_syslog": false, "user_attachment_limit": null, "web_vault_enabled": true, "web_vault_folder": "web-vault/", "websocket_address": "0.0.0.0", "websocket_enabled": true, "websocket_port": 3012, "yubico_client_id": null, "yubico_secret_key": null, "yubico_server": null } ``` </details> ### Steps to reproduce Import a json file with an entry that has more than 10000 characters in the notes field. ### Expected behaviour Import the json or give an error message similar to the Bitwarden SaaS. ### Actual behaviour Non descriptive error message. ### Troubleshooting data
OVERLORD added the enhancementdocumentationtroubleshootinglow priority labels 2026-02-05 00:08:09 +03:00
Author
Owner

@cksapp commented on GitHub (Nov 9, 2021):

There is a 10k limit on upstream Bitwarden, mind you this is also 10k characters when encrypted as well, so decrypted in the vault it will have to be much less. Do you have vault items with more than the 10k character limit in notes field? In my limited testing, it seems when attempting to create a vault item with more than 10k characters the same above error occurs, this is on the backend server side after encryption. In plain-text within your vault this appears to be approx. ~7400k characters

https://bitwarden.com/help/article/import-data/#troubleshooting-import-errors

Tip
On import to Bitwarden, the character count of any given field is increased due to encryption, meaning that an 8000-character Notes field in your .csv will scale to 10,000+ characters when it comes into contact with Bitwarden, triggering this error. As a rule of thumb, character counts will grow between 30-50% when encrypted.

It appears Vaultwarden does not have this same 10K character limit when encrypted server side, as the web vault allows notes well over 10k characters.
Though it does seem to break somewhere in between 5mil - 10 million characters.

This would best be solved with a feature request (PR) to have Vaultwarden honour the 10k encrypted limit server side when the client attempts to import, create, or edit and save an item over this limit.

@cksapp commented on GitHub (Nov 9, 2021): There is a 10k limit on upstream Bitwarden, mind you this is also 10k characters when **_encrypted_** as well, so decrypted in the vault it will have to be much less. Do you have vault items with more than the 10k character limit in notes field? In my limited testing, it seems when attempting to create a vault item with more than 10k characters the same above error occurs, this is on the backend server side after encryption. In plain-text within your vault this appears to be approx. ~7400k characters https://bitwarden.com/help/article/import-data/#troubleshooting-import-errors > Tip > On import to Bitwarden, the character count of any given field is increased due to encryption, meaning that an 8000-character Notes field in your .csv will scale to 10,000+ characters when it comes into contact with Bitwarden, triggering this error. As a rule of thumb, character counts will grow between 30-50% when encrypted. It appears Vaultwarden does not have this same 10K character limit when encrypted server side, as the web vault allows notes well over 10k characters. _Though it does seem to break somewhere in between 5mil - 10 million characters._ This would best be solved with a feature request (**PR**) to have Vaultwarden honour the 10k encrypted limit server side when the client attempts to import, create, or edit and save an item over this limit.
Author
Owner

@BlackDex commented on GitHub (Dec 23, 2021):

I'm not able to reproduce this, at least not on the latest version currently available.
It could be that they fixed something in the latest web-vault version v2.25.x.

Could you please try that and see if this is solved?

Update/Edit:
btw, i used a string with 30.000+ characters in both a Login and Secure Note, exported and imported using both Bitwarden's JSON and CSV formats.

@BlackDex commented on GitHub (Dec 23, 2021): I'm not able to reproduce this, at least not on the latest version currently available. It could be that they fixed something in the latest web-vault version v2.25.x. Could you please try that and see if this is solved? Update/Edit: btw, i used a string with 30.000+ characters in both a `Login` and `Secure Note`, exported and imported using both Bitwarden's `JSON` and `CSV` formats.
Author
Owner

@BlackDex commented on GitHub (Dec 24, 2021):

I just also checked a different database besides SQLite3, with MariaDB/MySQL it also works fine.
I think this is related to an issue i discovered last week that sometimes imports are aborted or timed-out because of some websocket refreshes which are generated.

it's either that, or the bitwarden cli (which is used by portwarden) doesn't support these items to read/write.
That is something i did not try yet.

@BlackDex commented on GitHub (Dec 24, 2021): I just also checked a different database besides SQLite3, with MariaDB/MySQL it also works fine. I think this is related to an issue i discovered last week that sometimes imports are aborted or timed-out because of some websocket refreshes which are generated. it's either that, or the bitwarden cli (which is used by portwarden) doesn't support these items to read/write. That is something i did not try yet.
Author
Owner

@fbartels commented on GitHub (Dec 27, 2021):

Hi @BlackDex,

thanks for your response (and @cksapp of course as well). I understood this as a general recommendation to break down large note fields and split them up. The migration carried out a few weeks ago and I no longer have access to the old installation.

Since it does no longer seem to be a problem in newer installations it sounds like a good idea to close this and the original error messages no longer appear.

@fbartels commented on GitHub (Dec 27, 2021): Hi @BlackDex, thanks for your response (and @cksapp of course as well). I understood this as a general recommendation to break down large note fields and split them up. The migration carried out a few weeks ago and I no longer have access to the old installation. Since it does no longer seem to be a problem in newer installations it sounds like a good idea to close this and the original error messages no longer appear.
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/vaultwarden#1147