initial icon caching spawns *lots* of processes #160

New Issue

OVERLORD · 2026-02-04T18:06:41+03:00

OVERLORD commented

2026-02-04 18:06:41 +03:00

Originally created by @tycho on GitHub (Dec 17, 2018).

I noticed that if my ICON_CACHE_FOLDER is empty and I visit my vault, the main bitwarden_rs process seems to spawn many processes (or threads, not sure) to cache all the icons at once, or at least, not with a reasonable upper bound. This should probably be done with only a few processes at a time (like 4 or 8 or something?), both to be friendly to process limits and to be friendly to icons.bitwarden.com.

The reason I noticed this issue is because I had LimitNPROC=64 in my bitwarden_rs.service file, which caused bitwarden_rs to crash with a "resource temporarily unavailable" error code on trying to load the vault page. I bumped the limit much higher and was able to load my vault page, but it'd be nice to not need to do that.

Originally created by @tycho on GitHub (Dec 17, 2018). I noticed that if my `ICON_CACHE_FOLDER` is empty and I visit my vault, the main bitwarden_rs process seems to spawn *many* processes (or threads, not sure) to cache all the icons at once, or at least, not with a reasonable upper bound. This should probably be done with only a few processes at a time (like 4 or 8 or something?), both to be friendly to process limits and to be friendly to icons.bitwarden.com. The reason I noticed this issue is because I had `LimitNPROC=64` in my `bitwarden_rs.service` file, which caused bitwarden_rs to crash with a "resource temporarily unavailable" error code on trying to load the vault page. I bumped the limit much higher and was able to load my vault page, but it'd be nice to not need to do that.

OVERLORD added the enhancement label 2026-02-04 18:06:41 +03:00

OVERLORD closed this issue

2026-02-04 18:06:42 +03:00

OVERLORD commented

2026-02-04 18:06:56 +03:00

@mprasil commented on GitHub (Dec 17, 2018):

I'm not sure how to approach this to be honest. The icons API is not authenticated so we don't necessarily know if it's one client spamming the API with tons of requests or more clients requesting few icons. We could look at the client IP, but that's often not correct or helpful. (all clients behind NAT, etc..) So I don't see any reliable way to do per-client throttling on the server side.

There were some issues raised for the client side upstream code, but I think it went kinda nowhere.

You can always attempt to limit the API calls on the proxy side, if you don't connect directly to server, but as I said, the in many cases you can't reliably limit the calls without risking very bad experience for some users.

@mprasil commented on GitHub (Dec 17, 2018): I'm not sure how to approach this to be honest. The icons API is not authenticated so we don't necessarily know if it's one client spamming the API with tons of requests or more clients requesting few icons. We could look at the client IP, but that's often not correct or helpful. (all clients behind NAT, etc..) So I don't see any reliable way to do per-client throttling on the server side. There were [some](https://github.com/bitwarden/web/issues/220) issues [raised](https://github.com/bitwarden/web/issues/217#issuecomment-406460038) for the client side upstream code, but I think it went kinda nowhere. You can always attempt to limit the API calls on the proxy side, if you don't connect directly to server, but as I said, the in many cases you can't reliably limit the calls without risking very bad experience for some users.

OVERLORD commented

2026-02-04 18:07:07 +03:00

@dani-garcia commented on GitHub (Dec 17, 2018):

We don't manage threads ourselves, it's rocket doing it, but it always launches the same number, so I don't know how the problem only appears in this case.

You can manage the number of threads with the ROCKET_WORKERS env variable (by default it's number of cores * 2).

Other than that or using a proxy as @mprasil said, there is not much we can do.

@dani-garcia commented on GitHub (Dec 17, 2018): We don't manage threads ourselves, it's rocket doing it, but it always launches the same number, so I don't know how the problem only appears in this case. You can manage the number of threads with the `ROCKET_WORKERS` env variable (by default it's number of cores * 2). Other than that or using a proxy as @mprasil said, there is not much we can do.

OVERLORD commented

2026-02-04 18:07:24 +03:00

@dani-garcia commented on GitHub (Dec 19, 2018):

Seeing as we can't do a lot ourselves, I'll close this.

Note that an alternative to avoid hammering the official server would be to use LOCAL_ICON_EXTRACTOR=true, which would download the favicons directly from each site, but that has its own negatives.

@dani-garcia commented on GitHub (Dec 19, 2018): Seeing as we can't do a lot ourselves, I'll close this. Note that an alternative to avoid hammering the official server would be to use `LOCAL_ICON_EXTRACTOR=true`, which would download the favicons directly from each site, but that has its own negatives.

OVERLORD referenced this issue

2026-02-05 04:50:14 +03:00

[PR #160] [MERGED] Add DELETE handlers fo cipher and attachment deletion (fixes #158) #2615

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: starred/vaultwarden#160