Server Connection Limit prevents backups from running
Description
We have noticed that some vaults go into a state where the maximum number of concurrent connections has been reached and the connections are not released automatically. This causes the Server Connection Limit to grow and no new backup connections are accepted.
By default, the maximum number of concurrent server connections is set to 60. This feature has been introduced to ensure that the server resources are not overutilized and should automatically allow new connections once existing backup connections complete.
In the case described in this article, the backup connections are not released even though the backups completed.
How to identify the problem
If you look at the Device Protection section on the Protection screen on Endpoint Cloud, you will see that there may be a number of devices marked as "Protected with Warning" or "Unprotected". If you have a further look under Protection Status, you will notice a category called "Server Connection Limit" with a number of devices next to it. The protection rating will continue to decrease.
If this "Server Connection Limit" continues to grow and does not move closer to zero, then the Vault may not be releasing connections.
In order to verify this behavior, you may want to look through the Vault logs for the following entries:
Vault
Max concurrent sessions 60 reached
Handling 'StorageServiceException' exception with error response:ErrorResponse(storeError=2, errorMessage=Max concurrent sessions reached. Please try again later.)
Log phrase to look for: "Max concurrent sessions reached"
Find the log here: Browse to the volume the Vault is installed on
[INSTALL VOLUME]:\Cibecs\Vault\logs\vault.log>
What caused this problem?
This is a known issue where the vault occasionally does not release stale connections. The issue has been fixed in version 2.9 and later of the Vault server.
How do I resolve this problem?
Option 1 - Upgrade the Vault to version 2.9 or later. See how: Upgrade Vault
Option 2 - If the Vault is already on the latest version and you are still seeing this issue then:
Open up the local server services and restart the Cibecs Vault Service:
If the issue occurs again, then please contact Cibecs Support for further assistance.
Check if the problem has been fixed
If the problem has been fixed successfully, then the 'Protected with Warning' and "Unprotected" counter will start to decrease and the Protection Rating will start to increase.