I have an open SR on this but was wondering if anyone in the community has come across this. Our vROps environment is configured for large, it has an HA pair, and three remote collectors. Two additional data nodes were added for the issues below. For some time now we have been unable to clear alerts.
My understanding is that I should be able to clear an alert, but once the check-in occurs, if that metric is still detected, the alert will fire again. However when we try to clear alerts, we either get error messages that vROps is unable to clear the alert, or we will get a successful message, however the alert still shows in the list and it never goes away. I have verified our permissions, and even performed these actions using the built in admin account. Since this SR was opened in May, I have been watching vROps, and it does clear the alerts on its own as the metric is no longer detected.
Troubleshooting
After several log bundles, VMWare thought that when the alerts trigger, vROps is not able to store the alarms in the database, so when we cancel the alert, it fails because the alarms are missing, so VMware basically purged the database. VMWare suggested we add two additional data nodes to our environment, and then perform a disk rebalance. The nodes were added, the disk rebalance was conducted on 6/6, however the DB was not balanced across the ha pair and nodes.
Currently, engineering is still investigating..
Thoughts?