Delayed Asset Collection / FlexReports failing to run / Interactive Drilldowns failing to load
Incident Report for CloudHealth Technologies
Resolved
This issue has been resolved. All affected services are now confirmed as operational.
Posted Dec 08, 2021 - 09:43 EST
Update
Updated Summary of the impact to the CloudHealth Platform from the earlier Incident.

1. Collection of AWS Asset Metadata was impacted between ~ Dec 7th 15:30 UTC - Dec 7th 22:04 UTC.

Any AWS policies that were configured on a per service/asset basis may not have triggered correctly during this period.


2. FlexReports for all Clouds were impacted and unable to execute up until ~ Dec 8th 1:20 UTC. These have since recovered, and you should be able to re-run any impacted reports.

3. Interactive Drilldown data for all Clouds may be unavailable until the next scheduled report processing cycles completes.

4. Partner Generated Billing for all Clouds may be delayed while we continue to recover and reschedule processing tasks now that we have recovered.

5. Cost and Usage reports for all Clouds are expected to have delayed data for at least 12 - 24 hours while we reschedule and recover from the outage. This will also impact Cost and Usage based policies.
Posted Dec 07, 2021 - 21:11 EST
Update
We are continuing to monitor for any further issues.
Posted Dec 07, 2021 - 18:21 EST
Update
In addition to the collection issues, we are now also observing issues with FlexReports returning errors when executing and Interactive Drilldowns failing to load.

We will continue to monitor for any further downstream platform issues that are experienced and provide further periodic updates while we await recovery.

Please reach out to support@cloudhealthtech.com for any questions.
Posted Dec 07, 2021 - 18:21 EST
Monitoring
The CloudHealth platform has experienced delayed collection of AWS assets since 16:00 UTC as a result of an AWS API issue.

Collection during this period has been intermittent so short-lived assets terminated during this period may not be present within the CloudHealth platform and other assets may have outdated metadata.

FlexReports are also reporting errors when users attempt to run or export the report.

Interactive Drilldowns are also failing to load.

A further update will be provided once the AWS has resolved the API issue and collection is confirmed as stable.
Posted Dec 07, 2021 - 11:37 EST
This incident affected: CloudHealth Platform.