A leading CPG company was incurring high billing costs post-migration of the customer, product, and pricing data from Azure to Google cloud platform (GCP). The monthly cost cap had almost doubled leading to aborting a few critical analytics initiatives. To solve the problem, the customer’s DevOps team set up Google cloud billing dashboards but could not achieve visibility in usage and wastage. Moreover, the billing reports were generated on a monthly basis making it difficult to keep a track of any anomaly in billing patterns and predict monthly expenses. They wanted us to analyze the reasons for the cost spike and find ways to control it by implementing best practices in monitoring, guard railing, and optimizing the cloud.
We exported billing data into BigQuery and created SQL views for visualization on Grafana dashboards. These dashboards showed visibility in the utilization of cloud resources across various GCP services at a granular level. We also created email-based alerts on a daily, weekly, and monthly basis to track cloud cost fluctuations and notify the customer in case of anomalies in billing patterns, the variance of change in cost, and the cause of change. We also limited the over-provisioning of resources by applying rightsizing recommendations to limit cost wastage across services such as Databricks and introducing auto-termination of resource utilization.