The client is one of the largest and oldest mining companies in the world with 12 sites in different locations. The metallurgical setups in these sites produce data which are site-specific, manual, and Excel spreadsheet-based, making it error-prone and not insightful for the business. The client was not only looking to create global automated reports across sites but create a dashboard that can act as a single source of truth for stakeholders to help them make business decisions and identify data anomalies.
Sigmoid automated the reporting process by converting Excel-based logic into SQL. We used the DBT framework to process complex SQL queries which are further dockerized and deployed to GKE using Google Cloud Build and Cloud Trigger on code commit. The deployed application was triggered via Google Pub-Sub, the results of which get written to BigQuery tables. These are queried by Tableau for visualization dashboards which help in detecting data anomalies by drilling down to the root cause of the problem.
Sigmoid created 1 global report to track business operations while providing both tabular and graphical representations of metrics data which made anomaly detection efficient by 2X. The report generation time was reduced from 12 hours to 2 minutes.