4. Setting up the Team
To build a team of data experts, often the first step includes the assessment of the current organizational data maturity.
Organizational Data And Analytics Maturity Assessment
This is a measurement of the extent to which an organization can make use of available data. The higher the maturity index, the better the data utilization is. To assess this, the first step is to clearly outline the desired end state in terms of data maturity and expertise and compare it to the current state. The roadmap for progress must include bridging this gap with tangible key performance indicators (KPIs) wherein important progress-based metrics are regularly measured and used as specific benchmarks for progress tracking. Often bridging the gaps requires hiring and extensive upskilling or reskilling based on current and the desired state.
Figure: Organizational data maturity index
Skill Sets Needed
The next step would be to build specific expertise for an effective data and analytics team. Some of the key expertise required will include:
They work closely with both data analyst and data scientists to design, build, and maintain datasets that are leveraged across data projects. Their overarching responsibilities include preparing the ecosystem and infrastructure that the organization and its data team rely on. This includes selecting and integrating tools and frameworks. Apart from this, they may also be required to optimize and maintain data warehouses, build data platforms that other D&A team members can use, and collect and integrate data from disparate sources.
A data architect designs, manages, deploys, and creates the organizational data architecture. Highly experienced data architects often lead technical teams and help manage data pipelines with high volumes. Although, in the case of smaller teams, the data engineer usually undertakes a portion of these responsibilities.
They work with cleaned and transformed user-friendly data formats to conduct reporting and direct analysis. They are responsible for using data to forecast and suggest business activities, maintain dashboards, prepare data visualizations, and generate reports using descriptive, prescriptive, predictive, or diagnostic analysis.
Business/ BI Analyst
They work alongside data scientists to prioritize problems and steer the projects into critical directions. They often act as a conduit between data objectives and related business outcomes. BI analysts are responsible for the conception, control, monitoring, and development of company-wide business intelligence systems.
They typically perform work related to informing and shaping of data projects leveraging advanced tools, programming and technologies such as artificial intelligence, machine learning, and statistical modeling to perform analysis on a large scale. This may involve the identification of challenges that can be addressed with data sources or a specific data project.They help analyze and interpret data and generate reports. They are adept at applying advanced analytical models to solve real world business problems, tune and improve current ML pipeline, test and suggest new data science approaches, and also help the engineering team in testing and scaling.
Will lead the overall team. Must be proficient developing Big Data architectures for Hadoop/Spark platform, and have big data development experience on Hadoop platform including Hive, Impala, Sqoop, Flume, Spark. Moreover, experience with data modelling, complex data structures, data processing, data quality and data lifecycle is also highly desired.
Primary expertise must include rich experience in managing end to end data projects. Also critical is the understanding and drive to incorporate customer requirements, translate them to objective tasks and manage the overall team towards success.