Data Engineer

Job title

Data Engineer

Location

Bryanston (Hybrid)

Salary

450 – 550K

Role overview

A data engineer is responsible for designing, developing, and maintaining the data infrastructure and systems required for efficient and reliable data processing, storage, and analysis. They work closely with data scientists, analysts, and other stakeholders to understand data requirements and ensure the availability, quality, and accessibility of data for organizational needs.

Duties

• Data Pipeline Development:

-      Designing, implementing, and optimizing scalable and efficient data pipelines to extract, transform, and load (ETL) data from various sources into data storage systems. 

• Data Warehouse Management:

-       Developing and managing data warehouse architectures, including data modeling, schema design, and performance tuning, to support data analytics and reporting needs

• Data Integration:

-       Integrating disparate data sources and systems, ensuring seamless data flow and compatibility between different platforms and formats

• Data Quality Assurance:

-       Implementing data quality checks, data validation processes, and data cleansing techniques to ensure the accuracy, consistency, and integrity of data.

• Database Management:

-      Administering and optimizing database systems, including configuration, maintenance, backup, and recovery, to ensure data availability and performance. 

• Data Security and Privacy:

-       Implementing appropriate security measures and data privacy protocols to protect sensitive and confidential information, adhering to relevant regulations and best practices. 

• Data Monitoring and Troubleshooting:

-      Proactively monitoring data processes and systems, identifying and resolving data-related issues, and optimizing performance bottlenecks. 

• Collaboration and Documentation:

-      Collaborating with cross-functional teams, such as data scientists, analysts, and software engineers, to understand data requirements and deliver efficient data solutions. Documenting data infrastructure, processes, and configurations for knowledge sharing and future reference. 

• Emerging Technologies:

-      Staying updated with industry trends and emerging technologies related to data engineering, recommending and implementinginnovative solutions to improve data management and analytics capabilities.