Data Engineer - Intermediate (12 month contract)

SUMMARY:
A data pipeline is a series of automated, programmatic steps. These data pipelines gather data from

POSITION INFO:

Reports to: Software Architect

Job Overview:

This person is responsible for creating data pipelines to support downstream data delivery.

A data pipeline is a series of automated, programmatic steps. These data pipelines gather data from numerous, potentially disparate data sources and bring everything together into a central place (data lake & warehouse) where it is made readily available for downstream consumption.

The incumbent works with a variety of technologies on a day to day basis. The nature of the job requires constant upskilling and personal development.

Duties:

There are two main duties:

1. Develop & maintain data pipelines in support of downstream data delivery.
2. Perform analysis on organisational data & create reports to communicate the results.

Other potential duties include, but are not limited to:

• Communicate and advocate for a data-driven mindset in the larger organisational community, especially the IT department.
• Investigate and evaluate emerging technologies like robotic process automation (RPA), new AWS services, new data management approaches, machine learning advancements, etc.
• Liaise with individuals in the organisation in support of all the above duties.

Qualifications & Skills required:

• Good programming skills in both Python and SQL.
• Good database design skills and an understanding of data warehousing techniques and approaches.
• An understanding of and ability to access data from sources such as:
  • Microsoft SQL Server
  • Oracle
  • MongoDB
  • Amazon S3
  • Windows and Linux file systems (Excel, delimited files, etc.)
  • HTTP APIs
  • SFTP sites
  • An ability to engineer cloud-first solutions in the Amazon Web Services cloud.
  • An interest in machine learning and advanced analytics, and the development of data pipelines in support thereof.
  • An understanding of the software development process, with proficiency in Git for version control and build & deployment pipelines in GitLab.
  • An ability to create reports and BI dashboards to communicate data & analysis results, using a variety of technologies, which might include:
    • Microsoft SQL Server Reporting Services
    • Microsoft PowerBI
    • Microsoft Excel

NB! This job is now closed. You can apply for other jobs by uploading your CV.