We are looking for a Data Engineer that will play a critical role in ensuring that we build data products that are not only capable of transforming the company but meet the highest standards of software development. You will be responsible for exciting projects covering the end-to-end data life cycle – from raw data integrations with primary and third-party systems, through advanced data modelling, to state-of-the-art data visualization and development of innovative data products.
- Build scalable data pipelines to ingest data from a variety of data sources, identify critical data elements and define data quality rules.
- Partner with business stakeholders to understand their needs and build impactful analytics solutions.
- Use various methods to analyse and transform raw data into useful data
- Develop and maintain large and complex datasets
- Improve data reliability, quality and efficiency
- Prepare data for prescriptive and predictive modelling to help build algorithms and prototypes
- Work with other engineers and product teams to identify and build tools to automate many data management/analysis tasks.
- Resolve and provide solutions to any data issues that may arrive
- Create proper technical documentation outlining the entire process
- Effectively research and benchmark technology against other best in class technologies.
Skills And Qualifications
- A degree in computer science, engineering, mathematics, or equivalent experience.
- 5+ years of previous Data Engineering experience, with strong skills in at least one programming
- 3+ years of experience in Data/Analytics platform architecture design.
- 3+ years of hands-on experience building data pipelines in production and ability to work across structured, semi-structured and unstructured data.
- 3+ years of experience in ML pipeline for batch workflow.
- Ability to write clean, maintainable, and robust code in Python, Scala, Java or similar languages.
- Deep understanding and expertise of software engineering concepts and best practices.
- Deep knowledge and expertise of testing frameworks and libraries.
- Excellent interpersonal, written and oral communication skills
- Must be able to multi-task and work in a fast-paced environment