Lead Data Engineer
Up to £110,000
This is a rare opportunity to head up a project building a brand new cloud data platform from scratch.
The role is with a data driven business who have used data to weather the storm during lock-down and as a result have managed to continue hiring and are in a strong financial position while competitors have faced redundancies. The business understand the value of data and are ready to invest in building out a new cloud data platform to house their multi terabyte data set. They have a strong team of BI Engineers, Software Engineers and Data Scientists. You'd be the first hire into data engineering and would have the opportunity to build the team out under you with no pressure to take on management responsibilities.
You'd be joining their growing data team in London and working closely with software engineers and business intelligence analysts on their data platform. This is a greenfield build and you'd have full autonomy over technical decision making. You'd be programming in Python, streaming data using spark and providing the data for advanced data science work.
- Program in Python while evangelising software engineering best practices (TDD, BDD, OOP)
- Stream data using Spark and Pypark or real time software such as Kafka or Kinesis
- Design and Architect the new cloud platform (AWS or Azure)
- Evangelise the use of data and the platform across the wider business
- Draw up the technical roadmaps and reccommend technologies for adoption across the team
- Gather data requirements from BI and Data science aswell as non technical teams