Lead Data Engineer

Location: Hybrid (60% Office / 40% Remote) Sheffield
Contract: 6 Month Contract (Extension possible)
Rate: £400+ per day inside IR35

Role Overview

We are seeking an experienced Lead Data Engineer to design, develop, and optimise enterprise-scale data platforms for large, regulated organisations, ideally in banking, financial services, or other regulated sectors. This role requires hands-on technical expertise, leadership, and a consulting mindset to deliver scalable, resilient data solutions while promoting best practices and operational excellence.

Key Responsibilities

• Lead the design, development, and optimisation of enterprise data engineering platforms

• Build and maintain robust ETL/ELT pipelines integrating large, complex datasets

• Work with structured, semi-structured, and unstructured data across SQL and NoSQL technologies

• Develop solutions using Hadoop, Spark, and Splunk in large-scale environments

• Write maintainable Python code, applying object-oriented and functional programming principles

• Implement and maintain CI/CD pipelines, automated testing, and version control

• Collaborate with BI, Analytics, and downstream teams to support reporting and insights

• Pair program and mentor other engineers to promote knowledge sharing and code quality

• Define and maintain technical test plans including unit and integration tests

• Promote SRE principles to ensure service resilience, sustainability, and recoverability

Essential Skills & Experience

• Proven experience as a Lead or Senior Data Engineer in enterprise-scale environments

• Hands-on expertise with Hadoop, Spark, and Splunk

• Advanced Python development skills

• Experience designing and optimising high-performance data pipelines

• Strong understanding of CI/CD, source control, and automated testing

• Analytical, problem-solving, and leadership skills

• Experience working in regulated, enterprise environments (e.g., banking, fintech, government)

• Agile delivery experience (Scrum/Kanban)

Consulting & Soft Skills

• Ability to mentor and uplift team performance

• Strong communication and stakeholder engagement skills

• Collaborative, delivery-focused mindset with high accountability

• Knowledge of control, compliance, and regulatory requirements

• Up-to-date awareness of modern tools, cybersecurity, and data privacy regulations

• Champions innovation, advanced technologies, and best practices

If this role aligns with your skills and experience, we'd love to hear from you. Apply today to be considered.

JBRP1_UKTJ