Lead Data Engineer / Architect – Databricks Active - SC Cleared

2 Days Old

Lead Data Engineer / Architect – Databricks - SC Cleared SR2 is supporting a critical greenfield transformation programme in the public sector and urgently seeks a hands-on Lead Data Engineer/Architect with deep Databricks experience to help set the direction and strategy for the project. This is a strategic role for someone who has not only delivered but also led end-to-end Databricks implementations, ideally across multiple programmes. You’ll define the technical architecture, lead Proof of Concepts (PoCs), and build a modern data platform from scratch in a highly visible public-sector environment. Key Responsibilities:
Act as the technical lead for a new Databricks implementation, working from greenfield through to full production deployment. Own and deliver streaming and batch data pipelines in Databricks for complex, sensitive use cases. Define and set architectural standards and delivery roadmaps for the data platform. Lead PoCs with hyperscalers to assess and select appropriate data services and tooling. Collaborate with engineering teams, stakeholders, and partners to ensure scalability, performance, and compliance. Document best practices, decisions, and technical architecture to support future scaling and handover.Essential Experience:
2–3+ years of strong, hands-on Databricks experience, including having led implementations from setup through to production. Demonstrable track record of delivering greenfield or ground-up data platform builds. Strong Python skills for data transformation and orchestration. Deep understanding of modern data architectures, pipelines, and cloud-native solutions – especially AWS. Able to operate strategically (architecture, direction-setting) and tactically (hands-on engineering). Active SC Clearance (must be current)
Apply
Location:
Farringdon
Salary:
£600 - £650 /day
Job Type:
FullTime
Category:
IT

We found some similar jobs based on your search