SC Cleared Senior Data Engineer
New Today
London, United Kingdom (Hybrid) Salary:
£65,000 - £85,000 per annum An experienced Senior Data Engineer is required to design, build, and optimise scalable data platforms for complex transformation programmes. This position demands strong expertise across cloud-based data engineering, architecture, and DevOps practices, working within both AWS and Azure environments. You'll play a key role in enabling seamless data integration, supporting advanced analytics, and delivering robust, reusable data solutions aligned with client data strategies.
Key Responsibilities
p]:pt-0 [&>p]:mb-2 [&>p]:my-0'> Design and develop scalable, automated data pipelines using cloud-native technologies (e.g. Spark), integrating with on-premise sources where required.
p]:pt-0 [&>p]:mb-2 [&>p]:my-0'> Implement event-driven data architectures and optimise pipeline performance, parallelisation, and risk detection (e.g. fraud use cases).
p]:pt-0 [&>p]:mb-2 [&>p]:my-0'> Build and maintain data access layers and ETL frameworks to support self-service and ad-hoc onboarding of data assets.
p]:pt-0 [&>p]:mb-2 [&>p]:my-0'> Develop reusable engineering patterns and tooling to improve scalability and consistency across programmes.
p]:pt-0 [&>p]:mb-2 [&>p]:my-0'> Apply DevOps and software engineering best practices, including Infrastructure as Code (IaC) and CI/CD pipelines.
p]:pt-0 [&>p]:mb-2 [&>p]:my-0'> Collaborate with cross-functional teams to ensure high-quality data integration, governance, and performance.
p]:pt-0 [&>p]:mb-2 [&>p]:my-0'> Deliver secure, resilient, and scalable data solutions across hybrid cloud environments (AWS and Azure).
p]:pt-0 [&>p]:mb-2 [&>p]:my-0'> Configure and optimise data services including: p]:pt-0 [&>p]:mb-2 [&>p]:my-0'>
AWS:
EC2, EMR, S3, Lambda
p]:pt-0 [&>p]:mb-2 [&>p]:my-0'>
Azure:
Blob Storage and related services
p]:pt-0 [&>p]:mb-2 [&>p]:my-0'> Ensure alignment with client data strategies, governance frameworks, and security standards.
p]:pt-0 [&>p]:mb-2 [&>p]:my-0'> Support knowledge sharing and capability uplift across teams.
Skills & Experience - Essential
p]:pt-0 [&>p]:mb-2 [&>p]:my-0'> Strong experience in data engineering and data architecture in complex programme environments.
p]:pt-0 [&>p]:mb-2 [&>p]:my-0'> Proven ability to develop and manage automated data pipelines using technologies such as Spark.
p]:pt-0 [&>p]:mb-2 [&>p]:my-0'> Expertise in event-driven architectures and pipeline performance optimisation.
p]:pt-0 [&>p]:mb-2 [&>p]:my-0'> Deep understanding of ETL processes, data integration, and self-service onboarding models.
p]:pt-0 [&>p]:mb-2 [&>p]:my-0'> Hands-on experience across both AWS and Azure platforms.
p]:pt-0 [&>p]:mb-2 [&>p]:my-0'> Practical knowledge of: p]:pt-0 [&>p]:mb-2 [&>p]:my-0'>
AWS:
EC2, EMR, S3, Lambda
p]:pt-0 [&>p]:mb-2 [&>p]:my-0'>
Azure:
Blob Storage and related services
p]:pt-0 [&>p]:mb-2 [&>p]:my-0'> Understanding of cloud networking concepts in AWS and Azure.
p]:pt-0 [&>p]:mb-2 [&>p]:my-0'> Experience with
Terraform
(IaC) and
GitLab
(CI/CD).
p]:pt-0 [&>p]:mb-2 [&>p]:my-0'> Solid grounding in DevOps and modern software engineering practices.
Desirable
p]:pt-0 [&>p]:mb-2 [&>p]:my-0'> Experience in government or regulated sectors.
p]:pt-0 [&>p]:mb-2 [&>p]:my-0'> Familiarity with large-scale data platforms and data access layers.
p]:pt-0 [&>p]:mb-2 [&>p]:my-0'> Exposure to fraud detection or risk analytics use cases.
p]:pt-0 [&>p]:mb-2 [&>p]:my-0'> Knowledge of data governance, security, and compliance frameworks.
Security Requirements
p]:pt-0 [&>p]:mb-2 [&>p]:my-0'> This role requires
Security Clearance (SC) .
p]:pt-0 [&>p]:mb-2 [&>p]:my-0'> Candidates must hold active clearance or be eligible and willing to undergo the process.
Engineering Practices
p]:pt-0 [&>p]:mb-2 [&>p]:my-0'> Strong commitment to automation, scalability, and reusability.
p]:pt-0 [&>p]:mb-2 [&>p]:my-0'> Advocate for DevOps principles, CI/CD, and Infrastructure as Code.
p]:pt-0 [&>p]:mb-2 [&>p]:my-0'> Experienced working in agile, collaborative teams.
p]:pt-0 [&>p]:mb-2 [&>p]:my-0'> Dedicated to delivering robust, production-grade data solutions.
TPBN1_UKTJ
- Location:
- United Kingdom
- Job Type:
- FullTime
- Category:
- IT