Senior / Lead Data Engineer (AI-Focused) in City of London
New Today
Energy Jobline is the largest and fastest growing global Energy Job Board and Energy Hub. We have an audience reach of over 7 million energy professionals, 400,000+ monthly advertised global energy and engineering jobs, and work with the leading energy companies worldwide.
We focus on the Oil & Gas, Renewables, Engineering, Power, and Nuclear markets as well as emerging technologies in EV, Battery, and Fusion. We are committed to ensuring that we offer the most exciting career opportunities from around the world for our jobseekers.
Job Description
PaymentGenes is proud to be partnering with a high-growth, international technology organisation to appoint a Senior / Lead Data Engineer (AI-Focused). This is a strategic hire for a business investing heavily in AI-enabled products and advanced analytics. If you are passionate about architecting modern data platforms that power real AI at scale — this opportunity is for you.
This is more than a data engineering role! You will define and scale the organisation’s data platform to power advanced analytics, machine learning, and AI-driven products. Combining deep technical expertise with architectural leadership, you’ll shape long-term data strategy while remaining hands-on in building robust, production-grade systems.
You will be accountable for platform reliability, scalability, governance, and AI enablement across the business.
What You’ll Do
Data Platform Architecture & Strategy
Define and evolve the data architecture roadmap
Design scalable ELT/ETL frameworks using DBT and cloud data warehouses
Establish orchestration standards (Dagster or equivalent)
Drive decisions across batch, streaming, and real-time pipelines
Champion data modelling standards, semantic layers, and metric governance
AI & ML Enablement at Scale
Architect data foundations supporting the ML lifecycle
Design feature stores, embedding pipelines, and AI-ready datasets
Enable MLOps workflows (data versioning, monitoring, retraining triggers)
Support production inference (batch and real-time)
Evaluate and integrate emerging AI tooling where strategically valuable
Technical Leadership
Set best practices for testing, documentation, lineage, and observability
Lead code reviews and mentor data & analytics engineers
Drive CI/CD and infrastructure-as-code adoption
Own platform reliability, performance optimisation, and cost efficiency
Establish SLAs for data freshness and quality
Cross-Functional Impact
Partner with Data Science, Product, and Engineering leadership
Translate business strategy into scalable data solutions
Influence KPI and metric governance across teams
Act as technical escalation point for complex data challenges
Technical Environment
Core Expertise
Advanced SQL & Python
Deep DBT experience (modelling, testing, macros, documentation)
Workflow orchestration (Dagster )
Cloud data warehouses (Snowflake, BigQuery, Redshift, etc.)
Data modelling for analytics and AI use cases
API integrations and ingestion design patterns
AI / ML Infrastructure
Feature engineering architecture
ML pipeline and deployment workflows
Experience supporting production ML systems
Familiarity with embeddings, vector databases, LLM orchestration (desirable)
Data observability and model monitoring
Platform & DevOps
CI/CD for data workflows
Git-based engineering standards
Docker / containerisation
Infrastructure-as-code (e.g., Terraform)
Monitoring and alerting systems
What We’re Looking For
6–10+ years in data engineering or related disciplines
Proven experience architecting and scaling modern data platforms
Experience enabling ML/AI production workflows
Demonstrated technical leadership and mentoring
Ability to influence senior stakeholders
You’ll Thrive If You Have:
Architectural thinking with long-term vision
Strong decision-making under ambiguity
The ability to balance innovation with reliability
Clear communication across technical and non-technical audiences
A strong ownership mindset and accountability for outcomes
What Success Looks Like
A scalable, reliable data platform powering AI and analytics growth
Reduced ML time-to-production
High levels of data quality, observability, and governance maturity
Improved cost-performance efficiency across the data stack
A strong, growing data engineering capability within the team
This is a high-impact leadership role within a forward-thinking technology environment where AI and data are core to the business strategy.
If this excites you, please reach out to the PaymentGenes team directly via LinkedIn, who can provide more detail and insight into this company's journey into AI and the opportunity.
If you are interested in applying for this job please press the Apply Button and follow the application process. Energy Jobline wishes you the very best of luck in your next career move.
- Location:
- City Of London
- Job Type:
- FullTime
- Category:
- Engineer, Data, Lead, Senior, Engineering
