Senior Data Engineer
New Yesterday
Job Description
The Client
This firm is a highly respected, technology-centric investment business operating across a broad range of asset classes. Their success is built on a mix of quantitative research, cutting-edge engineering and scalable data infrastructure. Engineers here play a central role: they design, build and maintain the platforms that underpin research, trading and large-scale data analysis.
It’s a collaborative environment where technical ownership is encouraged, engineering craft is valued, and impactful work directly supports sophisticated investment strategies.
What You'll Get
- Work on the design and build of fast, scalable market-data systems used across trading and research groups.
- Contribute to a modern engineering ecosystem: Python, cloud-native tooling, containerisation, large-scale data lake technologies.
- Partner closely with exceptional quantitative researchers, data engineers and traders.
- Influence architectural decisions and continuously refine pipeline performance.
- Join a culture that values rigour, curiosity and continual improvement.
- Benefit from strong compensation and long-term career growth within a high-performing engineering organisation.
Role Overview
- Design, implement, and maintain high-throughput, low-latency pipelines for ingesting and processing tick-level market data at scale.
- Operate and optimise timeseries databases (KDB, OneTick) to efficiently store, query, and manage granular datasets.
- Architect cloud-native solutions for scalable compute, storage, and data processing, leveraging AWS, GCP, or Azure.
- Develop and maintain Parquet-based data layers; contribute to evolving the data lake architecture and metadata management.
- Implement dataset versioning and management using Apache Iceberg.
- Collaborate closely with trading and quant teams to translate data requirements into robust, production-grade pipelines.
- Implement monitoring, validation, and automated error-handling to ensure data integrity and pipeline reliability.
- Continuously profile and optimise pipeline throughput, latency, and resource utilisation, particularly in latency-sensitive or HFT-like environments.
- Maintain clear, precise documentation of data pipelines, architecture diagrams, and operational procedures.
What You Bring
- 3+ years of software engineering experience, preferably focused on market-data infrastructure or quantitative trading systems.
- Strong Python expertise with a solid grasp of performance optimisation and concurrency.
- Proven experience designing, building, and tuning tick-data pipelines for high-volume environments.
- Hands-on experience with Parquet storage; experience with Apache Iceberg or similar table formats is a plus.
- Practical experience with containerisation (Docker) and orchestration platforms (Kubernetes).
- Strong background in profiling, debugging, and optimising complex data workflows.
- Experience with timeseries databases (KDB, OneTick) and/or performance-critical C++ components.
- Deep understanding of financial markets, trading data, and quantitative workflows.
- Excellent communication skills with the ability to articulate technical solutions to engineers and non-engineers alike.
- Location:
- London
- Job Type:
- FullTime
- Category:
- Technology
We found some similar jobs based on your search
-
New Yesterday
Senior Data Engineer
-
London
- Technology
Job Description The Client This firm is a highly respected, technology-centric investment business operating across a broad range of asset classes. Their success is built on a mix of quantitative research, cutting-edge engineering and scalable d...
More Details -
-
New Yesterday
Senior Data Engineer | Commodities & Energy Trading | ClickHouse-Centric Data Platform | Next-Gen...
-
City Of London
- Technology
Job Description Senior Data Engineer | Commodities & Energy Trading | ClickHouse-Centric Data Platform | Next-Gen Lakehouse | Python, SQL | Up to £89K + Bonus + Benefits A Senior Data Engineer is required for a leading global Energy & Commoditie...
More Details -
-
New Yesterday
Senior Data Engineer
-
City Of London
- Technology
Job Description The Client This firm is a highly respected, technology-centric investment business operating across a broad range of asset classes. Their success is built on a mix of quantitative research, cutting-edge engineering and scalable d...
More Details -
-
New Yesterday
Senior Data Engineer
-
Highbury
-
£475 - £550 /day
- IT
Rate: £550 p/day Environment: Hybrid (3 days) Location: North London Company: Retelligence About Retelligence Retelligence is partnering with a high-growth, forward-thinking organization that specializes in digital innovation and marketing acros...
More Details -
-
1 Days Old
Senior Data Engineer
-
N5, Highbury, Greater London
-
£475 - £550 /day
- IT
Rate: £550 p/day Environment: Hybrid (3 days) Location: North London Company: Retelligence About Retelligence Retelligence is partnering with a high-growth, forward-thinking organization that specializes in digital innovation and marketing acros...
More Details -
-
1 Days Old
Senior Data Engineer, SQL, RDBMS, AWS, Python, Mainly Remote
-
Holborn And Covent Garden
-
£85,000 - £95,000 /annum Excellent benefits
- IT
Senior Data Engineer, SQL, RDBMS, Python, Celery, RabbitMQ, AWS, Part Central London, Mainly Remote Senior Data Engineer (SQL, RDBMS, Python, AWS) required to work for a fast growing and exciting business based in Central London. However, this role ...
More Details -