Technical Lead / Data Architect

Start date: ASAP

Duration: 6 months

Location: London, flexible working

*Please note. - You must be registered as a freelance contractor (Ltd Co, Sole Trader) or work via an umbrella company for this assignment.*

ABOUT THE ROLE

The Technical Lead / Data Architect owns the end-to-end technical architecture, engineering organisation, platform maturity evolution, and multi-layered replatforming agenda for the VWG DTO ecosystem. This senior role reports directly to the DTO Overall Lead and operates as a peer to the Head of Governance & Strategy and the Enablement

Lead. The role is accountable for delivering a modern, scalable, AI-ready, governance-aligned data ecosystem spanning ingestion, transformation, modelling, storage, sharing, observability, and reporting.

The remit includes working with engineering teams (Ingestion, ETL/Transformation, BI/Viz, QA, DevOps, Automation), embedding governance and compliance expectations into the architecture, and driving the platform transition from both Starburst→Redshift and Airbyte→Adverity.

ROLE SUMMARY

The Technical Lead / Data Architect is accountable for:

• End-to-end technical architecture across the full e2e stack.

• Engineering leadership across ingestion, transformation, BI, QA, DevOps.

• Replatforming One Reporting’s analytical substrate to Redshift, enabling zero-copy data sharing, unified lineage and AI/LLM workloads.

• Replatforming the ingestion and integration layer from Airbyte→Adverity to support harmonised vendor management, higher observability and stronger schema governance.

• Delivering 24-hour latency expectations and 0–1% discrepancy tolerances.

• Embedding automation, observability and controlled backfills across all pipelines.

• Translating expanded client expectations (2025→2026) into scalable architectural patterns.

KEY RESPONSIBILITIES

ARCHITECTURE

1. Own the end-to-end technical architecture covering ingestion → harmonisation → modelling → metadata → storage → sharing → BI.

2. Lead architectural redesign driven by 2025 expansions (multi-source reconciliation, dual-environment support, URL governance, finance reconciliation).

3. Deliver the 2026 requirements: creative-level granularity, lineage versioning, governed metadata history, automation-first operations, AI-ready data layers.

4. Define canonical modelling standards, entity relationships, lineage contracts, and schema evolution frameworks.

5. Embed governance frameworks (taxonomy, CSREF/URL rules, tagging, QA rules) directly into the platform architecture.

ENGINEERING LEADERSHIP

1. Work with all engineering teams: Ingestion, Transformation/ETL, BI/Viz, QA, DevOps, and Automation.

2. Drive consolidation under a single engineering leadership structure.

3. Ensure deterministic, auditable, highly reliable pipelines supporting 80+ markets.

4. Establish modern software engineering practice: code reviews, CI/CD, IaC, automated QA gates, telemetry-driven operations.

5. Partner with Governance to ensure rule-driven, compliant, “Right First Time” execution across markets.

REPLATFORMING

Starburst → Redshift

• Lead end-to-end replatforming from federated Starburst architecture to Redshift analytical substrate.

• Deliver native zero-copy data sharing for the client.

• Establish unified compute+storage lineage, enabling audit-ready transparency.

• Enable SQL-native AI/LLM workloads

Airbyte → Adverity

• Lead migration of ingestion layer to Adverity to consolidate connectors, improve vendor-supported reliability, and reduce ingestion failure domains.

• Deliver a unified ingestion governance layer: schema validation, drift detection, automated reprocessing rules, lineage tagging at source.

• Support increasing platform complexity (additional local DSPs, retailer platforms, local publishers, custom feeds).

PLATFORM MATURITY

1. Deliver automated, rule-driven, end-to-end QA with 0–1% tolerance.

2. Implement full observability: SLA/SLO telemetry, heartbeat checks, freshness monitoring, discrepancy detection, error patterning.

3. Build governed, deterministic backfill mechanisms.

4. Create AI-ready, metadata-rich, versioned, machine-consumable data layers.

5. Ensure the platform “explains itself” to audits, client pipelines and LLM-based validation systems.

PEOPLE LEADERSHIP

• Lead, mentor and develop engineering leads and multi-disciplinary technical teams.

• Build an accountable, proactive engineering culture.

• Create clear KPIs, SLAs, maturity models and progression paths.

• Forecast capability needs and drive hiring aligned to 2026 requirements.

• Provide documentation, architectural decisions, and transparency to audits and client councils.

IDEAL CANDIDATE PROFILE

• Deep experience in cloud data architecture (AWS, Redshift, Glue, S3, Lambda, Bedrock).

• Strong expertise in ingestion frameworks (Airbyte, Adverity), schema governance and pipeline orchestration.

• Hands-on understanding of BI, modelling, lineage, metadata and harmonisation.

• Strong understanding of data governance, taxonomy, ID hygiene and compliance.

• Excellent communication and client-facing leadership capability.

• Strong proficiency in SQL and analytical modelling for high-volume datasets.

• Hands-on experience with dbt / dbt Cloud for modular transformations and testing.

• Experience with pipeline orchestration tools such as Airflow.

• Proficiency in DevOps/DataOps practices, including CI/CD, Git, environment automation and deployment strategies.

• Experience with Infrastructure-as-Code (Terraform, CloudFormation).

• Exposure to containerisation (Docker, ECS, Kubernetes).

• Familiarity with observability stacks (Datadog, CloudWatch, Grafana) including SLA/SLO telemetry.

• Experience with AI-ready data architectures and embedding LLM workflows into warehouse layers.

• The contractor will not need to undertake team lead responsibilities, but hands on architecture and dev experience is essential.

KPIs & SUCCESS MEASURES

• Replatforming delivered successfully and adopted.

• Airbyte→Adverity ingestion migration completed with improved reliability.

• 24h latency achieved consistently across platforms.

• 0–1% discrepancy tolerance achieved across reporting.

• Reduction in manual remediation and engineering intervention.

• Market satisfaction and client audit performance.