Senior Data Platform Engineer

We're on a mission to make a big green dent in the universe by building a truly sustainable energy system globally.

That means making power greener, smarter, and more affordable for everyone.

We put customers at the heart of everything we do: with always-fair prices, transparency and outrageous customer service.

We drive system change: with transformative tech to make renewable energy the norm and end global reliance on fossil fuels.

Octopus Energy Group is made up of 10 businesses spanning 6 countries across 3 continents, including: Octopus Energy Retail, Kraken Technologies, Octopus Energy Generation, Kraken Flex, Octopus Electric Vehicles, and the Octopus Centre for Net Zero. More on OEG @ octopusenergy.group

Sitting within the Systems and Data / Transformation team, you will turn existing platform architectures, governance specs, and prototype patterns into production-grade infrastructure. You’ll work alongside our Tech and Transformation Lead, data engineers, application builders, and business stakeholders across Finance, Asset Management, and Fund Operations.

This is not a general DevOps or blank-page architecture role. The primary focus is designing and implementing enterprise-grade identity, access, and governance controls across human users, systems, and non-human agents.

Our Tech & Ecosystem: Databricks, Lakebase, Unity Catalog, Entra ID / Azure Identity, Okta, ZScaler, GitHub Actions, MCP patterns, and Terraform.

 

 

 

Platform Engineering & CI/CD

Build and maintain Infrastructure-as-Code (IaC) using Terraform across development, test, and production environments.

Strengthen CI/CD practices in GitHub Actions, establishing repeatable deployment patterns, environment promotions, automated rollbacks, and release controls.

Support architecture decisions by converting designs into maintainable, well-documented engineering patterns.

Identity, Access & Security

Design and implement federated identity architecture across Okta, Entra ID, and ZScaler.

Build identity, token exchange, and permission propagation models that work for both human users and non-human AI agents.

Implement SCIM provisioning/deprovisioning and access models with single-revoke capability and full auditability.

Harden existing AI gateway and agent orchestration layers (built on MCP patterns) into secure, production-ready capabilities.

Implement API gateway controls: authentication, routing, rate limiting, token budgets, policy-as-code, and action logging.

Build platform-, tool-, and agent-level kill switches, alongside dry-run/safe-testing modes for agent workflows.

Observability & FinOps

Implement AI system observability, including prompt logging, output monitoring, quality scoring, and drift detection.

Establish operational monitoring for gateway usage, latency, error rates, and model routing logic.

Enforce token budgets, cost alerts, and usage/cost attribution by team and use case.

Enablement & Integration

Deploy vector database and retrieval infrastructure (e.g., Databricks AI Search) to support Document AI and RAG use cases.

Align platform controls with Databricks and Unity Catalog governance patterns.

Provide clear documentation, reusable patterns, and self-service onboarding guidance so delivery squads can deploy tools without bespoke manual intervention.

Experience with AI gateways, LLM proxies, or model-routing patterns.

Exposure to AI observability tools, vector databases (Databricks AI Search), RAG pipelines, or non-deterministic system monitoring.

Hands-on experience implementing policy-as-code, action logging, or AI kill switches.

Direct experience with Okta, ZScaler, Entra ID, Unity Catalog, or M365 migration environments.

Background in regulated or high-governance industries (Financial Services, Energy, Infrastructure Investment, Fintech, or Enterprise SaaS).

Familiarity with FinOps practices (token budgeting, usage tracking, cost attribution).