Data Engineer 🇦🇺

We're on a mission to make a big green dent in the universe by building a truly sustainable energy system globally.

That means making power greener, smarter, and more affordable for everyone.

We put customers at the heart of everything we do: with always-fair prices, transparency and outrageous customer service.

We drive system change: with transformative tech to make renewable energy the norm and end global reliance on fossil fuels.

Octopus Energy Group is made up of 10 businesses spanning 6 countries across 3 continents, including: Octopus Energy Retail, Kraken Technologies, Octopus Energy Generation, Kraken Flex, Octopus Electric Vehicles, and the Octopus Centre for Net Zero. More on OEG @ octopusenergy.group

At Kraken we have developed a platform to provide data services to our retail businesses and clients across 14 deployments globally. The data platform engineering team develops and manages the back-end systems and processes supporting this platform, predominantly working with time-series data fed into the system by our customers’ smart meters. We have developed a suite of applications to process, transform, and make this data available to downstream services. To process the volume of data we typically see, our applications run PySpark or dbt jobs using Databricks as our Spark engine. We also leverage Airflow and Kubernetes heavily. We also employ software engineering best practices to design, test, and deploy our data platform and services.

This is a fantastic opportunity to work on data problems that genuinely move us closer to Net Zero with a company that is passionate about building great technology to change the way customers use energy. As a data engineer, you will be responsible for supporting the constant improvement of our data systems. As well as working on our core datalake transformations you may be developing custom applications to process third-party data, managing infrastructure in our AWS deployment, or working to expand our presence across our APAC (Australia, New Zealand, Japan) deployments.

Key responsibilities:

- Supporting the development of our core PySpark applications

- Supporting the development of our dbt models

- Building and maintaining data pipelines in Airflow

- Responding to internal requests for tooling and data pipelines

We are looking for someone who will self-manage, adapt to our changing business requirements, and proactively work to scope problems and deliver pragmatic solutions.

Ideally, you have experience:

- Building applications in Python

- Working in a Spark or Databricks environment

- Working in a software or data engineering environment

We also want our data engineers to be great software engineers with a passion for writing quality code, so it would be helpful to have experience in at least some of the following: 

- SQL

- dbt

- Cloud data platforms (Ideally AWS)

- Kubernetes

- Version control and DevOps best practices

We would prefer someone who can work in our Melbourne office hybrid / remote of 1-2 days a week onsite but it's not a deal breaker. You do need to have the rights to work in Australia.