Principal Data Engineer

Number of employees

100

Remote, United States

Posted on: 2023-03-09

Category: sustainability

Apply now

Please let Zignal Labs know you found this job on Work in Green. This will help us grow!

Expired

Employment type:

Full time

Remote?

Yes

Experience required:

Intermediate

Salary

Salary not provided

About the company:

Zignal Labs’ real-time intelligence technology helps the world’s largest organizations protect their people, places, and position. Zignal’s AI-powered platform analyzes billions of data points in real time, accelerating mission critical decision making by empowering leaders with contextual situational awareness of the information environment.


About Zignal Labs

Zignal Labs’ real-time intelligence technology helps the world’s largest organizations protect their people, places, and position. Analyzing billions of data points in real time,  Zignal's AI-powered platform accelerates mission-critical decision making by empowering leaders with contextual situational awareness of the information environment.

Fully remote, with Silicon Valley roots and team members in over 20 states, Zignal serves customers around the world. Learn more at zignallabs.com.

As the Principal Data Engineer on the Platform team at Zignal Labs, you will get to use your Scala and Java experience to build a best-in-class distributed data and analytics infrastructure by leveraging open source technologies such as Apache Spark, Apache Storm, and Elasticsearch.  We use social media, news, blogs and other media sources to empower our users with key insights based on real-time analysis.

Why join Zignal Labs?
- Competitive salary based on the work you do
- Flexible time off – work with your manager to take the time you need
- Excellent medical, dental, and vision coverage
- Paid parental leave plan
- Professional development and growth programs 
- A tight knit, collaborative, and transparent environment to help you succeed

Zignal Labs is proud to be an Equal Opportunity employer. We do not discriminate based upon race, religion, color, national origin, sex (including pregnancy, childbirth, or related medical conditions), sexual orientation, gender, gender identity, gender expression, transgender status, sexual stereotypes, age, status as a protected veteran, status as an individual with a disability, or other applicable legally protected characteristics. We also consider qualified applicants with criminal histories, consistent with applicable federal, state and local law.

Applicants must be authorized to work in the United States for any employer. No sponsorship is available for this position.

In this role, you will have the opportunity to:

  • Solve complex real-time data collection & analysis problems with cutting edge technical solutions
  • Iterate on our high performance and scalable platform for massive data collection, real-time analytics, NLP, machine learning, and backend data services
  • Build high performance, scalable, real-time, server-side technologies
  • Write scalable code with extensive test coverage, working in a professional software engineering environment with source control, dev/stage/production release cycles, continuous integration, and deployment
  • Work closely with product management, design, quality assurance and operations teams to understand our customers’ needs and effectively translate them to technical specifications
  • Lead projects from translating product requirements into architecture to production

  • Tech Stack:

  • Scala, Java, Python
  • Apache Spark, Spark Streaming, Databricks/Delta Lake, Apache Storm, Elasticsearch, Apache Nifi
  • Kafka, MongoDB, Redis
  • AWS

  • In order to be successful in this role, you will need:

  • Bachelor's degree (or higher) in Computer Science, Engineering, or similar and/or relevant work experience
  • Experience providing technical leadership at the enterprise level for the design of information technology systems
  • Crafted and implemented operational data stores, as well as data lakes in production environments
  • Ability to analyze, diagnose and resolve complex architectural problems using industry standard engineering principles
  • Design and build data ingestion pipelines and ETL processing, including stream processing, while factoring in performance and cost
  • Identify and solve issues concerning data management to improve data quality
  • Clean, prepare and optimize data for ingestion and consumption
  • Experience solving performance problems with Lucene based search solutions like Elasticsearch or Solr
  • 9+ years experience in server-side/back-end full cycle product development in a production environment
  • 4+ years developing with Apache Spark, including Structured Streaming.   Experience with Databricks is a big plus
  • Knowledge of Scala or Java with exposure to or interest in Scala
  • Leads and mentors other team members
  • Provides partners with coaching and feedback in order to build effective teams
  • Provides effective support to cross-functional teams

  • Similar climate jobs

    These are some of our top picks for great climate jobs on Work in Green.

    View all jobs
    Zignal Labs logo
    United States
    Rachio logo
    United States
    Zignal Labs logo
    United States
    Zignal Labs logo
    United States
    Rachio logo
    United States

    1 Sustainability jobs at Zignal Labs

    Zignal Labs is hiring Senior Account Executive, National Security, and more.

    View all jobs at Zignal Labs