Logo of Huzzle

Sr. Staff Software Engineer, Batch & Stream Processing

image

Pinterest

21d ago

  • Job
    Full-time
    Senior Level
  • Software Engineering
  • $175.5K - $361K
  • San Francisco
    Remote

AI generated summary

  • You need 8+ years of industry experience, extensive knowledge of big data technologies like Spark and Hadoop, leadership skills, and proficiency in Java, Scala, or Python. Familiarity with Kubernetes and AWS is a plus.
  • You will lead Spark platform strategy, improve internals, build batch processing infrastructure, support business use cases, provide thought leadership, and contribute to technical vision and roadmap.

Requirements

  • 8+years of industry experience with a proven track record of technical excellence
  • 5+ years of experience of building and support large scalable big data infrastructure/platform
  • 2+ years of experiences in contributing to open-source Spark
  • Deep knowledge of big data technologies (e.g. Spark, Hadoop, Parquet/ORC, Flink)
  • Experience in leading cross-team engineering efforts
  • Proficiency in one or more programming languages (Java, Scala, Python)
  • Experiences in Kubernetes and AWS technologies

Responsibilities

  • Lead the strategy and technical direction of Pinterest’s Spark platform
  • Improve and customize the internals of open source Spark to meet our challenges at scale and actively participate in open source community
  • Build and scale batch processing frameworks and infrastructure to process petabytes-scale datasets
  • Work with internal customers on critical business use cases that rely on batch processing
  • Provide thought leadership to the entire company on how data should be processed and stored more reliably, quickly and efficiently at scale
  • Contribute to the team’s technical vision and long-term roadmap

FAQs

What technologies will I be working with in this role?

You will be working with exciting big data open source technologies such as Spark, Kafka, and Kubernetes at Pinterest.

What is the scale of data I will be working with in this role?

You will be working at the scale of hundreds of petabytes of data to help Pinners discover and do what they love.

Bring everyone the inspiration to create a life they love.

Technology
Industry
1001-5000
Employees
2010
Founded Year

Mission & Purpose

Pinterest's mission is to bring everyone the inspiration to create a life they love. It’s the biggest dataset of ideas ever assembled, with over 200 billion recipes, home hacks, style inspiration and other ideas to try. More than 430 million people around the world use Pinterest to dream about, plan and prepare for things they want to do in life.