Logo of Huzzle

Software Engineer (m/f/t) - Machine Learning Pipelines

image

RWE

27d ago

  • Job
    Full-time
    Junior & Mid Level
  • Software Engineering
  • Seattle

AI generated summary

  • You need 2+ years of complex pipeline experience, proficiency in Python, cloud platforms, containerization, CI/CD, and strong problem-solving skills. Familiarity with stream processing, serverless computing, GCP MLOps, and monitoring tools is an advantage.
  • You will design and operate machine learning pipelines, collaborate with data scientists/engineers, optimize performance, and stay up to date with the latest ML technologies.

Requirements

  • 2+ years of experience building, deploying, and monitoring complex pipelines that operate at an hours-level cadence
  • Experience in large scale data processing and distributed systems such as Apache Spark
  • Proficiency in Python, Julia, and/or Mojo
  • Experience with cloud computing platforms such as AWS, Azure, or Google Cloud Platform
  • Strong understanding of containerization technologies (Docker, Kubernetes) and orchestration tools
  • Familiarity with version control systems (e.g., Git), continuous integration/continuous deployment (CI/CD) pipelines, and infrastructure-as-code tools
  • Excellent problem-solving skills, attention to detail, and a passion for learning and innovation
  • Strong communication and collaboration skills, with the ability to work effectively in cross-functional teams and contribute to a positive team climate
  • Advantageous, but not a must:
  • Experience with stream processing frameworks (e.g., Apache Kafka, Apache Flink) for real-time data ingestion and processing
  • Knowledge of serverless computing and event-driven architectures for building scalable and cost-effective pipelines
  • Experience with Google Cloud Platform MLOps tools (e.g., Vertex AI)
  • Familiarity with monitoring, logging, and debugging tools for distributed systems (e.g., Prometheus, ELK Stack)
  • Proficiency in database technologies for data storage and retrieval (e.g., PostgreSQL, MongoDB, BigQuery)
  • Understanding of security best practices for securing data pipelines
  • Expertise working with large geospatial weather and climate datasets as well as the Pangeo stack

Responsibilities

  • If you're passionate about building scalable pipelines for state of the art production machine learning models and thrive in a fast-paced, collaborative environment, we'd love to hear from you! Your responsibilities will include:
  • Design, develop, and operate complex machine learning and multi-faceted data fusion pipelines in a production environment
  • Collaborate closely with data scientists, machine learning engineers, and scientists to integrate machine learning models into scalable production systems
  • Develop reusable components, libraries, and frameworks to streamline pipeline development and maintenance
  • Optimize pipeline performance, reliability, and resource utilization
  • Continually expand your knowledge of the latest advancements in machine learning technologies, cloud services, and software engineering best practices with an eye towards identifying and integrating exceptional new components into the AI lab’s production repertoire
  • Contribute to a team culture where diverse viewpoints, backgrounds and expertise are welcomed

FAQs

What kind of experience is required for the Software Engineer position in Machine Learning Pipelines?

The ideal candidate should have at least 2+ years of experience in building, deploying, and monitoring complex pipelines operating at an hours-level cadence. Experience with large scale data processing, distributed systems like Apache Spark, cloud computing platforms, and containerization technologies is also required.

What programming languages should the candidate be proficient in for this role?

The candidate should be proficient in Python, Julia, and/or Mojo for building machine learning pipelines. Familiarity with other languages may be advantageous but not required.

What are the key responsibilities of the Software Engineer working on Machine Learning Pipelines?

The responsibilities include designing, developing, and operating machine learning and data fusion pipelines in a production environment, collaborating with data scientists and engineers, developing reusable components, optimizing pipeline performance, and staying updated on the latest advancements in machine learning technologies.

What skills are valued for this role beyond technical expertise?

In addition to technical skills, we value traits like passion, collaboration, diversity in backgrounds, willingness to learn and innovate, and a determined work ethic. Strong communication and collaboration skills are also important for working effectively in cross-functional teams.

Are there any preferred qualifications for the Software Engineer position in Machine Learning Pipelines?

While not mandatory, experience with stream processing frameworks, serverless computing, Google Cloud Platform MLOps tools, monitoring tools for distributed systems, database technologies, security best practices, and expertise in working with geospatial weather and climate datasets are advantageous.

Our energy for a sustainable life.

Energy
Industry
10,001+
Employees

Mission & Purpose

RWE is leading the way to a green energy world. With an extensive investment and growth strategy, the company will expand its powerful, green generation capacity to 50 gigawatts internationally by 2030. RWE is investing €50 billion gross for this purpose in this decade. The portfolio is based on offshore and onshore wind, solar, hydrogen, batteries, biomass and gas. RWE Supply & Trading provides tailored energy solutions for large customers. RWE has locations in the attractive markets of Europe, North America and the Asia-Pacific region. The company is responsibly phasing out nuclear energy and coal. Government-mandated phaseout roadmaps have been defined for both of these energy sources. RWE employs around 19,000 people worldwide and has a clear target: to get to net zero by 2040. On its way there, the company has set itself ambitious targets for all activities that cause greenhouse gas emissions. The Science Based Targets initiative has confirmed that these emission reduction targets are in line with the Paris Agreement. Very much in the spirit of the company’s purpose: Our energy for a sustainable life.