Logo of Huzzle

Principal Data Engineer - Azure Cloud Infrastructure

image

Optum

18d ago

  • Job
    Full-time
    Senior & Expert Level
  • Data
    IT & Cybersecurity
  • Dublin

AI generated summary

  • You need strong experience in data pipelines, Azure services, Python, SQL, Spark, Terraform, and APIs. Knowledge of AI/ML, healthcare data, DevOps, and agile methodologies is essential.
  • You will design and manage Azure data engineering infrastructure, ensure data quality, troubleshoot issues, optimize performance, mentor juniors, and collaborate on analytic pipeline standards.

Requirements

  • Extensive hands-on experience developing data pipelines that demonstrate a strong understanding of software engineering principles
  • Excellent knowledge of Azure Services such as – Blobs, Functions, Azure Data Factory, Service Principal, Containers, Key Vault etc.
  • Experience working with both real-time and batch data, knowing the strengths and weaknesses of both and when to apply one over another
  • Experience developing and deploying APIs
  • Expert skills in Python, SQL, Spark
  • Experience designing best practice Data Architecture, Data Quality and Data Security process and implementing those utilizing the latest tools and technologies
  • Experience working with Terraform for cloud infrastructure automation and management
  • Bring innovative ideas to help continuously improve the application portfolio including the design, build, deploy, test and runtime within cloud ecosystems
  • Proven critical thinking skills and persistence to solve complex problems
  • Flexibility to take on various responsibilities in a start-up like team atmosphere
  • Experience using DevOps tools, Git workflow and building CI/CD pipelines
  • Experience working in projects with agile/scrum methodologies
  • Have a solid understanding of Artificial Intelligence and Machine Learning capabilities
  • Work on Proof of Concepts for Big Data and Data Science
  • Understanding of United States Healthcare data
  • Please note you must currently be eligible to work and remain indefinitely without any restrictions in the country to which you are making an application. Proof will be required to support your application.

Responsibilities

  • Design, configure, manage data engineering infrastructure on the cloud (Databricks/Azure DataFactory/Azure Data Lake)
  • Maintain data ingestion and exposure (including via APIs) with high security and availability
  • Ensure data quality standards are established and enforced
  • Build infrastructure using code and repeatable designs in Terraform
  • Maintain Azure cloud infrastructure across prod and non-prod environments
  • Identify and resolve security and compliance vulnerabilities
  • Troubleshoot production support issues post-deployment and come up with solutions as required
  • Identify opportunities for improvement, optimize performance and scalability of existing solutions
  • Setup and maintain related documentation and standards
  • Prioritize and manage work to critical project timelines in a fast-paced environment
  • Partner with Data Science and Product leaders to design best practices and standards for developing and productionalizing analytic pipelines
  • Mentor and train junior members of the team

FAQs

What is the job title for this position?

The job title is Principal Data Engineer - Azure Cloud Infrastructure.

Where is the job located?

The job is located in Dublin, Ireland.

What is the primary responsibility of the Principal Data Engineer?

The primary responsibility is to manage Azure-related data architecture and cloud infrastructure supporting complex risk models, as well as designing, developing, and maintaining data architecture components.

What kind of experience is required for this role?

Extensive hands-on experience developing data pipelines and knowledge of Azure Services, along with expertise in Python, SQL, and Spark is required.

Are there flexible work arrangements available?

Yes, there are flexible work arrangements, allowing individuals to split their monthly work hours between the Dublin office and telecommuting.

What technologies will the Principal Data Engineer work with?

The position will involve working with Azure Cloud services such as Databricks, Azure Data Factory, Azure Data Lake, and infrastructure automation using Terraform.

Is mentoring part of the responsibilities for this role?

Yes, mentoring and training junior members of the team is part of the responsibilities.

What qualifications are preferred for candidates?

Preferred qualifications include experience with DevOps tools, understanding of AI and ML capabilities, and familiarity with agile/scrum methodologies.

Is previous experience in the healthcare field beneficial for this role?

Yes, understanding United States Healthcare data is listed as a preferred qualification.

What are the expectations regarding security and compliance?

The Principal Data Engineer is expected to identify and resolve security and compliance vulnerabilities as part of their responsibilities.

Science & Healthcare
Industry
10,001+
Employees

Mission & Purpose

Optum is a health services and technology company that provides a wide range of solutions to improve healthcare delivery and outcomes. They offer services in healthcare management, data analytics, pharmacy benefit management, and technology solutions to healthcare providers, payers, employers, and government agencies. Optum's ultimate mission is to improve the health system's efficiency and effectiveness, creating a healthier world for everyone. Their purpose lies in collaborating with healthcare partners to address complex challenges, such as improving care coordination, reducing healthcare costs, and enhancing patient experiences. By leveraging data-driven insights and innovative technology, Optum aims to empower healthcare professionals and organisations to deliver high-quality, patient-centered care and drive positive healthcare outcomes for individuals and communities worldwide.

Culture & Values

  • Integrity

    Honour commitments. Never compromise ethics.

  • Compassion

    Walk in the shoes of people we serve and those with whom we work.

  • Relationships

    Build trust through collaboration.

  • Innovation

    Invent the future, learn from the past.

  • Performance

    Demonstrate excellence in everything we do.