Logo of Huzzle

Lakehouse

image

Virtusa

9d ago

  • Job
    Full-time
    Senior Level
  • Data
    IT & Cybersecurity
  • Chennai

AI generated summary

  • You should have 6 years IT experience, 4 in data lakehouses (Apache Iceberg, Delta Lake), Python, PySpark, SQL, AWS, BASH, and Agile. Kafka, Mulesoft, and healthcare data knowledge are pluses.
  • You will implement lakehouse architecture, develop ETL pipelines, optimize query performance, enforce data governance, and collaborate with teams to enhance data accessibility.

Requirements

  • 6 Years overall IT experience with minimum 4 years of work experience in below tech skills
  • Work experience in data lakehouse in any of these Apache Iceberg Databricks Delta Lake
  • Proficient in Python scripting and PySpark for data processing tasks
  • Strong SQL capabilities with hands on experience managing big data using ETL tools
  • Experience with the AWS cloud platform and its data services
  • Skilled in BASH Shell scripting
  • Preferred Experience with Kafka and Mulesoft API
  • Understanding of healthcare data systems is a plus
  • Experience in Agile methodologies
  • Strong analytical and problem solving skills
  • Effective communication and teamwork abilities

Responsibilities

  • Implement lakehouse architecture to integrate and optimize data storage and analytics processes
  • Develop ETL pipelines for efficient data ingestion transformation and loading from various sources
  • Optimize query performance by analyzing and tuning data workflows to minimize latency
  • Implement data governance policies to ensure compliance and protect sensitive data
  • Collaborate with data teams to gather requirements and deliver solutions that enhance data accessibility

FAQs

What is the primary responsibility of a Lakehouse Developer?

The primary responsibility of a Lakehouse Developer is to design, implement, and maintain data lakehouse solutions that integrate data storage and analytics, while collaborating with data engineers and analysts to enhance data workflows and ensure data integrity.

What minimum qualifications are required for this position?

The minimum qualifications include 6 years of overall IT experience with at least 4 years of work experience in data lakehouse technologies, Python scripting, PySpark, SQL, AWS cloud platform, BASH Shell scripting, and familiarity with Kafka and Mulesoft API.

What technology skills are essential for this role?

Essential technology skills include experience with data lakehouse solutions (such as Apache Iceberg and Databricks Delta Lake), proficiency in Python and PySpark, strong SQL capabilities, and experience with ETL tools and the AWS cloud platform.

Is knowledge of healthcare data systems necessary for this role?

While not mandatory, having an understanding of healthcare data systems is considered a plus for this position.

What methodologies should the candidate be familiar with?

The candidate should have experience in Agile methodologies.

What are the responsibilities related to data governance in this role?

Responsibilities related to data governance include implementing data governance policies to ensure compliance and protect sensitive data.

Is experience with data ingestion and transformation part of the job responsibilities?

Yes, developing ETL pipelines for efficient data ingestion, transformation, and loading from various sources is a key responsibility of the Lakehouse Developer.

What kind of team environment does Virtusa promote?

Virtusa promotes a collaborative team environment that values teamwork, quality of life, and both professional and personal development.

Does Virtusa offer equal employment opportunities?

Yes, Virtusa was founded on principles of equal opportunity for all and does not discriminate on the basis of race, religion, color, sex, gender identity, sexual orientation, age, disability, national origin, veteran status, or any other basis covered by law.

What qualities does Virtusa look for in candidates?

Virtusa looks for candidates with strong analytical and problem-solving skills, effective communication and teamwork abilities, and the potential to nurture new ideas and foster excellence.

Business transformation that lasts starts with Engineering First.

Technology
Industry
10,001+
Employees
1996
Founded Year

Mission & Purpose

Virtusa Corporation provides digital engineering and technology services to Forbes Global 2000 companies worldwide. Our Engineering First approach ensures we can execute all ideas and creatively solve pressing business challenges. With industry expertise and empowered agile teams, we prioritize execution early in the process for impactful results. We combine logic, creativity and curiosity to build, solve, and create. Every day, we help clients engage with new technology paradigms, creatively building solutions that solve their most pressing business challenges and move them to the forefront of their industry.