Senior Data Engineer

Team Segment : Music

Team Information

The main aim of the Data Platform team is leveraging data to provide insights and evidence to help stakeholders take meaningful business actions. With the help of key findings and reports from the team, the Business Leader will be all set to make informed decisions.

In addition to supporting data-driven decision making, the team is also responsible for the development and ongoing operation of personalized recommendation systems. By integrating machine learning models—such as those powered by AWS Personalize—into our data platform, we aim to enhance user experiences through intelligent and scalable content recommendations, including music recommendation features in the near future.

If you’re passionate about leveraging data to drive business and deliver personalized user experiences, we want to hear from you.

Responsibilities

Design and implement ETL/ELT processes to integrate data from a variety of internal and external sources into an analytical-based data model to support business analytics or product development.

Develop procedures for tracking data quality, completeness, redundancy, and improvement.

Work closely with data analysts and development teams to implement solutions that enable them to do their jobs.

Build and optimize recommendation pipelines, including implementing models using AWS Personalize for personalized content ranking and user-item interaction prediction.

Collaborate with cross-functional teams to explore and prototype personalized recommendation systems, especially in the context of music content delivery.

Contribute to strategic initiatives involving machine learning-driven personalization systems, with a focus on increasing product engagement through recommendation accuracy.

Requirements

4+ years of experience in building and operating large-scale distributed systems or applications.

Strong programming skills in one or more of the following languages: Python / Scala / PySpark / SparkSQL.

Experience in ETL or ELT (designing, coding, and tuning data processes from various sources).

Experience in implementing machine learning algorithms.

Hands-on experience with AWS Personalize or similar recommendation engines.

Experience with cloud solutions like AWS or GCP.

Experience with tech stack tools such as Airflow / Jenkins.

Strong interest or prior experience in music recommendation systems or personalization use cases.

Team-oriented, open-minded, communicative and eager to learn.

A proactive, data-driven and logical mindset.

Nice to Have

Experience in container technology, such as Docker and Kubernetes.

Experience in backend system design and API development.

Experience in evaluating recommendation systems through offline metrics and online A/B testing.

Originally posted on Himalayas