Data Engineer_AI

Job Description

Develop and maintain ETL/ELT pipelines using Databricks.

Write efficient Pyspark & Python Scripts for large-scale data processing.

Implement data models, transformations and CI/CD pipelines for automation.

Develop reusable data engineering frameworks, libraries and templates for ingestion, transformation, validation and publishing.

Ensure Data Quality, Reliability, and Performance across pipelines and shared frameworks.

Design, develop, test and deploy Power BI scripts and generate visually appearing and insightful reports.

Integrate Power BI reports into Applications to provide enhanced business capabilities.

Build data models and develop dashboards using Power BI desktop.

Work with Reliability Engineers to design and develop Power BI data Visualizations.

Proficient in using DAX expressions and Power Queries for developing and optimizing Power BI dashboards and reports.

Knowledge of Azure Cloud Data Engineering Tools.

Analysis of Validation Test Results.

Development of Interfaces for Measurement and Simulation.

Development and Maintenance of Test Automation system related Tools.

Support Downstream consumers such as analytics, reporting, testing and Reliability teams.

Desired Skills

Good knowledge of data engineering concepts, ETL/ELT pipelines, and data lifecycle management.

Excellent understanding of data modeling, data transformation, and dashboarding concepts for analytics and reporting.

Proficiency in programming languages including Python and PySpark, with experience in large-scale data processing.

Strong knowledge of data visualization and reporting using Microsoft Power BI including DAX expressions and Power Query.

Familiarity with cloud-based data engineering tools, preferably Microsoft Azure services.

Basic knowledge of electronics and industrial/validation systems is an added advantage.

Education & Experience

Bachelor’s or Master’s degree in Electronics or Electrical Engineering or Computer Science Engineering with 2 to 5 years of experience.

Hands-on experience in building and maintaining data pipeline solutions using Databricks.

Exposure to data engineering concepts including ETL/ELT workflows, data processing, and automation.

Soft Skills

Team player with a proactive mind set

Good communication skills

Willing to work in an international team with diverse cultures

Openness for technical discussions, assessments and reviews

Btech/Mtech

3-6 Years