to design, develop, and maintain scalable data pipelines and systems that empower business insights and analytics. The ideal candidate will work closely with data scientists, analysts, and software engineers to ensure efficient data flow and high-quality data across the organization.
Key Responsibilities
Design, build, and maintain
ETL/ELT pipelines
to collect, process, and transform large volumes of structured and unstructured data.
Integrate data from various sources including APIs, databases, and third-party services.
Develop and optimize
data models
, warehouses, and lake architectures for analytics and reporting.
Ensure
data quality, reliability, and security
across all systems.
Work with
cloud platforms
such as AWS, Azure, or GCP to deploy and monitor data infrastructure.
Collaborate with
data analysts, data scientists, and business teams
to deliver accessible, high-performance datasets.
Implement
best practices in data engineering
, including version control, testing, and CI/CD for data pipelines.
Monitor data flows and troubleshoot pipeline or performance issues.
Skills & Qualifications
Education
Bachelor's or Master's degree in Computer Science, Information Technology, Data Engineering, or related field.
Technical Skills
Programming:
Python, SQL, Scala, or Java.
Data Processing Tools:
Apache Spark, Kafka, Airflow, Hadoop, or similar.
Database Systems:
Experience with relational (PostgreSQL, MySQL) and NoSQL (MongoDB, Cassandra) databases.