Nothing can limit your
dreams when you are at
Zoft

Work Location : Kochi, Kerala

Key Responsibilities

• Design, build, and maintain ETL/ELT pipelines for structured and unstructured data in AWS.
• Work with AWS services (e.g., S3, Glue, EMR, Lambda, Kinesis, Step Functions, Redshift, Athena, DynamoDB, RDS) to manage data ingestion, transformation, and storage.
• Ensure data quality, governance, and security across the full lifecycle.
• Enable real-time and batch data processing to support AI-driven workflows.
• Collaborate with AI/ML teams to prepare datasets for model training, inference, and fine-tuning.
• Optimize data infrastructure for scalability, cost-efficiency, and performance.
• Implement CI/CD pipelines and best practices for data engineering in a cloud-native environment.
• Monitor and troubleshoot data pipelines to ensure high availability and reliability.

Required Skills & Qualifications

• 3–7 years of experience in data engineering (adjust based on seniority).
• Strong programming skills in Python, SQL, and/or Scala/Java.
• Expertise in AWS cloud ecosystem (S3, Glue, EMR, Redshift, Athena, Kinesis, Lambda, etc.).
• Experience with data pipeline orchestration tools (Airflow, Step Functions, Dagster, or similar).
• Proficiency with big data frameworks (Spark, Hadoop, or Flink).
• Familiarity with data modeling, warehousing, and schema design.
• Solid understanding of data governance, lineage, and security (IAM, Lake Formation, encryption).
• Experience with real-time streaming data (Kafka, Kinesis, or equivalent).
• Knowledge of DevOps practices (Terraform/CloudFormation, CI/CD, Git, Docker).