Sr Data Engineer (AWS | Multi-Cloud | Databricks)
- Location
- Bengaluru, KA, India
- Work mode
- On-site
- Employment type
- Full-time
About the job
Experience: 5+ Yrs
Job Description – Data Engineer (AWS | Multi-Cloud | Databricks)
Experience: 5+ Years Role: Senior Data Engineer Primary Cloud Platform: AWS Secondary Cloud Platforms: GCP and Microsoft Azure Core Technology: Databricks, Apache Spark, PySpark, SQL, Data Engineering
Job Overview We are looking for an experienced Senior Data Engineer with 5+ years of hands-on experience in designing, developing, and implementing scalable data engineering solutions across cloud platforms. The ideal candidate should have strong expertise in AWS as the primary cloud platform , along with working knowledge and exposure to data engineering services across Google Cloud Platform (GCP) and Microsoft Azure. The candidate must possess excellent data engineering fundamentals, strong programming and problem-solving skills, and very good hands-on experience with Databricks and Apache Spark . The role involves building modern data pipelines, implementing data lakes and lakehouse architectures, optimizing large-scale data processing workloads, and developing reliable, scalable, and cost-efficient data platforms.
Key Responsibilities
•
- Design, develop, and maintain scalable end-to-end data pipelines using AWS and Databricks.
- Build and optimize batch and real-time data processing solutions using Apache Spark, PySpark, and SQL.
- Design and implement modern data lake and lakehouse architectures using S3, Delta Lake, Apache Iceberg, and Databricks.
- Work with AWS data engineering services for data ingestion, transformation, orchestration, storage, cataloging, and analytics.
- Develop ETL/ELT workflows integrating structured, semi-structured, and unstructured data from multiple sources.
- Implement data transformation, data quality, validation, and reconciliation frameworks.
- Optimize Spark jobs, SQL queries, cluster configurations, and distributed processing workloads.
- Implement workflow orchestration using Airflow, AWS Step Functions, Databricks Workflows, or equivalent services.
- Work with streaming technologies and services for real-time data ingestion and processing.
- Develop reusable frameworks, automation scripts, and standardized data engineering components.
- Implement security, access control, encryption, monitoring, logging, and governance across data platforms.
- Collaborate with architects, analysts, data scientists, and business stakeholders to translate requirements into scalable technical solutions.
- Participate in architecture discussions, technical design, code reviews, and performance optimization initiatives.
Required Experience & Qualifications
•
- 5+ years of experience in Data Engineering.
- Strong hands-on experience with AWS data engineering services.
- Very good practical experience with Databricks and Apache Spark.
- Strong programming skills in Python, PySpark, and SQL.
- Practical exposure to at least one of GCP or Azure, with familiarity with the data engineering ecosystem of both.
- Experience designing and implementing end-to-end data pipelines.
- Understanding of modern data lakehouse architecture and distributed data processing.
- Experience working with large-scale datasets and performance optimization.
Good to Have
•
- Experience working on multi-cloud data engineering architectures.
- Exposure to AWS, Azure, and GCP migrations.
- Experience with Apache Kafka, Amazon MSK, or other streaming platforms.
- Knowledge of Apache Iceberg, Delta Lake, and open table formats.
- Exposure to data governance, lineage, and metadata management.
- Knowledge of DevOps practices and CI/CD automation.
- Experience with Infrastructure as Code (Terraform / CloudFormation).
- Exposure to Generative AI data pipelines and vector databases.
Ideal Candidate Profile The ideal candidate should be a hands-on Data Engineer who can independently design, develop, troubleshoot, and optimize complex data engineering pipelines. We are specifically looking for someone who:
•
- Has strong practical expertise in AWS data engineering services.
- Has very good hands-on experience with Databricks and PySpark.
- Understands data engineering ecosystems across AWS, GCP, and Azure.
- Has strong fundamentals in distributed data processing, SQL, Python, and data architecture.
- Can work independently on end-to-end data engineering implementations.
- Can contribute to technical architecture and engineering best practices.
From Cloudattack Edutech's careers page (Keka), checked 5 Nov 2026.
How to apply
- Select Apply on cloudthat.com. It opens Cloudattack Edutech's own application form in a new tab, run on Keka.
- Fill in the form and upload your résumé there. Jobiwork never sees your application.
- Cloudattack Edutech will reply to you directly. No real employer asks you for money to apply.
Posted 5 Nov 2026 · checked 5 Nov 2026 · Report this job