Senior Data Engineer

Job Description Roles & Responsibilities Accountabilities Treat the data platform as a core product, ensuring scalability, reliability, and innovation. Collaborate with Architecture on the development and prioritization of the Databricks based platform roadmap. Integrate new services such as Delta Lake, Unity Catalog, and DLT into the platform ecosystem. Prepare and maintain documentation for Databricks workflows, CI/CD pipelines, and infrastructure automation. Partner with stakeholders to resolve platform related technical issues and identify areas for improvement. Implement internal process improvements: automate ingestion with Autoloader, optimize streaming with Structured Streaming, and enforce data quality with DQX. Adopt Agile and DevOps practices, ensuring CI/CD pipelines and Terraform based infrastructure are production ready. Provide operational support for Databricks clusters, jobs, and pipelines. Perform fallout & impact analysis, ensure recovery of impacted transactions, and provide business impact reports for production incidents. Propose new ideas and technologies to foster continuous improvement and enhance user experience. Define, share, and drive adoption of best practices for DataOps and cloud native data engineering. Desired Candidate Profile Education & Experience 6+ years of experience as a Data/Platform Engineer, with proven expertise in Databricks and modern data lakehouse architectures. Graduate degree in Computer Science, Information Systems, or a related quantitative field. Hands on knowledge of PySpark, Structured Streaming, Autoloader, DQX, and Databricks workflows. Strong experience with CI/CD pipelines, Terraform, DevOps, and DataOps practices. Ability to troubleshoot and solve complex problems across distributed data systems. Experience working with Linux, shell scripting, public cloud platforms, and big data technologies. Strong communication skills and experience collaborating with cross functional teams in Agile environments. Ability to work independently with minimal supervision and eagerness to learn new technologies. Company Industry AirlinesAviation Department / Functional Area IT Software Keywords Senior Data Engineer Get real-time job updates only on our App