Databricks Architect

Role: Databricks Architect

Location: New York, NY

Duration: Contract

Key Responsibilities

As an Architect in the Databricks practice, you will leverage your extensive experience in architecture design and expertise in data engineering technologies to craft innovative solutions that meet our clients' data needs.

You are a leader in your field, ready to experiment and drive projects forward.

Key Responsibilities

Design and implement scalable data architectures using Databricks, with hands-on experience in specific platform features such as Delta Lake, Uniform (Iceberg), Delta Live Tables, and Unity Catalog.

Lead and mentor engineering teams, fostering a culture of learning and innovation, while driving best practices in data management and performance optimization.

Engage with clients to understand their business challenges and deliver solutions that align with their goals, utilizing Databricks' capabilities to enhance outcomes.

Demonstrate hands on technical leadership in designing and developing different components of a Data Lakehouse platform to meet client s business needs.

Requirements

8-10+ years of experience in data engineering or architecture, including 4+ years of direct experience with Databricks and specific products like Delta Lake, Delta Live Tables, and Unity Catalog.

Deep expertise in Big Data Platforms and Cloud Data Warehouses.

Strong understanding of Databricks platform technical architecture on public cloud platforms like AWS, Azure, Google Cloud Platform etc. with experience in standing up scalable Databricks environments from scratch.

Strong expertise in building industry standard, extensible data models for Silver and Gold layers of the Lakehouse solution following design standards like Star Schema, Data Vault etc.

Advanced proficiency in Object-Oriented programming languages (like Java, Python, PySpark) and NoSQL Databases including performance tuning and optimization of complex data pipeline solutions.

Experience with Container Management Systems and AI/ML platforms.

Strong skills in streaming data ingestion and modern data workflows.

Experience in delivering large scale migration, modernization initiative from legacy Data Warehouses to Databricks Lakehouse platform.

Exposure to Databricks consumption estimates calculation considering different Data & AI pipeline workloads for multiple environments.

Exposure to building Disaster Recovery (DR) solution strategy and implementation for a large Databricks platform is a big plus.

An ideal candidate will have Databricks Data Engineering Professional ceritification completed with multiple complex Databricks project delivery experience.