Data Engineering Manager, Azure Databricks

NexTech Solutions

Pay not stated in the posting

, · Remote

Posted 3 days ago · Confirmed live 16 hours ago

Skills this job asks for

  • Python
  • Apache Kafka
  • SQL
  • Azure
  • Azure Event Hubs
  • Azure Data Lake
  • Databricks
  • Unity Catalog
  • Data Warehousing
  • AWS Kinesis
  • Data Modeling
  • Confluent

Tailor your resume for this job

See which of these skills you have, and tailor your resume for this job.

Sign up free

Tailor your resume for this job

Start from your profile and shape your resume to this role.

At a glance

Type
Not stated
Level
Lead
Experience
7+ years
Work model
Remote
Schedule
Not stated
Education
Not stated
Benefits
Not stated

Description

To Apply for this Job Click Here Job Title: Data Engineering Manager, Azure Databricks Location: Fully Remote (on-site in Dallas, Memphis, Cleveland, or Pittsburgh is optional) Employment Status: Applicant must be currently authorized to work in the U.S. without the need for employment-based sponsorship, now or in the future, including: H-1B, L-1, TN, O-1, E-3, H-1B1, F-1, J-1, OPT, CPT, or any other employment-based visa programs. Position Overview Lead and develop a team of data engineering professionals while overseeing the design, development, and ongoing optimization of the Azure Databricks platform and enterprise data warehouse, supporting business analytics, reporting, and the evolution toward real-time, event-driven data integration. Key Responsibilities •  Own source-system ingestion, Delta Lake pipelines, Medallion Architecture, and enterprise data warehouse design, development, and operations within Azure Databricks. •  Provide hands-on technical leadership across Azure Databricks, Azure Data Lake Storage, Unity Catalog, PySpark, Python, and SQL including architecture reviews, code reviews, performance optimization, and technical mentoring. •  Support transition from batch processing to continuous, near-real-time data ingestion and processing using Azure Event Hubs and Structured Streaming to meet demanding service-level requirements within a highly regulated environment. •  Monitor and optimize data warehouse reliability, processing performance, scalability, and cost efficiency while maintaining the integrity and availability of critical business data. •  Direct production incident response, troubleshoot complex pipeline and data processing issues, and establish proactive monitoring, documentation, and knowledge-sharing practices. •  Establish and enforce engineering standards for code quality, testing, deployment, security, and documentation, ensuring reliable, governed data operations within a highly regulated healthcare environment. •  Collaborate with Data Architecture, Analytics Engineering, QA, and Data Governance on Gold-layer design, data validation, lineage, metadata, and observability. •  Support internal business requirements and external partner data services through the development, enhancement, and integration of data pipelines. •  Lead, coach, mentor, and develop a team of Data Engineers, establishing clear accountability, managing performance, and fostering a collaborative, high-performing engineering culture prepared for significant growth. Qualifications •  7+ years of progressive data engineering experience, including 2+ years of team leadership or people management. •  Deep, hands-on Databricks expertise, including medallion architecture, Delta Lake, Unity Catalog, and production pipeline development and operations. •  Strong enterprise data warehouse experience encompassing architecture, data modeling, transformation logic, and production operations. •  Proficiency in PySpark, SQL, and Python with the technical depth to review code, evaluate architectural decisions, and mentor engineers. •  Proven experience with Databricks, Azure Data Lake Storage, and Azure Event Hubs. •  Strong experience developing and operating data pipelines, preferably with Azure Event Hubs or comparable technologies such as Kafka, Kinesis, or Confluent Cloud. •  Demonstrated ability to lead and develop engineers, manage competing priorities, and maintain accountability in a high-volume production environment. •  Strong communication, organizational, and cross-functional collaboration skills with the ability to work effectively across technical, business, partner, and client teams.   To Apply for this Job Click Here Billy. Wingo

About NexTech Solutions

NexTech Solutions

See all 16 open roles at NexTech Solutions