KData Inc.
MLOps Engineer (Databricks Specialist)
This is a remote position. Position Overview We are seeking a highly skilled MLOps Engineer with deep expertise in the Databricks ecosystem to join our data team for a critical 6-month initiative. In this role, you will bridge the gap between Data Science and Data Engineering, focusing on automating, scaling, and managing the end-to-end lifecycle of our machine learning models. The ideal candidate will have a strong foundation in software engineering and production-grade DevOps practices, specifically optimized for machine learning pipelines (MLOps) within cloud-native Databricks environments. Key Responsibilities • Pipeline Automation: Design, build, and maintain robust CI/CD and MLOps pipelines for machine learning model training, evaluation, deployment, and batch/real-time scoring using Databricks Jobs and Workflows. • Model Lifecycle Management: Implement and manage experiment tracking, model registration, versioning, and environment promotion policies using MLflow and Unity Catalog . • Infrastructure & Optimization: Optimize Databricks clusters and computational workloads for ML training and inference to ensure both cost-efficiency and high performance. • Data & Feature Engineering: Collaborate with data engineers to build and maintain scalable feature pipelines utilizing Databricks Feature Store / Delta Lake. • Monitoring & Observability: Establish proactive monitoring frameworks to track model performance, data drift, concept drift, and system health in production environments. • Collaboration: Partner closely with Data Scientists to transition proof-of-concept (PoC) code into scalable, production-ready ML products. Requirements Required Qualifications • Experience: 6+ years of professional experience in Software Engineering, Data Engineering, or DevOps, with at least 3+ years dedicated to MLOps . • Databricks Mastery: Hands-on experience architecting ML workflows within Databricks (including MLflow, Unity Catalog, Delta Lake, and Databricks Repos). • Core Languages: Advanced proficiency in Python and SQL . Strong skills in PySpark are highly desired. • CI/CD & DevOps: Proven experience building automated deployment pipelines using tools such as GitHub Actions, GitLab CI, Jenkins, or Azure DevOps. • Cloud Infrastructure: Familiarity with major cloud environments (AWS, Azure, or GCP) and cloud data infrastructure. • Education: Bachelor’s degree in Computer Science, Data Science, Engineering, or equivalent practical experience. Preferred (Nice-to-Have) Skills • Active Databricks certifications (e.g., Databricks Certified Machine Learning Professional ). • Experience with Infrastructure as Code (IaC) tools like Terraform. • Familiarity with containerization (Docker, Kubernetes). • Exposure to LLMOps or serving GenAI models on Databricks. Why Work With Us? • 100% Remote: Enjoy the flexibility of a fully remote setup. • Impactful Work: Own a dedicated stream of work on high-priority ML initiatives over the next 6 months. • Cutting-Edge Stack: Work on modern, clean Databricks infrastructure. Originally posted on Himalayas
