Databricks Engineer-Canada

About V4C At V4C, we empower enterprise organizations to unlock the full value of their data. As a fast-growing technology consultancy, our Databricks practice partners with industry leaders to architect modern data platforms, build high-performance data pipelines, and accelerate enterprise AI and analytics initiatives. We are seeking a production-focused Senior Data Engineer with exceptional communication skills and a proven track record of delivering reliable data pipelines. In this role, you will have end-to-end production ownership of our data platforms, ensuring high data quality and robust system performance. The ideal candidate prioritizes practical problem-solving and hands-on execution over high-level architecture. You excel at debugging complex data issues and can clearly communicate technical resolutions using a direct, impact-driven approach. Key Responsibilities • Pipeline Development: Build, optimize, and maintain robust data pipelines using SQL, dbt, and Python. • Data Quality & Testing: Implement rigorous data-quality checks and automated testing frameworks within dbt to proactively catch anomalies. • Production Ownership: Monitor production workflows, manage alerts, and rapidly debug pipeline failures from root cause to permanent fix. • Data Transformation: Write efficient PySpark transformations to process large-scale datasets within our Lakehouse environment. • Collaboration & Communication: Translate complex data issues into clear, concise action plans for both technical teammates and business stakeholders. Required Skills & Experience Core Technical Expertise • SQL & dbt: Expert-level SQL skills and extensive hands-on experience building modular models, snapshots, and custom schema tests in dbt. • Python: Strong proficiency in writing clean, maintainable Python scripts for data manipulation, automation, and API integrations. • PySpark: Decent, practical experience utilizing PySpark for data transformation and basic performance tuning (e.g., handling joins, DataFrame API). • Lakehouse Knowledge: Functional understanding of Delta Lake or similar Lakehouse technologies, including ACID transactions and MERGE operations. Execution & Soft Skills • Strong Communication: Exceptional written and spoken communication skills. Ability to explain problems and results directly and concisely. • Debugging & Problem Solving: Superior troubleshooting skills with a methodical approach to identifying root causes and validating fixes. • Production Mindset: Experience owning pipelines end-to-end, including CI/CD deployment, monitoring, and handling schema drift. Originally posted on Himalayas