Senior Data Engineer (Databricks)

Job Title: Senior Data Engineer (Databricks)

Location: Chicago, IL / Remote
Job Type: Full-Time

Job Summary

We are seeking a highly skilled Senior Data Engineer with strong expertise in Databricks, Apache Spark, SQL, and cloud-based ETL pipelines . The ideal candidate should possess strong software engineering fundamentals and experience building scalable, high-performance data engineering solutions in cloud environments.

Required Skills

• Hands-on expertise in:
  • Databricks
  • Apache Spark
  • SQL
  • Cloud-based ETL pipelines
  • CI/CD implementation
• Strong experience building scalable and distributed data engineering solutions
• Strong software engineering and problem-solving skills
• Experience with cloud platforms and modern data architectures
• Knowledge of performance optimization, monitoring, and cost management

Key Responsibilities

• Design, develop, test, and deliver high-quality data engineering solutions aligned with business needs
• Build, optimize, and maintain scalable data pipelines using Databricks and Apache Spark
• Architect distributed data processing frameworks ensuring reliability, scalability, and performance
• Troubleshoot and resolve complex application and data issues through root cause analysis
• Implement code reviews, engineering standards, and best practices to ensure maintainability and quality
• Develop and support CI/CD pipelines, automated testing, and monitoring solutions for cloud data platforms
• Optimize cloud data platform performance and operational costs
• Mentor junior and mid-level engineers and contribute to continuous engineering improvements

Preferred Skills

• Experience with Azure, AWS, or GCP data platforms
• Exposure to Delta Lake, Data Lakes, or Lakehouse architectures
• Familiarity with Agile/Scrum development methodologies