Data Engineer- INTL India

Job Description

A Consulting client of Insight Global is hiring for a Senior Data Engineer to join their team. This Data Engineer will work on a product development team responsible for data lakes and architecture of a healthcare application in an AWS environment. Responsibilities include:

• Provide innovative AWS data lake architecture solutions and recommendations

• Setup, configure, and troubleshoot AWS data migration services

• Catalog, setup, configure, and troubleshoot Glue or DBT - Perform heavy ETL work - Create tables and manage a Hudi database

• Provide SQS development

• Work extensively with Event Bridge Schedule

• Understand creation of tables/views/MV's, and provide performance and tuning within RedShift

• Work within Aurora MySQL database management:

• Develop by writing Stored Procedures

• Setup Read replication

• Troubleshoot and monitor AWS resources for performance improvement

This is a fully remote role, ideally starting ASAP. The pay rate for this role varies depending on level and years of experience and will range from 27-36 LAKH.

We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment regardless of their race, color, ethnicity, religion, sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military or uniformed service member status, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or recruiting process, please send a request to [email protected] learn more about how we collect, keep, and process your private information, please review Insight Global's Workforce Privacy Policy:

Skills and Requirements

• 7-10+ years of data engineer experience

• Strong hands on engineering experience designing and operating Data Lakes on AWS

• AWS Glue or DBT experience

• Setup and configuration of Glue Catalog/metadata management

• Troubleshooting schema, crawler, and catalog issues

• Strong developer skills for building scalable ETL/ELT pipelines including job setup, configuration, and orchestration

• Amazon Redshift experience with development and operations

• AWS SQS - designing and -developing message-driven data pipelines

• Amazon Event Bridge Schedule experience for event-driven orchestration of data workflows and scheduling and dependency management.

• Experience owning end-to-end monitoring and optimization of AWS-based data pipelines, including performance, cost, and reliability, with strong root-cause analysis, proactive logging, alerting, and continuous performance improvements. - AWS data migration services

• Apache Hudi database experence

• AWS Aurora MySQL database management

• Python or Scala experience