Data engineer

Job Description

A Consulting client of Insight Global is hiring for a Senior Data Engineer to join their team. In this role, you will be building data pipeline solutions by designing, adopting, and applying big data strategies and architectures. You must have knowledge of scalable system implementations with a focus on complex data processing and analytics pipelines. You must demonstrate an understanding of data integration, data transformation, data modeling, and data cleansing.

• The Sr. Data Engineer must be able to demonstrate innovative approaches to complex problems which deliver industry-leading experiences for business stakeholders.

• Design and develop a logical data model for a large multi-tenant application with intent to facilitate consistency and efficiency.

• Collaborate with analysts and internal clients to understand the goals of the overall system data architecture, and further develop on business needs/issues.

• Test/troubleshoot problems and conduct root cause analysis.

• Own communications with technical contacts at software vendors to escalate software issues, recommend enhancements, and find sustainable workarounds to issues.

• Work in tandem with analysts and other stakeholders to develop and execute necessary processes and controls around the flow of data to meet data governance standards.

• Verify accuracy of table changes and data transformation processes

• Deliver fully tested code prior to prod-deployment when appropriate.

• Recommend and implement enhancements that address system performance requirements, streamline processes and improve data integrity.

• Create sound technical documentation and train peer developers on this documentation as development completes.

• Additional duties as assigned to ensure company success.

The compensation for this role varies depending on depth and length of experience, and can range from 27-36 LAKH.

We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment regardless of their race, color, ethnicity, religion, sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military or uniformed service member status, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or recruiting process, please send a request to [email protected] learn more about how we collect, keep, and process your private information, please review Insight Global's Workforce Privacy Policy:

Skills and Requirements

Data Engineer with 5+ years of experience

• Strong experience with AWS services (any combination):

o RedShift

o Athena

o S3

o SNS

o SQS

o Lambda

o Glue

o DMS

o EMR

o EC2

o CloudFormation

o StepFunction

o CloudWatch

• SQL

• Strong communication - working across the teams between India and the US - Python/PySpark

• AWS certification