Senior Data Engineer
We're looking for a hands-on, collaborative Senior Data Engineer to join our Data Engineering team. When you get out of bed in the morning, you look forward to building solutions, you love working with data, and you enjoy working as a team.
In this role, you'll have the opportunity to make code decisions and build cloud infrastructure and pipelines that will deliver data solutions to our Food and Beverage and Sports industry clients. You will report to our Director of Data Engineering and help bring best practices to the team while working with a team of Data Engineers.
If you have a git repository, we'd be excited to see it!
- Collaborate with our reporting, analytics, and data science teams to understand data sources and business requirements
- Work within a collaborative team, adhering to Agile best practices, documentation, and knowledge sharing
- Gather, clean, enrich, and transform data to feed internal and external client needs
- Define, build, test, and implement data pipelines, batch and streaming
- Monitor pipeline performance and document infrastructure changes
- Make code decisions and adhere to best practices for ETL and programming
- Contribute to our overall architecture and pipeline design and make contributions to the product roadmap
Qualifications and requirements:
- 3+ years experience programming with Python is required
- 3+ years in an ETL or Data Engineering role, building and implementing data pipelines
- Strong skills with PySpark and SQL with the ability to write efficient queries
- Familiarity with AWS big data services: S3, Redshift, Glue, EC2, Lambda, SageMaker, Dynamo
- Experience with Airflow, Docker, and other open source technologies
- Experience working in a highly collaborative environment -- we do Agile using sprints, planning, retro, etc.
- Excited and willing to learn new things
- Background in computer science, engineering, mathematics, related field, or equivalent work experience
Nice to have:
- Interest or experience in DevOps and CI/CD
- Experience with building data lake solutions
- Experience with Netezza and Data Stage