We are looking for a Big Data / Data Engineer to join our growing team. In this role, you will be responsible for the design and development of our big data processing engine a heavy ETL pipeline handling complex logic including geo-location analysis, running across both cloud and on-prem environments.
Requirements:
-
4+ years of experience in Data Engineering
-
2+ years of hands-on experience with Scala (must) or Java (optional)
-
2+ years of experience with SQL and NoSQL DBs
-
Hands-on experience with Spark & PySpark (must)
-
Experience with Airflow (must)
-
Experience with AWS and/or GCP
-
Experience with Kubernetes (k8s) and on-prem deployments
-
Solid understanding of distributed systems design, pipeline architecture, and partitioning strategies
-
Deep understanding of Agile principles, practices and values
-
Solid understanding of design patterns, software development techniques, and clean code practices
Requirements:
Advantage:
-
Experience with Apache Iceberg, Helm Charts, or data catalog tools
-
Experience with Spring Framework
-
Experience with microservice architecture
-
Experience working with DevOps and CI/CD practices
-
Customer-facing experience or comfort working directly with clients
-
Bachelor's, BSc or Master's degree in Computer Science or Engineering
