A degree in Computer Science, Information Systems, Engineering, or Mathematics preferred.
3+ years of professional software development experience with data engineering and data warehouse projects hosted on-prem and on-cloud environments
2+ Years of Apache Spark programming experience using Scala & Spark REQUIRED with Hadoop platform including Spark/Scala, SparkSQL, Hbase, Hive, Sqoop and HDFS in multi-tenant environments
Must have hands on experience of Amazon Redshift Architecture AWS Redshift experience for DB design, data ingestion and writing advanced SQL Queries
Must have good understanding of ETL concepts and able to write ETL code for Big Data
Familiarity with HBase or other NoSQL technologies
Familiarity with HiveQL to perform queries, summaries, and analysis of the stored data in Hive
Prior Experience with development of Data Ingestion Pipeline and services on Hadoop Eco system.
Experience writing automated tests using tools like Junit, Scala Test
Working knowledge of Hadoop-based infrastructure and security configuration
Strong analytical and critical thinking skills
Experience in Agile software development.
Ability to prioritize and multitask
Deadline and detail-oriented
Java Programming Experience is a plus
Data lake experience is a pl