Developer

Skill: Spark, Scala Developer


Proven big data experience, either from an implementation or a data science prospective. Knowledge of Scala and Hive which requires coding experience using parquet 4. Advanced SQL knowledge requiring writing SQL queries. Experience having worked with GCP data lake. Experience in working in an Agile environment. Expert knowledge of at least one big data technology such as Spark, Hadoop, or Elasticsearch. A strong coding background in either Java, Python or Scala. Experience with testing libraries of common programmingfor example scalatest (or equivalent packages for other languages). Knows the difference between different test types (unit test vs integration test for example). Can explain unit testing and how they have been automated. Experience of building data processing pipelines for use in production hands off batch systems, including either traditional ETL pipelines and/or analytics pipelines.

#LI-NS2