Data Engineer

Job Description

  • Working with unstructured data to cleanse and convert it into structured data
  • Be responsible for building big data platforms using a combination of frameworks and Big Data processing technologies in Azure Cloud Platform
  • Take ownership of platform components and help set the vision and architecture for it.

Key Qualifications

  • Skills with Python, Shell, and SQL as this is a hands-on position writing code
  • Knowledge in MPP/NoSQL databases like Elasticsearch, MongoDB, Redis, and Druid in large scale environments
  • Ability to evaluate, benchmark, and improve the scalability, robustness, efficiency, and performance of big data platforms and applications
  • Great troubleshooting and problem-solving abilities, along with an ability to collaborate cross-functionally in a fast-paced environment

Education & Experience

 B.S or M.S in Computer Science, Mathematics, Statistics, or equivalent work experience.

Additional Requirements

  • Strong ability for learning new technologies related to Big Data Processing and Data Management.
  • Knowledge in machine learning, feature engineering is a plus
  • Experience with Spark and Kafka is a plus
  • Experience in using open source framework to build applications is a plus
  • Experience in using K8S to build applications is a plus
  • Ability to speak both English and Mandarin is a plus