- Working with unstructured data to cleanse and convert it into structured data
- Be responsible for building big data platforms using a combination of frameworks and Big Data processing technologies in Azure Cloud Platform
- Take ownership of platform components and help set the vision and architecture for it.
- Skills with Python, Shell, and SQL as this is a hands-on position writing code
- Knowledge in MPP/NoSQL databases like Elasticsearch, MongoDB, Redis, and Druid in large scale environments
- Ability to evaluate, benchmark, and improve the scalability, robustness, efficiency, and performance of big data platforms and applications
- Great troubleshooting and problem-solving abilities, along with an ability to collaborate cross-functionally in a fast-paced environment
Education & Experience
B.S or M.S in Computer Science, Mathematics, Statistics, or equivalent work experience.
- Strong ability for learning new technologies related to Big Data Processing and Data Management.
- Knowledge in machine learning, feature engineering is a plus
- Experience with Spark and Kafka is a plus
- Experience in using open source framework to build applications is a plus
- Experience in using K8S to build applications is a plus
- Ability to speak both English and Mandarin is a plus