1. Develop in-house tools or use 3rd party tools for data acquisition, processing, modeling, monitoring.
2. Align and optimize data pipeline architecture for data collection, cleaning, storage, processing and analytics with business requirements
3. Develop and integrate scalable, reliable, maintainable web-service backend systems with current data processing framework to represent data insights.
4. Identify ways to improve data reliability, efficiency and quality
5. Document architecture design and features implementation.
6. Deploy sophisticated analytics programs, machine learning and statistical methods to find hidden patterns using data
7. Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.
Criteria:
1. Experience in hands-on ETL task design, components and modules development of data process
2. Ability to build services in Linux/Unix environments and familiar with shell script
3. Experience in Scala or Golang development, or any programing language for backend development
4. Familiar with Hadoop ecosystem, such as Spark, Hadoop
5. Experience of Cassandra, Clickhouse, Redis or any other database performance tuning
6. Experience in operating large scale distributed systems or applications
7. Positive, can-try attitude, good communication skill to cowork with talent team members
Plus:
1. Experience as a data engineer or in a similar role
2. Experience in performance tuning via algorithm or architecture improvements
3. Experience in kubernetes as a devops engineer
4. Experience with Kafka, Prometheus, Grafana
5. Experience in unit test, integration test, security test, stress test
6. Advanced database schema design knowledge, data sharding, replica usage, etc.
7. Numerical and analytical skills
8. Experience in implementing data mining and machine learning algorithm
Interview Process:
1. Coding on this platform: https://www.codingame.com/training/community/apple-tree One might want to test the environment before the interview. Interviewees can bring their own laptops or use the MAC provided by GT.
2. The tech lead of the team will introduce the current software structure and then ask some technical questions.
3. Interview with the BU head.
因應疫情調整
