Big data development engineer- Streaming computing direction Undefined

The real-time computing direction of the ByteDance recommendation architecture team is responsible for the design and development of the real-time computing system of the product recommendation system architecture of ByteDance’s over 1 billion users, ensuring system stability and high availability abstracting general real-time computing systems and building unified recommendations Feature middle platform implements flexible and scalable high-performance storage systems and computing models, and implements advanced real-time data systems such as deduplication, counting, and feature services for recommendation services 1. Design and implement reasonable real-time (streaming) for large-scale recommendation systems (formula computing) data system 2. Design and implement flexible, scalable, stable, high-performance storage systems and computing models 3. Trouble-shooting of production systems, design and implement necessary mechanisms and tools to ensure the stability of the overall operation of the production system 4. Create industry-leading distributed systems such as streaming computing frameworks to provide reliable infrastructure for massive data and large-scale business systems.

1. Have an in-depth understanding of streaming computing systems, have TB-level Flink real-time computing system development experience in a production environment, and have an in-depth understanding of Flink DataStream, FlinkSQL, Flink Checkpoint, Flink State and other modules, experience in reading Flink source code is preferred 2. Familiar with common message queue principles and application tuning, experience in reading source code of Kafka, Plusar, RocketMQ and other projects is preferred 3. Familiar with Java, C++ , Scala, Python and other programming languages, and have strong coding and trouble-shooting abilities 4. Be willing to challenge problems with no obvious answers, have a strong enthusiasm for learning new technologies, and have PB-level data processing experience as a bonus 5. Have data Lake development experience, familiar with at least one data lake technology such as Hudi, Iceberg, DeltaLake, etc., and experience in reading source code is preferred 6. Candidates familiar with experience in other big data systems are preferred, such as YARN, K8S, Spark, SparkSQL, Kudu, etc. 7. Storage experience is preferred System experience is a plus, HBase, Casscandra, RocksDB, etc.