Log event stream processing in Flink way
講者: 賴宗智 / Quanta Cloud Technology 時段:15:10~15:50 地點:4F – 國際會議廳 講題: Log event stream processing in Flink way |
摘要:
Logs are one of the most important sources to monitor and reveal some significant events of interest. In this presentation, we introduced an implementation of log streams processing architecture based on Apache Flink. With fluentd, different kinds of emitted logs are collected and sent to Kafka. After having processed by Flink, we try to build a dash board utilizing elasticsearch and kibana for visualization.
講者簡介:
喜歡化繁為簡,以簡馭繁,曾擔任軟體研發工程師及資料科學家的職務,希望有朝一日成為架構師。目前在雲達科技擔任技 術經理,主導 Container 相關技術(如:Mesos, Kubernetes等)的研發工作。經常須面對大量自動化部署的問題及 需求進行研發,故對 CI/CD 有所涉獵並持續學習中。此外也對串流資料處理有濃厚的興趣,正努力學習 Apache Flink 及 Akka。
- Potential Applications using Class Frequency Distribution of Maximal Repeats extracted from Tagged Sequential data
- 用 Jupyter Notebook hold 住一個上線 Spark Machine Learning 專案實戰