Stream Processing with Beam and Google Cloud Dataflow

講者: Eric Anderson / Product Manager, Google
地點:4F – 國際會議廳 Conference Hall
講題:Stream Processing with Beam and Google Cloud Dataflow


Apache Beam, a new distributed processing framework from Google that’s now incubating at the ASF,
provides a single programming model (in multiple languages) for data processing that unifies
streaming and batch, on-prem and cloud, big and small data. Every data job can be written in Beam
and then deployed anywhere at run-time. Google Cloud Dataflow is a managed service for executing
Beam pipelines. Dataflow provides autoscaling, work-rebalancing and integrated monitoring. In this
talk we'll first explore the Beam model to understand why you might use it to write pipelines,
especially stream processing pipelines. Then we'll explore Beam's portability and how pipelines
are executed. Finally, we'll look at Dataflow and some of the contemporary data processing
challenges Dataflow tries to address.


Eric Anderson is a Product Manager at Google working on Beam and Cloud Dataflow. Previously he was
a Product Manager at Amazon Web Services on EC2. He studied Mechanical Engineering at the
University of Utah and received an MBA from Harvard.