Meta Programming with Streams and Pipes
-
Upload
darach -
Category
Technology
-
view
1.840 -
download
2
description
Transcript of Meta Programming with Streams and Pipes
![Page 1: Meta Programming with Streams and Pipes](https://reader037.fdocuments.net/reader037/viewer/2022102823/5491dd7bb479593c2b8b45f3/html5/thumbnails/1.jpg)
![Page 2: Meta Programming with Streams and Pipes](https://reader037.fdocuments.net/reader037/viewer/2022102823/5491dd7bb479593c2b8b45f3/html5/thumbnails/2.jpg)
Essentially, programs that can be created by themselves.
We’re twisting the term a little for streams
We will be defining streaming constructs using streaming constructs
By streams I mean beams which are just streams and pipes for compute as opposed to IO
![Page 3: Meta Programming with Streams and Pipes](https://reader037.fdocuments.net/reader037/viewer/2022102823/5491dd7bb479593c2b8b45f3/html5/thumbnails/3.jpg)
Any info? Please contact #nodeland police at @cianomaidin on twitter…
![Page 4: Meta Programming with Streams and Pipes](https://reader037.fdocuments.net/reader037/viewer/2022102823/5491dd7bb479593c2b8b45f3/html5/thumbnails/4.jpg)
Readable Streams IO / throughput oriented
Events – data, end, error, close
Methods – pause, resume, end, destroy
Writeable Streams IO / throughput oriented
Events - drain, pause, resume, error, close
Methods – write, end, destroy
![Page 5: Meta Programming with Streams and Pipes](https://reader037.fdocuments.net/reader037/viewer/2022102823/5491dd7bb479593c2b8b45f3/html5/thumbnails/5.jpg)
IO Oriented
Finite, not infinite
Asynchronous
Lazy
Assumes in transit data, not in memory forms
Great for IO. Not efficient for compute
![Page 6: Meta Programming with Streams and Pipes](https://reader037.fdocuments.net/reader037/viewer/2022102823/5491dd7bb479593c2b8b45f3/html5/thumbnails/6.jpg)
![Page 7: Meta Programming with Streams and Pipes](https://reader037.fdocuments.net/reader037/viewer/2022102823/5491dd7bb479593c2b8b45f3/html5/thumbnails/7.jpg)
![Page 8: Meta Programming with Streams and Pipes](https://reader037.fdocuments.net/reader037/viewer/2022102823/5491dd7bb479593c2b8b45f3/html5/thumbnails/8.jpg)
Sources Where data pours in
Sinks Where results pop out
Operators Filters – Drop or pass on data based on a UDF Transforms – Mutate data based on a UDF Branch with Filters Combine with Sinks
![Page 9: Meta Programming with Streams and Pipes](https://reader037.fdocuments.net/reader037/viewer/2022102823/5491dd7bb479593c2b8b45f3/html5/thumbnails/9.jpg)
Compute Oriented
Infinite, not finite
Synchronous
Eager
Assumes in memory form, not encoded or wire data
Convenient for compute. Not good for IO
![Page 10: Meta Programming with Streams and Pipes](https://reader037.fdocuments.net/reader037/viewer/2022102823/5491dd7bb479593c2b8b45f3/html5/thumbnails/10.jpg)
![Page 11: Meta Programming with Streams and Pipes](https://reader037.fdocuments.net/reader037/viewer/2022102823/5491dd7bb479593c2b8b45f3/html5/thumbnails/11.jpg)
![Page 12: Meta Programming with Streams and Pipes](https://reader037.fdocuments.net/reader037/viewer/2022102823/5491dd7bb479593c2b8b45f3/html5/thumbnails/12.jpg)
![Page 13: Meta Programming with Streams and Pipes](https://reader037.fdocuments.net/reader037/viewer/2022102823/5491dd7bb479593c2b8b45f3/html5/thumbnails/13.jpg)
![Page 14: Meta Programming with Streams and Pipes](https://reader037.fdocuments.net/reader037/viewer/2022102823/5491dd7bb479593c2b8b45f3/html5/thumbnails/14.jpg)
![Page 15: Meta Programming with Streams and Pipes](https://reader037.fdocuments.net/reader037/viewer/2022102823/5491dd7bb479593c2b8b45f3/html5/thumbnails/15.jpg)
Eager: a.pipe(operation).pipe(z).pipe(a);
Implies:
RangeError: Maximum call stack size exceeded
![Page 16: Meta Programming with Streams and Pipes](https://reader037.fdocuments.net/reader037/viewer/2022102823/5491dd7bb479593c2b8b45f3/html5/thumbnails/16.jpg)
![Page 17: Meta Programming with Streams and Pipes](https://reader037.fdocuments.net/reader037/viewer/2022102823/5491dd7bb479593c2b8b45f3/html5/thumbnails/17.jpg)
![Page 18: Meta Programming with Streams and Pipes](https://reader037.fdocuments.net/reader037/viewer/2022102823/5491dd7bb479593c2b8b45f3/html5/thumbnails/18.jpg)
npm install beam Compute oriented streams Branch, Combine, Filter, Transform data Extend with goto and modules.
npm install eep Beam will be integrated into eep soon Eep provides aggregate windows Slide, Tumble, Monotonic, Periodic Stats Library Ordering now supported
npm install nodesapiens Robosapiens V1 for node.js
Code on github https://github.com/darach