THIS IS A TEST INSTANCE. ALL YOUR CHANGES WILL BE LOST!!!!
...
| Expand | ||
|---|---|---|
| ||
SELECT Rstream(auction, price)FROM Bid [NOW]WHERE auction = 1007 OR auction = 1020 OR auction = 2001 OR auction = 2019 OR auction = 2087; |
...
Query
...
3:
...
Who
...
is
...
selling
...
in
...
particular
...
US
...
states?
Illustrates incremental join of the auctions and the persons collections
uses global window and using per-key state and timer APIs
Apply global window to events with triggerrepeatedly after at least nbEvents in pane => results will be materialized each time nbEvents are received.
input1: collection of auctions events filtered by category and keyed by seller id
input2: collection of persons events filtered by US state codes and keyed by person id
CoGroupByKey to groupauctions and persons by personId/sellerId + tags to distinguish persons and auctions
ParDo to do the incrementaljoin: auctions and person events can arrive out of order
person element stored in persistent state in order to match future auctions by that person. Set a timer to clear the person state after a TTL
auction elements stored in persistent state until we have seen the corresponding person record. Then, it can be output and cleared
output NameCityStateId(person.name, person.city, person.state, auction.id) objects
...
| Expand | ||
|---|---|---|
| ||
SELECT Istream(AVG(Q.final)) FROM Category C, (SELECT Rstream(MAX(B.price) AS final, A.category) FROM Auction A [ROWS UNBOUNDED], Bid B [ROWS UNBOUNDED] WHERE A.id=B.auction AND B.datetime < A.expires AND A.expires < CURRENT_TIME GROUP BY A.id, A.category) Q WHERE Q.category = C.id GROUP BY C.id; |
...
Query 5: Which auctions have seen the most bids in the last period?
Illustrates sliding windows and combiners (i.e. reducers) to compare the elements in auctions Collection
Input: (sliding) window (to have a result over 1h period updated every 1 min) collection of bids events
ParDo to replacebid elements by their auction id
Count.PerElement to count the occurrences of each auction id
Combine.globally to select only the auctions with the maximumnumber of bids
BinaryCombineFnto compare one to one the elements of the collection (auction id occurrences, i.e. number of bids)
Return KV(auction id, max occurrences)
output:AuctionCount(auction id, max occurrences) objects
...