the biodiversity and the  intact forest systems in particular that are buffering this.

I ended up writing a custom store that "buffers" sets for both a small time interval and ensuring only one async action is in flight (and triggering an extra round of async processing if a set was seen after the last async action was launched).

unbuffer disables the output buffering that occurs when program output is redirected. For example, suppose you are watching the output from a fifo by running it through od and then more.    od -c /tmp/fifo | more You will not see anything until a full page of output has been produced. You can disable this automatic buffering as follows:    unbuffer od -c /tmp/fifo | more

If there are some pipe between these commands, you need to flush stdout. for example: script -q /dev/null commands... | ruby -ne 'print "....\n";STDOUT.flush'