Streams are a simple concept, but they are actually really weird if think about it.

What is a stream?

A stream is a sequence of elements. An example of a stream is file, where each element is a byte.

Element type is weird

Element types could be…

• …an array of block sized bytes, for performance reasons
• …entire JavaScript data structures
• …consistent, as in stream of bytes
• …inconsistent, which would be a hard to use

Reading a stream is weird

The reader of a stream only see a sequence of elements. Something else determines the order.

Reading a stream could…

• …be non-destructive from “start” to “end” is the same as reading a file
• …read “oldest element” when stream is an queue
• …read “element last written” when stream is a stack
• …read “least used element” when stream is a least recently used cache
• …trigger I/O as the internal batch runs out of data
• …block the thread as it waits for more data
• …be inverted with events being pushed to the reader
• …be a from a finite ring buffer

Writing a stream is weird

The writer of a stream submits one element at a time.

Writing a stream could…

• …be overwriting a file
• …be appending to a file
• …be pushing to a queue
• …be pushing to a stack
• …submitting to a command queue
• …trigger I/O as the internal buffer is filled
• …block the thread due to backpressure

Elements are weird

Elements in stream could be…

• …guaranteed to be in-order as in TCP but then cause performance issues
• …unordered or lost as in UDP
• …could be duplicated due to at-least-once delivery

Higher order streams are weird

Streams themselves are can be input into other functions.

Streams could be…

• …recursive stream of streams
• …cloned and sent to two different places
• …merged with another
• …transformed into another element by element
• …transformed from a stream of characters to a stream of lines
• …paired up to form a bidirectional communication channel

Streams are weird because so many things are just streams with another name. With our stream goggles on, we can even reconceptualise “random memory access” as two streams. A stream of load address commands and another stream of memory values.

Do you want to learn all the ways streams are weird? You’re in luck, Battlefy is hiring.