Optimize Streaming

Configure BUFFER_CONNECTIONS_THRESHOLD

The stream service deployed as part of your primary site serves MCAP data to end-users. It reads and merges topic files from your lake bucket. When an end-user requests an MCAP stream with many topics, the stream may pause frequently as the it waits to establish a storage connection for each new file.

The stream service exposes an environment variable named BUFFER_CONNECTIONS_THRESHOLD. When set to a non-negative number N, this will cause the stream service to buffer storage connections ahead of time when more than N lake files must be merged. This will have the effect of smoothing throughput from the service, at the cost of double the memory consumption for the affected requests. The impact of the change will be most noticeable for requests that merge a large number of topics.