Use case for xreadgroup with multiple streams and count

Redis commands & data structures
1

We are building an app that has to process messages from multiple streams in priority - at any point of time it should process the message from the Highest priority stream.

We tried using xreadgroup … count 1 streams s1 s2 s3 …, but realized that we will get up to 1 message from EACH stream. We hope to get 1 message from the first stream with data.

Looking for help/info:

  1. What is the use case for reading N messages from every stream ?
  2. Simple alternative to achieve the requested functionality is to iterate over the streams one at a time - is there a better solution ?
  3. Is this a good use case to use LUA. scripting to speed up the polling ?
  4. The BLPOP command provides the intended behavior, without the semantics of a stream. The drawback is that the stored item must be a string. Is this a good alternative (using JSON to store complex messages) ?

Thanks,
Dasho

2

Hi Dasho,

How do you determine which stream / message has the highest priority? Is this an attribute of the message?

Kyle

3

In my design s1 is the highest priority, followed by s2, s3, … similar to the way BLPOP Is used in: 6.4.1 First-in, first-out queues | Redis Labs

4

Hi Dasho,

In that case, I suggest creating a loop in your code that reads from these in priority order.

Best,
Kyle