Howdy folks.
We are using Redis streams with pruning set to 10k items. We also have rdb snapshots enabled. This all works as expected with the stream size being maintained at ~10k and the rdb file being continuously updated as well.
We have a need for debugging to be able to recreate the entries in the stream for an extended period of time in certain cases. Given that the stream gets pruned continuously, we are considering the best approach to achieve this.
One approach we tried is to periodically save away the rdb snapshots (dump1.rdb,dump2.rdb,…). The idea being that in case we need to restore redis stream entries, then we would restore the corresponding set of rdb files using the “DEBUG RELOAD MERGE” option command in redis-cli.
The issue with this is that as stream contents are restored, the stream keeps getting capped at 10k as new rdb files are added. Which defeats our objective of recreating the stream over an extended period of time. Any thoughts on how to circumvent this?
Would appreciate any other suggestions for solving our need.
PS: on a related note, we tried parsing rdb files using rdbtools · PyPI, https://rdb.fnordig.de/; they seem to load the name of the stream correctly, but contents of the stream are reported as empty. Any other rdb parsers that support streams correctly?
Thanks
Ben