The calculation speed is very slow when I create several rules in a Redis Timeseries

Redis modules RedisTimeSeries
1

Hi there,

I am new to Redis TimeSeries. I define couple of rules in Timeseries e.g. the combination of (“avg”, “min”, “max”, “first”, “last”) and different bucket (60000, 600005, 6000015). It works very well for the real time update. However, I am try to rebuild the time series from time 0. The calculation is very slow. It takes 80 second to add 60,000 input data.

Although I am using the pipeline in redistimeseries-py (https://github.com/RedisTimeSeries/redistimeseries-py)
However, I did not see significant improvement on the speed.

Can anyone give me some suggestions? It seems the create rules is only used to do real time update. If I add data from time 0, it is much slower than aggregation query. Is my understanding correct?

Thanks

2

Hi Vonka,
Which version do you use? We have recently made a change to our rdb saving so can you check if the issue continues on the master branch?
Thanks.

3

Hi,

You should try the TS.MADD command which works faster for bulk loading. just make sure your order it according to the timestamp.

Danni.

4

Hi ashtul,
Thanks for your sharing. Actually, when I did not create this rules in my time series, pipeline do improve the speed. It is around 3 times faster. However, after I add rules in my time series. It is only 1.2 times faster. I think the bottleneck is coming from these rules.

Thanks

5

Hi Danni,

This is a really good point. I did not notice this before. For TS.MADD, my previous understanding is this function is used to add sample for multiple time series. However, it also works for the same series if provide the timestamp of the samples in an increasing order.
Let me have a try with this method.

Thanks,

6

Hi danni-m,

TS.MADD solved my issue. It is 30 times faster now! Amazing!

Thanks!