strange behavior in BF.INSERT?

Redis modules RedisBloom
1

Hi all,

When I tried to create a new key, “filter”, and insert new items on it with BF.INSERT and ERROR = 1, the response is 0 for each item added:

127.0.0.1:6379> BF.INSERT filter ERROR 1.0 ITEMS foo bar

  1. (integer) 0

  2. (integer) 0

Shouldn’t be:

  1. (integer) 1

  2. (integer) 1

I cannot check it out since after set ERROR to the maximum value (1.0) the probability of false positives is very high, so the following is ok since “foo” should exist in the filter:

127.0.0.1:6379> BF.EXISTS filter foo

(integer) 1

That is not correct since “redis” was not added/inserted int the filter, but for ERROR=1 it happens (I guess):

127.0.0.1:6379> BF.EXISTS filter redis

(integer) 1

This is ok:

127.0.0.1:6379> BF.EXISTS filter bar

(integer) 1

So my questions is, is it correct to get that response above?:

  1. (integer) 0

  2. (integer) 0

If that is the case, can someone explain it to me, please?

Regards!

Rafa

2

Hi Rafa,

Thanks again for your interest and feedback.

Is this behavior repeats itself with a lower error rate?

If not, the bug here is not rejecting the error rate on initialization. It makes little sense to allow 100% error on a probabilistic data structure.

Ariel

3

Hi Ariel, thank you for your quick answer.

I found that behavior only with ERROR = 1.0, even with ERROR = 0.99 doesn’t happen. Please, take a look in the following output:

127.0.0.1:6379> BF.INSERT filter2 ERROR 0.5 ITEMS foo bar

  1. (integer) 1
  2. (integer) 1
    127.0.0.1:6379> BF.INSERT filter3 ERROR 0.7 ITEMS foo bar
  3. (integer) 1
  4. (integer) 1
    127.0.0.1:6379> BF.INSERT filter4 ERROR 0.9 ITEMS foo bar
  5. (integer) 1
  6. (integer) 1
    127.0.0.1:6379> BF.INSERT filter5 ERROR 0.99 ITEMS foo bar
  7. (integer) 1
  8. (integer) 1

So, even using ERROR = 0.99 the response is that both items were inserted

Please don’t try:

BF.INSERT filter5 ERROR 0.9999 ITEMS foo bar

It crashed my local Redis server!!

But when I try the following:
127.0.0.1:6379> BF.INSERT filter6 ERROR 1.0 ITEMS foo bar

  1. (integer) 0
  2. (integer) 0

seems that foo and bar items are not inserted since the response is [0, 0]

Then, if a check the existence of those items in every filter inserted before:

127.0.0.1:6379> BF.MEXISTS filter2 foo bar

  1. (integer) 1
  2. (integer) 1
    127.0.0.1:6379> BF.MEXISTS filter3 foo bar
  3. (integer) 1
  4. (integer) 1
    127.0.0.1:6379> BF.MEXISTS filter4 foo bar
  5. (integer) 1
  6. (integer) 1
    127.0.0.1:6379> BF.MEXISTS filter5 foo bar
  7. (integer) 1
  8. (integer) 1
    127.0.0.1:6379> BF.MEXISTS filter6 foo bar
  9. (integer) 1
  10. (integer) 1

So far, so good, but executing the command below it is impossible to know if they were inserted since ‘noitem’ which was not inserted appear as inserted:

127.0.0.1:6379> BF.MEXISTS filter6 foo bar noitem

  1. (integer) 1
  2. (integer) 1
  3. (integer) 1

Anyway, this is not the problem since having 100% allowed error gives a false positive calling BF.MEXISTS.

I’m with you, having 100% allowed error is ok, what I don’t know if it is a bug is the response coming when both items are inserted with ERROR = 1.0, which is:

127.0.0.1:6379> BF.INSERT filter6 ERROR 1.0 ITEMS foo bar

  1. (integer) 0
  2. (integer) 0

I think that even having a 100% allowed error the response still should be:

  1. (integer) 1
  2. (integer) 1

What do you think?

4

I think we should block it altogether.

As for your question about ‘noitem’, with all probabilistic data structures, you never care about an individual result. You should test a large enough pool of samples in order to draw conclusions about the behavior of the PDS.

Hope this helps

5

Yes, it helps, thank you so much!