Tensorflow model yields different outputs on Redis Gear - correct redisAI implementation?

Redis modules RedisGears
1

Executing a Tensorflow model on Redis Gear seems to yield wrong and different outputs than those of Redis CLI. CLI matches the output running on native Tensorflow.

Please see below:

Running from Redis CLI (Correct outputs)

127.0.0.1:6379> AI.TENSORSET tA FLOAT 2 44 VALUES 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 1 0 0 1 0 0 0 1 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 1 0 0 1 0 0 0 0 1
OK
127.0.0.1:6379> AI.TENSORGET tA META
1) "dtype"
2) "FLOAT"
3) "shape"
4) 1) (integer) 2
   2) (integer) 44
127.0.0.1:6379> AI.MODELRUN mymodel INPUTS tA OUTPUTS tResult
OK
127.0.0.1:6379> AI.TENSORGET tResult META
1) "dtype"
2) "FLOAT"
3) "shape"
4) 1) (integer) 2
   2) (integer) 7
127.0.0.1:6379> AI.TENSORGET tResult VALUES
 1) "0.00067968893563374877"
 2) "2.365510908930446e-06"
 3) "5.9218251635684283e-07"
 4) "0.99931156635284424"
 5) "2.4811367893562419e-06"
 6) "1.7273650882998481e-06"
 7) "1.5625516880390933e-06"
 8) "1.0150862181035336e-05"
 9) "1.2889394156445633e-06"
10) "2.7037253857997712e-07"
11) "0.052372831851243973"
12) "0.00010688313341233879"
13) "0.94654035568237305"
14) "0.00096822890918701887"

See that, #4 and #13 is cleary the best two here.

Redis Gear (Incorrect outputs)

from redisgears import executeCommand as execute
import redisAI


def xlog(*args):
    execute('XADD', 'logs2', '*', 'msg', ' '.join(map(str, args)))

def model_run():
    input = [1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 1, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 1]
    bytes_arr = bytearray(input)
    input_tensor = redisAI.createTensorFromBlob('FLOAT', [2, 44], bytes_arr)

    runner = redisAI.createModelRunner('mymodel')
    redisAI.modelRunnerAddInput(runner, 'x', input_tensor)
    redisAI.modelRunnerAddOutput(runner, 'Identity')

    model_replies = redisAI.modelRunnerRun(runner)
    model_output = redisAI.tensorToFlatList(model_replies[0])
    xlog("model_run model_output", model_output)

model_run()
GearsBuilder().run()


127.0.0.1:6379> xrange logs2 - +
1) 1) "1590438625297-0"
   2) 1) "msg"
      2) "model_run model_output [0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0]"

The outputs don’t match that of CLI. Clearly the second 1.0 (#11) should not be there by any means.

Can you guys please check the script for Gear and tell me if this is a correct implementation of the above CLI actions using redisAI C API? Or am I missing something? I couldn’t find any documentation of the module, so I gathered this from the source code and a couple of example repos.

Thank you for your patient cooperation!

2

Please note that if I use:

input_tensor = redisAI.createTensorFromValues('FLOAT', [2, 44], input)

Instead of:

input_tensor = redisAI.createTensorFromBlob('FLOAT', [2, 44], bytes_arr)

I get the following error:

'spam.error: dims argument must be iterable\n'

Tried converting list [2, 44], to tuple (2, 44,). The same error persists. Please advice.

3

Can you log the size of bytes_arr (len(bytes_arr)). It should be 2 * 44 * 4(FLOAT SIZE). I suspect python convert it to double (2 * 44 * 8(DOUBLE SIZE)) and RedisAI took just half of the data.

On createTensorFromValues we had an issue that it took the dims instead of the values and it only will be fixed here: https://github.com/RedisGears/RedisGears/pull/334/files#diff-29034498583e2c77f1a8c06502d468fcR2023

4

Thank you @meirsh. I pulled redislabs/redismod:edge and the above error of createTensorFromValues went away. However, a new one is appearing:

'spam.error: values argument must be iterable\n'

When calling: redisAI.createTensorFromValues('FLOAT', [2, 44], input), input being a python list.

I believe this is because of PyIter_Check here: AI api crash fixes and type safe checking. by MeirShpilraien · Pull Request #334 · RedisGears/RedisGears · GitHub

PyIter_Check is for checking whether an object is an iterator, not whether it can provide one. (ref)

Maybe this should be done checking if PyObject_GetIter returns NULL like the rest of the checks?

5

Opened a PR here: https://github.com/RedisGears/RedisGears/pull/336

6

Yes you are right, thanks for the PR :slight_smile: