I just started using RedisAI, I was able to setup a gpu container with docker that correctly sees my gpus via nvidia-smi. I correctly loaded a ONNX GPU model, a tensor and I executed the model on the tensor.
There was no error in the whole pipeline but the RedisAI container doesn’t seem to be using my gpu.
Nothing shows up in nvidia-smi and the GPU inference takes the same time as the CPU inference.
Am I missing something? Am I supposed to be doing something except for loading the ONNX module?
Thank you in advance
Can you please share the command you are running the docker with?
it’s a docker-compose up of a this service
- driver: nvidia
command: ["/usr/local/bin/redis-server","--loadmodule","/usr/lib/redis/modules/redisai.so","ONNX","/usr/lib/redis/modules/backends/redisai_onnxruntime/redisai_onnxruntime.so", "TORCH","/usr/lib/redis/modules/backends/redisai_torch/redisai_torch.so","THREADS_PER_QUEUE" ,"4","MODEL_EXECUTION_TIMEOUT","100000" ]