Error: The RDB file contains module data for the module type 'ai', that the responsible module is not able to load

Redis modules RedisAI
1

Hi, I want to revisit a project I last touched a couple years back but I cannot get the dump.rdb to load using the latest redis-server and modules. I did mention a similar issue here Check RDB serialization · Issue #302 · RedisAI/RedisAI · GitHub and they say it got fixed in RedisAI 1.0 but it does not work for me still.

Below is the current output. If I cannot load this rdb, is there any way to extract the data from it?

11837:M 15 Jan 2022 13:27:55.448 # Server initialized
11837:M 15 Jan 2022 13:27:55.449 * RedisGears version 1.0.6, git_sha=71740d307029c2b0ef49633ec7b1f12da23290c8, compiled_os=linux-bionic-x64
11837:M 15 Jan 2022 13:27:55.449 * Redis version found by RedisGears : 6.2.4 - oss
11837:M 15 Jan 2022 13:27:55.449 * MaxExecutions:1000
11837:M 15 Jan 2022 13:27:55.449 * MaxExecutionsPerRegistration:100
11837:M 15 Jan 2022 13:27:55.449 * ProfileExecutions:0
11837:M 15 Jan 2022 13:27:55.449 * PythonAttemptTraceback:1
11837:M 15 Jan 2022 13:27:55.449 * DependenciesUrl:http://redismodules.s3.amazonaws.com/redisgears/redisgears-dependencies.linux-bionic-x64.1.0.6.tgz
11837:M 15 Jan 2022 13:27:55.449 * DependenciesSha256:cd49f491e6ef9725b23dfa2f8c272d8b25ee129a73856279a27a26166051eba6
11837:M 15 Jan 2022 13:27:55.449 * CreateVenv:1
11837:M 15 Jan 2022 13:27:55.449 * ExecutionThreads:3
11837:M 15 Jan 2022 13:27:55.449 * ExecutionMaxIdleTime:5000
11837:M 15 Jan 2022 13:27:55.449 * PythonInstallReqMaxIdleTime:30000
11837:M 15 Jan 2022 13:27:55.449 * PythonInstallationDir:/var/opt/redislabs/modules/rg
11837:M 15 Jan 2022 13:27:55.449 * DownloadDeps:1
11837:M 15 Jan 2022 13:27:55.449 * ForceDownloadDepsOnEnterprise:0
11837:M 15 Jan 2022 13:27:55.449 * SendMsgRetries:3
11837:M 15 Jan 2022 13:27:55.449 # could not initialize RedisAI_GetLLAPIVersion

11837:M 15 Jan 2022 13:27:55.449 # could not initialize RediAI api, running without AI support.
11837:M 15 Jan 2022 13:27:55.449 * PYENV_DIR: /home/tommy/redisEdge/rg/RedisGears/bin/linux-x64-release///python3_1.0.6/
11837:M 15 Jan 2022 13:27:55.449 * PYENV_HOME_DIR: /home/tommy/redisEdge/rg/RedisGears/bin/linux-x64-release///python3_1.0.6//.venv/
11837:M 15 Jan 2022 13:27:55.449 * PYENV_BIN_DIR: /home/tommy/redisEdge/rg/RedisGears/bin/linux-x64-release///python3_1.0.6//.venv//bin
11837:M 15 Jan 2022 13:27:55.449 * PYENV_ACTIVATE: /home/tommy/redisEdge/rg/RedisGears/bin/linux-x64-release///python3_1.0.6//.venv//bin/activate_this.py
11837:M 15 Jan 2022 13:27:55.449 * PYENV_ACTIVATE_SCRIPT: /home/tommy/redisEdge/rg/RedisGears/bin/linux-x64-release///python3_1.0.6//.venv//bin/activate
11837:M 15 Jan 2022 13:27:55.450 * log file is

11837:M 15 Jan 2022 13:27:55.450 * Found python installation under: /home/tommy/redisEdge/rg/RedisGears/bin/linux-x64-release///python3_1.0.6/
11837:M 15 Jan 2022 13:27:55.450 * Found venv installation under: /var/opt/redislabs/modules/rg/.venv-
11837:M 15 Jan 2022 13:27:55.457 * Initializing Python environment with: exec(open(’/var/opt/redislabs/modules/rg/.venv-/bin/activate_this.py’).read(), {‘file’: ‘/var/opt/redislabs/modules/rg/.venv-/bin/activate_this.py’})
11837:M 15 Jan 2022 13:27:55.483 * Module ‘rg’ loaded from /home/tommy/redisEdge/modules/redisgears1.so
11837:M 15 Jan 2022 13:27:55.484 * RedisTimeSeries version 999999, git_sha=d242c827e6fed5dd87c744e211e5d871aada8b2a
11837:M 15 Jan 2022 13:27:55.484 * Redis version found by RedisTimeSeries : 6.2.4 - oss
11837:M 15 Jan 2022 13:27:55.484 # could not initialize RedisGears_BufferCreate

11837:M 15 Jan 2022 13:27:55.484 # Failed initialize RedisGears API
11837:M 15 Jan 2022 13:27:55.484 * Module ‘timeseries’ loaded from /home/tommy/redisEdge/modules/redistimeseries.so
11837:M 15 Jan 2022 13:27:55.484 * Got module load event, trying to reinitialize RedisAI api
11837:M 15 Jan 2022 13:27:55.484 # could not initialize RedisAI_GetLLAPIVersion

11837:M 15 Jan 2022 13:27:55.484 # could not initialize RediAI api, running without AI support.
11837:M 15 Jan 2022 13:27:55.484 * Redis version found by RedisAI: 6.2.4 - oss
11837:M 15 Jan 2022 13:27:55.484 * RedisAI version 999999, git_sha=ccdc5e8ff1654b7317391a9e34ebe1504709e68f
11837:M 15 Jan 2022 13:27:55.484 * Module ‘ai’ loaded from /home/tommy/redisEdge/RedisAI/bin/linux-x64-release/install-cpu/redisai.so
11837:M 15 Jan 2022 13:27:55.484 * Got module load event, trying to reinitialize RedisAI api
11837:M 15 Jan 2022 13:27:55.484 * RedisAI api loaded successfully.
11837:M 15 Jan 2022 13:27:55.485 * version: 999999 git sha: de32406 branch: master
11837:M 15 Jan 2022 13:27:55.485 * Exported RedisJSON_V1 API
11837:M 15 Jan 2022 13:27:55.485 * Enabled diskless replication
11837:M 15 Jan 2022 13:27:55.485 * Created new data type ‘ReJSON-RL’
11837:M 15 Jan 2022 13:27:55.485 * Module ‘ReJSON’ loaded from /home/tommy/redisEdge/modules/RedisJSON/target/release/librejson.so
11837:M 15 Jan 2022 13:27:55.485 * Got module load event, trying to reinitialize RedisAI api
11837:M 15 Jan 2022 13:27:55.485 * RedisAI api loaded successfully.
11837:M 15 Jan 2022 13:27:55.485 * Module ‘bf’ loaded from /home/tommy/redisEdge/modules/RedisBloom/redisbloom.so
11837:M 15 Jan 2022 13:27:55.485 * Got module load event, trying to reinitialize RedisAI api
11837:M 15 Jan 2022 13:27:55.485 * RedisAI api loaded successfully.
11837:M 15 Jan 2022 13:27:55.485 * Loading RDB produced by version 5.0.8
11837:M 15 Jan 2022 13:27:55.485 * RDB age 54942308 seconds
11837:M 15 Jan 2022 13:27:55.485 * RDB memory usage when created 222.64 Mb
11837:M 15 Jan 2022 13:27:55.549 # Experienced a short read while reading a model from RDB
11837:M 15 Jan 2022 13:27:55.549 # Internal error in RDB reading offset 4194303, function at rdb.c:2228 → The RDB file contains module data for the module type ‘ai’, that the responsible module is not able to load. Check for modules log above for additional clues.
[offset 0] Checking RDB file dump.rdb
[offset 26] AUX FIELD redis-ver = ‘5.0.8’
[offset 40] AUX FIELD redis-bits = ‘64’
[offset 52] AUX FIELD ctime = ‘1587310967’
[offset 67] AUX FIELD used-mem = ‘233458432’
[offset 83] AUX FIELD aof-preamble = ‘0’
[offset 95] MODULE AUX for: GEARS_DT0
[offset 98] Selecting DB ID 0
[offset 42982996] MODULE AUX for: GEARS_DT0
[offset 43146539] Checksum OK
[offset 43146539] \o/ RDB looks OK! \o/
[info] 158472 keys read
[info] 0 expires
[info] 0 already expired

2

Hi @irthomasthomas

  1. From the log I see that the module encounter a “short read” meaning, it tried to load a value which is either did not exist or was corrupted. Can you specify which version of RedisAI exported the RDB?
  2. I see that you are running for a quite dated master branch build. Care to update to 1.2.5 (official 1.2 GA build)
3

Hi thanks for suggestion. I updated it and it still makes this errop

20919:C 17 Jan 2022 13:19:30.239 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
20919:C 17 Jan 2022 13:19:30.239 # Redis version=6.2.4, bits=64, commit=00000000, modified=0, pid=20919, just started
20919:C 17 Jan 2022 13:19:30.239 # Configuration loaded
20919:M 17 Jan 2022 13:19:30.239 * Increased maximum number of open files to 10032 (it was originally set to 1024).
20919:M 17 Jan 2022 13:19:30.239 * monotonic clock: POSIX clock_gettime
.
.-__ ''-._ _.- . . ‘’-._ Redis 6.2.4 (00000000/0) 64 bit
.- .-```. ```\/ _.,_ ''-._ ( ' , .-` | `, ) Running in standalone mode |`-._`-...-` __...-.-.|’_.-'| Port: 6379 | -. ._ / _.-' | PID: 20919 -._ -._ -./ .-’ .-’
|-._-. -.__.-' _.-'_.-'| | -.-._ _.-'_.-' | https://redis.io -._ -._-..-’.-’ .-’
|-._-. -.__.-' _.-'_.-'| | -.-._ _.-'_.-' | -._ -._-..-’_.-’ _.-’
-._ -..-’ _.-’
-._ _.-' -..-’

20919:M 17 Jan 2022 13:19:30.240 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
20919:M 17 Jan 2022 13:19:30.240 # Server initialized
20919:M 17 Jan 2022 13:19:30.241 * RedisGears version 1.0.6, git_sha=71740d307029c2b0ef49633ec7b1f12da23290c8, compiled_os=linux-bionic-x64
20919:M 17 Jan 2022 13:19:30.241 * Redis version found by RedisGears : 6.2.4 - oss
20919:M 17 Jan 2022 13:19:30.241 * MaxExecutions:1000
20919:M 17 Jan 2022 13:19:30.241 * MaxExecutionsPerRegistration:100
20919:M 17 Jan 2022 13:19:30.241 * ProfileExecutions:0
20919:M 17 Jan 2022 13:19:30.241 * PythonAttemptTraceback:1
20919:M 17 Jan 2022 13:19:30.241 * DependenciesUrl:http://redismodules.s3.amazonaws.com/redisgears/redisgears-dependencies.linux-bionic-x64.1.0.6.tgz
20919:M 17 Jan 2022 13:19:30.241 * DependenciesSha256:cd49f491e6ef9725b23dfa2f8c272d8b25ee129a73856279a27a26166051eba6
20919:M 17 Jan 2022 13:19:30.241 * CreateVenv:1
20919:M 17 Jan 2022 13:19:30.241 * ExecutionThreads:3
20919:M 17 Jan 2022 13:19:30.241 * ExecutionMaxIdleTime:5000
20919:M 17 Jan 2022 13:19:30.241 * PythonInstallReqMaxIdleTime:30000
20919:M 17 Jan 2022 13:19:30.241 * PythonInstallationDir:/var/opt/redislabs/modules/rg
20919:M 17 Jan 2022 13:19:30.241 * DownloadDeps:1
20919:M 17 Jan 2022 13:19:30.241 * ForceDownloadDepsOnEnterprise:0
20919:M 17 Jan 2022 13:19:30.241 * SendMsgRetries:3
20919:M 17 Jan 2022 13:19:30.241 # could not initialize RedisAI_GetLLAPIVersion

20919:M 17 Jan 2022 13:19:30.241 # could not initialize RediAI api, running without AI support.
20919:M 17 Jan 2022 13:19:30.241 * PYENV_DIR: /home/tommy/redisEdge/rg/RedisGears/bin/linux-x64-release///python3_1.0.6/
20919:M 17 Jan 2022 13:19:30.241 * PYENV_HOME_DIR: /home/tommy/redisEdge/rg/RedisGears/bin/linux-x64-release///python3_1.0.6//.venv/
20919:M 17 Jan 2022 13:19:30.241 * PYENV_BIN_DIR: /home/tommy/redisEdge/rg/RedisGears/bin/linux-x64-release///python3_1.0.6//.venv//bin
20919:M 17 Jan 2022 13:19:30.241 * PYENV_ACTIVATE: /home/tommy/redisEdge/rg/RedisGears/bin/linux-x64-release///python3_1.0.6//.venv//bin/activate_this.py
20919:M 17 Jan 2022 13:19:30.241 * PYENV_ACTIVATE_SCRIPT: /home/tommy/redisEdge/rg/RedisGears/bin/linux-x64-release///python3_1.0.6//.venv//bin/activate
20919:M 17 Jan 2022 13:19:30.241 * log file is

20919:M 17 Jan 2022 13:19:30.241 * Found python installation under: /home/tommy/redisEdge/rg/RedisGears/bin/linux-x64-release///python3_1.0.6/
20919:M 17 Jan 2022 13:19:30.241 * Found venv installation under: /var/opt/redislabs/modules/rg/.venv-
20919:M 17 Jan 2022 13:19:30.248 * Initializing Python environment with: exec(open(’/var/opt/redislabs/modules/rg/.venv-/bin/activate_this.py’).read(), {‘file’: ‘/var/opt/redislabs/modules/rg/.venv-/bin/activate_this.py’})
20919:M 17 Jan 2022 13:19:30.269 * Module ‘rg’ loaded from /home/tommy/redisEdge/modules/redisgears1.so
20919:M 17 Jan 2022 13:19:30.269 * Redis version found by RedisAI: 6.2.4 - oss
20919:M 17 Jan 2022 13:19:30.269 * RedisAI version 10205, git_sha=746b199e69f272d12aab00c26d31c8e35eb48b29
20919:M 17 Jan 2022 13:19:30.269 * Module ‘ai’ loaded from /home/tommy/RedisAI/bin/linux-x64-release/install-cpu/redisai.so
20919:M 17 Jan 2022 13:19:30.269 * Got module load event, trying to reinitialize RedisAI api
20919:M 17 Jan 2022 13:19:30.269 * RedisAI api loaded successfully.
20919:M 17 Jan 2022 13:19:30.269 * version: 999999 git sha: de32406 branch: master
20919:M 17 Jan 2022 13:19:30.269 * Exported RedisJSON_V1 API
20919:M 17 Jan 2022 13:19:30.269 * Enabled diskless replication
20919:M 17 Jan 2022 13:19:30.269 * Created new data type ‘ReJSON-RL’
20919:M 17 Jan 2022 13:19:30.269 * Module ‘ReJSON’ loaded from /home/tommy/redisEdge/modules/RedisJSON/target/release/librejson.so
20919:M 17 Jan 2022 13:19:30.269 * Got module load event, trying to reinitialize RedisAI api
20919:M 17 Jan 2022 13:19:30.269 * RedisAI api loaded successfully.
20919:M 17 Jan 2022 13:19:30.269 * Module ‘bf’ loaded from /home/tommy/redisEdge/modules/RedisBloom/redisbloom.so
20919:M 17 Jan 2022 13:19:30.269 * Got module load event, trying to reinitialize RedisAI api
20919:M 17 Jan 2022 13:19:30.269 * RedisAI api loaded successfully.
20919:M 17 Jan 2022 13:19:30.269 * Loading RDB produced by version 5.0.8
20919:M 17 Jan 2022 13:19:30.269 * RDB age 55114603 seconds
20919:M 17 Jan 2022 13:19:30.269 * RDB memory usage when created 222.64 Mb
20919:M 17 Jan 2022 13:19:30.326 # Experienced a short read while reading a model from RDB
20919:M 17 Jan 2022 13:19:30.326 # Internal error in RDB reading offset 4194303, function at rdb.c:2228 → The RDB file contains module data for the module type ‘ai’, that the responsible module is not able to load. Check for modules log above for additional clues.
[offset 0] Checking RDB file dump.rdb
[offset 26] AUX FIELD redis-ver = ‘5.0.8’
[offset 40] AUX FIELD redis-bits = ‘64’
[offset 52] AUX FIELD ctime = ‘1587310967’
[offset 67] AUX FIELD used-mem = ‘233458432’
[offset 83] AUX FIELD aof-preamble = ‘0’
[offset 95] MODULE AUX for: GEARS_DT0
[offset 98] Selecting DB ID 0
[offset 42982996] MODULE AUX for: GEARS_DT0
[offset 43146539] Checksum OK
[offset 43146539] \o/ RDB looks OK! \o/
[info] 158472 keys read
[info] 0 expires
[info] 0 already expired