redigo client scan returns error with latest redisearch

Redis modules RediSearch
1

I installed the latest redisearch from HEAD, and I am now getting an error message from my Go client (one of the most popular Go clients - hasn’t changed in a while) when I do an FT.SEARCH and parse the results:

redigo.Scan: cannot assign to dest 0: cannot convert from Redis simple string to string

I can confirm that I DO get results back from FT.SEARCH, it’s just that the format is no longer recognized (or some other problem):

2019/01/29 15:45:48 scannning [[_t [49 53 52 56 55 52 52 49 54 51] ph [50 48 49 57 45 48 49 45 50 57 32 48 54 58 52 50 58 53 54 32 85 84 67 1 49 50 55 53] tags [116 58 101 110 103 108 105 115 104 44 116 58 117 110 105 116 101 100 115 116 97 116 101 115 44 119 58 99 111 117 99 104 114 105 100 101 58 116 58 111 119 110 101 114 44 119 58 99 111 117 99 104 114 105 100 101 58 116 58 117 115 101 100 44 116 58 105 109 97 103 101 44 119 58 99 111 117 99 104 114 105 100 101 58 116 58 99 114 102 44 119 58 99 111 117 99 104 114 105 100 101 58 116 58 104 111 110 100 97 44 119 58 99 111 117 99 104 114 105 100 101 44 102 58 99 108 110 111 114 116 104 99 101 110 116 114 97 108 102 108 111 114 105 100 97 44 99 102 58 109 58 99 108 44 70 76] mpid [99 102 58 109 58 99 108] imgsmallx [53 48 48] ct [108] created [50 48 49 57 45 48 49 45 50 57 32 48 54 58 52 50 58 52 51] imgx [51 48 48] st [70 76] geo imgy [50 50 53] ctitle [72 111 110 100 97 32 67 82 70 49 48 48 70 32 68 105 114 116 32 66 105 107 101] location currency [85 83 68] url [104 116 116 112 115 58 47 47 106 97 99 107 115 111 110 118 105 108 108 101 46 99 114 97 105 103 115 108 105 115 116 46 111 114 103 47 109 99 121 47 100 47 106 97 99 107 115 111 110 118 105 108 108 101 45 104 111 110 100 97 45 99 114 102 49 48 48 102 45 100 105 114 116 45 98 105 107 101 47 54 56 48 54 50 53 50 56 53 48 46 104 116 109 108] title [72 111 110 100 97 32 67 82 70 49 48 48 70 32 68 105 114 116 32 66 105 107 101] imgurl [104 116 116 112 115 58 47 47 105 109 97 103 101 115 46 99 114 97 105 103 115 108 105 115 116 46 111 114 103 47 48 48 85 48 85 95 106 49 51 80 69 68 51 84 102 83 115 95 51 48 48 120 51 48 48 46 106 112 103] feed [102 58 99 108 110 111 114 116 104 99 101 110 116 114 97 108 102 108 111 114 105 100 97] price [49 50 55 53] imgurlsmall [104 116 116 112 115 58 47 47 115 116 111 114 97 103 101 46 103 111 111 103 108 101 97 112 105 115 46 99 111 109 47 108 119 105 109 103 47 109 104 47 108 47 99 108 110 111 114 116 104 99 101 110 116 114 97 108 102 108 111 114 105 100 97 47 55 51 48 51 54 56 48 98 48 97 56 101 52 57 52 102 49 52 52 50 100 57 97 52 51 56 51 50 54 55 48 99 46 106 112 103] imgsmally [51 48 48] mileage year vin ]]

2019/01/29 15:45:48 ParseFTSearchResponse scan post hash: redigo.Scan: cannot assign to dest 0: cannot convert from Redis simple string to string

Before I dive into debugging what changed, does this ring a bell to anybody in the redisearch team? I updated to HEAD today - my previous version was from mid-Dec.

Thanks!

Michael

2

A lot of internals have changed between the previous release and current HEAD. I’m not surprised that something broke, but I’m surprised it broke in that fashion. It sounds like a bug in redigo itself, but if you figure out what part of the response specifically is failing, I can try and see if we can change that part back.

Regards,
Mark Nunberg | Senior Software Engineer
Redis Labs - home of Redis

Email: mark@redislabs.com

3

I can’t verify yet, but the library used to get both keys and values of hashes as byte arrays, while now the key is a string. I am working on a PR that you can see, but maybe this description rings a bell.

the keys below are strings and the values byte arrays:

scan.Scan d &[ ] src[0] [_t [49 53 52 55 55 56 56 52 55 49] ph [50 48 49 57 45 48 49 45 49 56 32 48 53 58 49 52 58 51 56 32 85 84 67 1 50 53 48 48] tags [116 58 101 110 103 108 105 115 104 44 116 58 117 110 105 116 101 100 115 116 97 116 101 115 44 119 58 99 111 117 99 104 114 105 100 101 58 116 58 111 119 110 101 114 44 119 58 99 111 117 99 104 114 105 100 101 58 116 58 117 115 101 100 44 116 58 105 109 97 103 101 44 119 58 99 111 117 99 104 114 105 100 101 58 116 58 98 109 119 44 119 58 99 111 117 99 104 114 105 100 101 44 102 58 99 108 115 102 98 97 121 109 111 116 111 114 99 121 99 108 101 115 44 99 102 58 109 58 99 108 44 67 65] mpid [99 102 58 109 58 99 108] imgsmallx [53 48 48] ct [108] created [50 48 49 57 45 48 49 45 49 56 32 48 53 58 49 52 58 51 49] imgx [51 48 48] st [67 65] imgy [50 50 53] ctitle [66 77 87 32 70 54 53 48 32 49 57 57 56] location [109 105 115 115 105 111 110 32 100 105 115 116 114 105 99 116] currency [85 83 68] url [104 116 116 112 115 58 47 47 115 102 98 97 121 46 99 114 97 105 103 115 108 105 115 116 46 111 114 103 47 115 102 99 47 109 99 121 47 100 47 115 97 110 45 102 114 97 110 99 105 115 99 111 45 98 109 119 45 102 47 54 55 57 55 54 54 56 56 55 55 46 104 116 109 108] title [66 77 87 32 70 54 53 48 32 49 57 57 56] imgurl [104 116 116 112 115 58 47 47 105 109 97 103 101 115 46 99 114 97 105 103 115 108 105 115 116 46 111 114 103 47 48 48 71 48 71 95 54 70 68 66 80 102 113 80 68 104 116 95 51 48 48 120 51 48 48 46 106 112 103] feed [102 58 99 108 115 102 98 97 121 109 111 116 111 114 99 121 99 108 101 115] mileage [51 49 56 48 48] price [50 53 48 48] imgurlsmall [104 116 116 112 115 58 47 47 115 116 111 114 97 103 101 46 103 111 111 103 108 101 97 112 105 115 46 99 111 109 47 108 119 105 109 103 47 109 104 47 108 47 99 108 115 102 98 97 121 109 111 116 111 114 99 121 99 108 101 115 47 51 100 100 49 98 52 52 98 55 54 97 56 101 48 56 49 98 56 55 57 99 102 57 51 55 48 49 101 99 48 57 51 46 106 112 103] imgsmally [51 48 48] geo [45 49 50 50 46 52 49 52 55 57 55 55 32 51 55 46 55 53 57 56 54 52 56]]2019/01/30 11:52:14 ParseFTSearchResponse scan post hash: redigo.Scan: cannot assign to dest 0: cannot convert from Redis simple string to string

4

https://github.com/gomodule/redigo/compare/master…mmasouras:redisearch_stringscan?expand=1

5

So turns out you return nil values for fields that don’t exist in the hash.
https://github.com/gomodule/redigo/issues/396

Surprisingly, you return these nil fields if any hash in the request has them. I.e. if I ONLY get one result back, you don’t return nil values:

[]interface {}{[]interface {}{"_t", []uint8{0x31, 0x35, 0x34, 0x37, 0x37, 0x35, 0x34, 0x34, 0x39, 0x32}, “vin”, []uint8{0x31, 0x68, 0x66, 0x73, 0x63, 0x34, 0x37, 0x30, 0x78, 0x35, 0x61, 0x34, 0x31, 0x31, 0x32, 0x33, 0x33}, “created”, []uint8{0x32, 0x30, 0x31, 0x39, 0x2d, 0x30, 0x31, 0x2d, 0x31, 0x37, 0x20, 0x31, 0x39, 0x3a, 0x34, 0x38, 0x3a, 0x31, 0x32}, “url”, []uint8{0x68, 0x74, 0x74, 0x70, 0x73, 0x3a, 0x2f, 0x2f, 0x77, 0x77, 0x77, 0x2e, 0x69, 0x6d, 0x6f, 0x74, 0x6f, 0x72, 0x73, 0x70, 0x6f, 0x72, 0x74, 0x73, 0x6f, 0x72, 0x6c, 0x61, 0x6e, 0x64, 0x6f, 0x2e, 0x63, 0x6f, 0x6d, 0x2f, 0x64, 0x65, 0x66, 0x61, 0x75, 0x6c, 0x74, 0x2e, 0x61, 0x73, 0x70, 0x3f, 0x70, 0x61, 0x67, 0x65, 0x3d, 0x78, 0x50, 0x72, 0x65, 0x4f, 0x77, 0x6e, 0x65, 0x64, 0x49, 0x6e, 0x76, 0x65, 0x6e, 0x74, 0x6f, 0x72, 0x79, 0x44, 0x65, 0x74, 0x61, 0x69, 0x6c, 0x26, 0x69, 0x64, 0x3d, 0x36, 0x35, 0x38, 0x31, 0x38, 0x32, 0x31, 0x26, 0x70, 0x3d, 0x33, 0x30, 0x26, 0x73, 0x3d, 0x59, 0x65, 0x61, 0x72, 0x26, 0x64, 0x3d, 0x44, 0x26, 0x76, 0x74, 0x3d, 0x6d, 0x6f, 0x74, 0x6f, 0x72, 0x63, 0x79, 0x63, 0x6c, 0x65, 0x25, 0x32, 0x30, 0x25, 0x32, 0x46, 0x25, 0x32, 0x30, 0x73, 0x63, 0x6f, 0x6f, 0x74, 0x65, 0x72, 0x26, 0x66, 0x72, 0x3d, 0x78, 0x50, 0x72, 0x65, 0x4f, 0x77, 0x6e, 0x65, 0x64, 0x49, 0x6e, 0x76, 0x65, 0x6e, 0x74, 0x6f, 0x72, 0x79}, “year”, []uint8{0x32, 0x30, 0x30, 0x35}, “geo”, []uint8{0x2d, 0x38, 0x31, 0x2e, 0x33, 0x37, 0x39, 0x32, 0x33, 0x36, 0x35, 0x20, 0x32, 0x38, 0x2e, 0x35, 0x33, 0x38, 0x33, 0x33, 0x35, 0x35}, “st”, []uint8{0x46, 0x4c}, “ctitle”, []uint8{0x6c}, “location”, []uint8{0x4f, 0x72, 0x6c, 0x61, 0x6e, 0x64, 0x6f, 0x2c, 0x20, 0x46, 0x4c}, “tags”, []uint8{0x77, 0x3a, 0x63, 0x6f, 0x75, 0x63, 0x68, 0x72, 0x69, 0x64, 0x65, 0x3a, 0x74, 0x3a, 0x64, 0x65, 0x61, 0x6c, 0x65, 0x72, 0x2c, 0x77, 0x3a, 0x63, 0x6f, 0x75, 0x63, 0x68, 0x72, 0x69, 0x64, 0x65, 0x3a, 0x74, 0x3a, 0x75, 0x73, 0x65, 0x64, 0x2c, 0x77, 0x3a, 0x63, 0x6f, 0x75, 0x63, 0x68, 0x72, 0x69, 0x64, 0x65, 0x3a, 0x74, 0x3a, 0x68, 0x61, 0x73, 0x76, 0x69, 0x6e, 0x2c, 0x77, 0x3a, 0x63, 0x6f, 0x75, 0x63, 0x68, 0x72, 0x69, 0x64, 0x65, 0x3a, 0x74, 0x3a, 0x67, 0x6f, 0x6c, 0x64, 0x77, 0x69, 0x6e, 0x67, 0x2c, 0x77, 0x3a, 0x63, 0x6f, 0x75, 0x63, 0x68, 0x72, 0x69, 0x64, 0x65, 0x3a, 0x74, 0x3a, 0x68, 0x6f, 0x6e, 0x64, 0x61, 0x2c, 0x77, 0x3a, 0x63, 0x6f, 0x75, 0x63, 0x68, 0x72, 0x69, 0x64, 0x65, 0x2c, 0x66, 0x3a, 0x69, 0x6d, 0x6f, 0x74, 0x6f, 0x72, 0x73, 0x70, 0x6f, 0x72, 0x74, 0x73, 0x75, 0x73, 0x65, 0x64, 0x2c, 0x63, 0x66, 0x3a, 0x64, 0x3a, 0x31, 0x30, 0x39, 0x2c, 0x63, 0x66, 0x3a, 0x6d, 0x3a, 0x64, 0x73, 0x2c, 0x46, 0x4c}, “mpid”, []uint8{0x63, 0x66, 0x3a, 0x6d, 0x3a, 0x64, 0x73}, “title”, []uint8{0x32, 0x30, 0x30, 0x35, 0x20, 0x48, 0x6f, 0x6e, 0x64, 0x61, 0xc2, 0xae, 0x20, 0x47, 0x6f, 0x6c, 0x64, 0x20, 0x57, 0x69, 0x6e, 0x67, 0x20, 0x54, 0x72, 0x69, 0x6b, 0x65}, “feed”, []uint8{0x66, 0x3a, 0x69, 0x6d, 0x6f, 0x74, 0x6f, 0x72, 0x73, 0x70, 0x6f, 0x72, 0x74, 0x73, 0x75, 0x73, 0x65, 0x64}, “mileage”, []uint8{0x34, 0x30, 0x30, 0x30, 0x30}, “guid”, []uint8{0x36, 0x35, 0x38, 0x31, 0x38, 0x32, 0x31}, “did”, []uint8{0x63, 0x66, 0x3a, 0x64, 0x3a, 0x31, 0x30, 0x39}}}

But if the response contained multiple documents, you fill-in the values as nil for all documents that didn’t have them.

This is the same document when it’s the second result in the response. It doesn’t have an imgy field, yet you return one. This also incurs performance penalty - although probably negligible.

[]interface {}{[]interface {}{"_t", []uint8{0x31, 0x35, 0x34, 0x37, 0x37, 0x35, 0x34, 0x34, 0x39, 0x32}, “imgy”, interface {}(nil), “ctitle”, []uint8{0x6c}, “location”, []uint8{0x4f, 0x72, 0x6c, 0x61, 0x6e, 0x64, 0x6f, 0x2c, 0x20, 0x46, 0x4c}, “ph”, interface {}(nil), “mpid”, []uint8{0x63, 0x66, 0x3a, 0x6d, 0x3a, 0x64, 0x73}, “tags”, []uint8{0x77, 0x3a, 0x63, 0x6f, 0x75, 0x63, 0x68, 0x72, 0x69, 0x64, 0x65, 0x3a, 0x74, 0x3a, 0x64, 0x65, 0x61, 0x6c, 0x65, 0x72, 0x2c, 0x77, 0x3a, 0x63, 0x6f, 0x75, 0x63, 0x68, 0x72, 0x69, 0x64, 0x65, 0x3a, 0x74, 0x3a, 0x75, 0x73, 0x65, 0x64, 0x2c, 0x77, 0x3a, 0x63, 0x6f, 0x75, 0x63, 0x68, 0x72, 0x69, 0x64, 0x65, 0x3a, 0x74, 0x3a, 0x68, 0x61, 0x73, 0x76, 0x69, 0x6e, 0x2c, 0x77, 0x3a, 0x63, 0x6f, 0x75, 0x63, 0x68, 0x72, 0x69, 0x64, 0x65, 0x3a, 0x74, 0x3a, 0x67, 0x6f, 0x6c, 0x64, 0x77, 0x69, 0x6e, 0x67, 0x2c, 0x77, 0x3a, 0x63, 0x6f, 0x75, 0x63, 0x68, 0x72, 0x69, 0x64, 0x65, 0x3a, 0x74, 0x3a, 0x68, 0x6f, 0x6e, 0x64, 0x61, 0x2c, 0x77, 0x3a, 0x63, 0x6f, 0x75, 0x63, 0x68, 0x72, 0x69, 0x64, 0x65, 0x2c, 0x66, 0x3a, 0x69, 0x6d, 0x6f, 0x74, 0x6f, 0x72, 0x73, 0x70, 0x6f, 0x72, 0x74, 0x73, 0x75, 0x73, 0x65, 0x64, 0x2c, 0x63, 0x66, 0x3a, 0x64, 0x3a, 0x31, 0x30, 0x39, 0x2c, 0x63, 0x66, 0x3a, 0x6d, 0x3a, 0x64, 0x73, 0x2c, 0x46, 0x4c}, “currency”, interface {}(nil), “created”, []uint8{0x32, 0x30, 0x31, 0x39, 0x2d, 0x30, 0x31, 0x2d, 0x31, 0x37, 0x20, 0x31, 0x39, 0x3a, 0x34, 0x38, 0x3a, 0x31, 0x32}, “imgx”, interface {}(nil), “url”, []uint8{0x68, 0x74, 0x74, 0x70, 0x73, 0x3a, 0x2f, 0x2f, 0x77, 0x77, 0x77, 0x2e, 0x69, 0x6d, 0x6f, 0x74, 0x6f, 0x72, 0x73, 0x70, 0x6f, 0x72, 0x74, 0x73, 0x6f, 0x72, 0x6c, 0x61, 0x6e, 0x64, 0x6f, 0x2e, 0x63, 0x6f, 0x6d, 0x2f, 0x64, 0x65, 0x66, 0x61, 0x75, 0x6c, 0x74, 0x2e, 0x61, 0x73, 0x70, 0x3f, 0x70, 0x61, 0x67, 0x65, 0x3d, 0x78, 0x50, 0x72, 0x65, 0x4f, 0x77, 0x6e, 0x65, 0x64, 0x49, 0x6e, 0x76, 0x65, 0x6e, 0x74, 0x6f, 0x72, 0x79, 0x44, 0x65, 0x74, 0x61, 0x69, 0x6c, 0x26, 0x69, 0x64, 0x3d, 0x36, 0x35, 0x38, 0x31, 0x38, 0x32, 0x31, 0x26, 0x70, 0x3d, 0x33, 0x30, 0x26, 0x73, 0x3d, 0x59, 0x65, 0x61, 0x72, 0x26, 0x64, 0x3d, 0x44, 0x26, 0x76, 0x74, 0x3d, 0x6d, 0x6f, 0x74, 0x6f, 0x72, 0x63, 0x79, 0x63, 0x6c, 0x65, 0x25, 0x32, 0x30, 0x25, 0x32, 0x46, 0x25, 0x32, 0x30, 0x73, 0x63, 0x6f, 0x6f, 0x74, 0x65, 0x72, 0x26, 0x66, 0x72, 0x3d, 0x78, 0x50, 0x72, 0x65, 0x4f, 0x77, 0x6e, 0x65, 0x64, 0x49, 0x6e, 0x76, 0x65, 0x6e, 0x74, 0x6f, 0x72, 0x79}, “title”, []uint8{0x32, 0x30, 0x30, 0x35, 0x20, 0x48, 0x6f, 0x6e, 0x64, 0x61, 0xc2, 0xae, 0x20, 0x47, 0x6f, 0x6c, 0x64, 0x20, 0x57, 0x69, 0x6e, 0x67, 0x20, 0x54, 0x72, 0x69, 0x6b, 0x65}, “imgurl”, interface {}(nil), “feed”, []uint8{0x66, 0x3a, 0x69, 0x6d, 0x6f, 0x74, 0x6f, 0x72, 0x73, 0x70, 0x6f, 0x72, 0x74, 0x73, 0x75, 0x73, 0x65, 0x64}, “mileage”, []uint8{0x34, 0x30, 0x30, 0x30, 0x30}, “price”, interface {}(nil), “imgsmallx”, interface {}(nil), “imgurlsmall”, interface {}(nil), “imgsmally”, interface {}(nil), “st”, []uint8{0x46, 0x4c}, “geo”, []uint8{0x2d, 0x38, 0x31, 0x2e, 0x33, 0x37, 0x39, 0x32, 0x33, 0x36, 0x35, 0x20, 0x32, 0x38, 0x2e, 0x35, 0x33, 0x38, 0x33, 0x33, 0x35, 0x35}, “vin”, []uint8{0x31, 0x68, 0x66, 0x73, 0x63, 0x34, 0x37, 0x30, 0x78, 0x35, 0x61, 0x34, 0x31, 0x31, 0x32, 0x33, 0x33}, “year”, []uint8{0x32, 0x30, 0x30, 0x35}, “guid”, []uint8{0x36, 0x35, 0x38, 0x31, 0x38, 0x32, 0x31}, “did”, []uint8{0x63, 0x66, 0x3a, 0x64, 0x3a, 0x31, 0x30, 0x39}}}

Michael

6

This sounds like a potential bug, or at least something that has the potential to add more network overhead: https://github.com/RedisLabsModules/RediSearch/issues/598

In any event, it’s recommended to use explicit RETURN statements anyway;
Mark Nunberg | Senior Software Engineer
Redis Labs - home of Redis

Email: mark@redislabs.com