KNN Search returning incorrect results beyond top 14

Redis modules RediSearch
1

Hi,

I’m running Redis (version 6.2.7) locally and testing the KNN Search feature. I’m indexing JSON documents and there are around 1822 documents in total. I’m using Redis Python client and the schema is as follows:

VectorField("embedding",                        # Vector Field Name
      "FLAT", {                                 # Vector Index Type: FLAT or HNSW
          "TYPE": "FLOAT64",                    # FLOAT32 or FLOAT64
          "DIM": 73,                            # Number of Vector Dimensions
          "DISTANCE_METRIC": "COSINE",   # Vector Search Distance Metric
      },
      as_name="embedding",
  )

Then when performing the search I’m using this query:

query = Query("*=>[KNN 14 @embedding $input_vec as score]").sort_by("score").paging(0, 20).dialect(2)

query_params = {
  "input_vec": np.array(doc.embedding).astype(np.float64).tobytes()
}

Here I’m requesting top 14 results and the scores are the following:

scores:  ['2.22044604925e-16', '0.0541322040298', '0.0615432837741', '0.0668670014771', '0.0742917691643', '0.116318601842', '0.131113306394', '0.131141935557', '0.131271860736', '0.13733404726', '0.137493637436', '0.140023820393', '0.165991334733', '0.169605765458']

As you can see the results are sorted in increasing order.

Now if I replace 14 with 15 (for top 15 results), then I’m seeing a completely different set of results:

scores:  ['0.244267143235', '0.403415188104', '0.471164056214', '0.482207966244', '0.513948161904', '0.53753812203', '0.565956169968', '0.649693288582', '0.721081956076', '0.783667936764', 'nan', '0.783887564905', '0.795059643469', '0.802236995757', '0.802672793199']

This happens to all input vectors and the break point seems to be 15. For K < 15, the results are as expected. Any idea why this is happening?

2

Hi @addarsh
Can you provide a sample dataset for reproducing the issue?