1. Create embeddings of your db entries by running through a nn in inference mode, save in a database in vector format.
2. Convert your query to an embedding by running it through a neural network in inference mode
3. Perform a nearest neighbor search of your query embedding with your db embeddings. There are also databases optimized for this, for example FAISS by meta/fb [1].
So if your network is already trained or you use something like OpenAI for embeddings, it can still be done in near real time just think of getting your embedding vector as part of the indexing process.
You can do more things too, like cluster your embeddings db to find similar entries.
1. Create embeddings of your db entries by running through a nn in inference mode, save in a database in vector format.
2. Convert your query to an embedding by running it through a neural network in inference mode
3. Perform a nearest neighbor search of your query embedding with your db embeddings. There are also databases optimized for this, for example FAISS by meta/fb [1].
So if your network is already trained or you use something like OpenAI for embeddings, it can still be done in near real time just think of getting your embedding vector as part of the indexing process.
You can do more things too, like cluster your embeddings db to find similar entries.
[1] https://engineering.fb.com/2017/03/29/data-infrastructure/fa...