zsz00/Faiss.jl: Julia wrapper around the Faiss library for similarity search with PythonCall.jl (github.com) A simple Julia wrapper around the Faiss library for similarity search with PythonCall.jl . While functional and faster then NearestNeighbors.jl . Faiss is a library for efficient similar…

Sounds cool! Do you have any benchmarks showing the performance difference between Faiss and NearestNeighbors?

A simple test comparison : ENV["JULIA_PYTHONCALL_EXE"] = "/home/xx/miniconda3/bin/python" using Faiss using NearestNeighbors function faiss_test_1() feats = rand(10^6, 128); top_k = 100 vs_gallery = feats; vs_query = feats[1:10^4, :]; D, I = local_rank(vs_query, vs_gallery, k=t…

A simple analogy: NearestNeighbors.jl vs Faiss such as Flux.jl vs PyTorch.

I am not sure that I entirely agree here. Your benchmark above suggests that Faiss is about 40x faster than NearestNeighbors, while some benchmarks here: Flux – Accelerating Flux.jl with PyTorch kernels suggest that Torch is about 2-3x faster than Flux. Also, I am not entirely convinced that your b…

[image] zsz00: A simple Julia wrapper around the Faiss library for similarity search with PythonCall.jl . Ideally, you’d skip Python and call Faiss’s underlying C API directly (building a Faiss binary with Yggdrasil), no?

That would be best, but I don’t know how to do it. Currently this way of wrapping Python is the simplest and compatible with the Python interface

Yes, NN can only use single thread, faiss automatically uses multiple threads. One of the reasons for not making a strict comparison is that NN has too few features, does not support multithreading, does not support GPU, and does not support various indexes . Dynamic index and add data are not sup…

One could test Faiss against IVFADC.jl which uses ANN search (hierarchical indexing) and product quantization for vector residuals.

Having a vector search algorithm single-threaded and CPU-only can also be an extremely powerful feature: simpler implementation, easier to reason about, no GPU dependency (good for low-resource devices), parallelization at a higher abstraction level (i.e. process), which may be desirable in environm…

Another thing to consider is, the scaling of search algorithms with data dimension. (both nominal and effective) Here’s another ANN library that I implemented a while back: https://github.com/JuliaNeighbors/HNSW.jl

[ANN] Faiss.jl, similarity search

General Usage

fabiangans March 21, 2022, 9:25am 13

Recently there was this package announced as well:

Topic		Replies	Views
[ANN] IVFADC.jl inverse file system for billion-scale ANN search Package Announcements	0	492	July 31, 2019
SimilaritySearch.jl: A package for searching approximate nearest neighbors efficiently Package Announcements package	8	1604	July 5, 2022
[ANN] VPTrees.jl - nearest neighbor search in metric spaces Package Announcements	20	2890	January 30, 2020
Vector databases General Usage vector-search	5	670	June 20, 2024
GPU based Nearest Neighbor (brute force) General Usage question	5	319	February 23, 2026

[ANN] Faiss.jl, similarity search

Related topics