FAISS GPU Installation

FAISS (Facebook AI Similarity Search) is a C++ library with Python bindings for nearest-neighbour search over dense vectors, the operation behind semantic search and the retrieval step of RAG. The faiss-cpu wheel on PyPI installs in seconds, but GPU indexes require a build compiled against your CUDA toolkit and your GPU's compute capability. This page walks through compiling FAISS from source with GPU support on Ubuntu, explains what each step does, and shows how to confirm the GPU path actually works.

1. Why build from source

Prebuilt GPU packages exist through conda (the pytorch and conda-forge channels publish faiss-gpu builds), and if one matches your CUDA version it is the easiest route. Building from source is worth it when you use pip-only environments, need a CUDA version no package targets, want the build optimized for exactly one GPU architecture (smaller binaries, faster compile), or need to patch the library. The trade-off is a long compile and a handful of version constraints between CMake, the compiler, CUDA, and SWIG that must line up.

2. How the build fits together

FAISS GPU build pipeline Dependencies cmake, OpenBLAS, OpenMP, gcc/g++ CUDA toolkit nvcc + libs CUDA_HOME on PATH CMake configure GPU=ON, arch 86, Release make -j libfaiss + SWIG bindings Verify import faiss GPU index search CMAKE_CUDA_ARCHITECTURES=86 targets Ampere consumer GPUs such as the RTX 3060; use your card's compute capability.

OpenBLAS supplies the matrix-multiply kernels used by CPU indexes and training; OpenMP parallelizes CPU loops; nvcc compiles the CUDA kernels for GPU indexes; SWIG generates the Python wrapper around the C++ classes. The CMake flags select which pieces get built. Python bindings are controlled by FAISS_ENABLE_PYTHON, which defaults to on and needs SWIG and NumPy available at configure time; after make, the bindings are installed with cd build/faiss/python && pip install . (or reached through PYTHONPATH as in Step 9).

3. Build steps 1-9

Step 1: Install Dependencies

Run the following commands to install the necessary dependencies:


sudo apt-get update
sudo apt-get install -y cmake libopenblas-dev libomp-dev libgtest-dev gcc-10 g++-10

Step 2: Install CUDA

If CUDA is not installed, follow these steps to install it:


sudo apt-get install cuda-12-2

Set up environment variables for CUDA:


export CUDA_HOME=/usr/local/cuda-12.2
export PATH=$CUDA_HOME/bin:$PATH
export LD_LIBRARY_PATH=$CUDA_HOME/lib64:$LD_LIBRARY_PATH

Step 3: Clone FAISS Repository


git clone https://github.com/facebookresearch/faiss.git
cd faiss

Step 4: File Editing - Modify CMakeLists.txt

Edit the CMakeLists.txt file to ensure proper configuration:


vi faiss/CMakeLists.txt

Add the following lines at the top:


cmake_minimum_required(VERSION 3.27)
project(Faiss VERSION 1.7.0 LANGUAGES C CXX)

Fix the path for `faiss-config.cmake.in` by editing the line around line 410 in `CMakeLists.txt`:


configure_file(${CMAKE_SOURCE_DIR}/cmake/faiss-config.cmake.in
               ${CMAKE_BINARY_DIR}/faiss-config.cmake @ONLY)

Step 5: Build FAISS with GPU Support


cmake -B build -DFAISS_ENABLE_GPU=ON -DCMAKE_CUDA_COMPILER=/usr/local/cuda-12.2/bin/nvcc -DCMAKE_CUDA_ARCHITECTURES=86 -DBUILD_TESTING=OFF -DCMAKE_BUILD_TYPE=Release
make -j$(nproc)

Step 6: Install FAISS (Optional)


sudo make install

Step 7: Verify FAISS Installation

Run the following test script to verify FAISS installation:


import faiss
import numpy as np

# Create random vectors
d = 128  # dimension
nb = 1000  # number of vectors
xb = np.random.random((nb, d)).astype('float32')

# Create an index for L2 distance
index = faiss.IndexFlatL2(d)

# Add vectors
index.add(xb)

# Search for nearest neighbors
D, I = index.search(xb[:1], 5)
print(f"Distances: {D}")
print(f"Indices: {I}")

Step 8: Install SWIG (if not installed)

If you encounter issues with Python bindings, install SWIG:


sudo apt-get install swig

Step 9: Set PYTHONPATH (Optional)

If FAISS Python bindings aren't being picked up, set the PYTHONPATH:


export PYTHONPATH=/home/open-webui/faiss/build/faiss/python:$PYTHONPATH

4. Worked example: using the GPU index

The Step 7 script only proves the CPU index works. This one proves the GPU build is in use by checking the GPU count and moving an index onto device 0:

import faiss
import numpy as np

print("GPUs visible to FAISS:", faiss.get_num_gpus())   # must be >= 1

d, nb, nq = 128, 100_000, 5
rng = np.random.default_rng(0)
xb = rng.random((nb, d), dtype=np.float32)
xq = rng.random((nq, d), dtype=np.float32)

res = faiss.StandardGpuResources()                 # scratch memory and CUDA streams
cpu_index = faiss.IndexFlatL2(d)
gpu_index = faiss.index_cpu_to_gpu(res, 0, cpu_index)
gpu_index.add(xb)
D, I = gpu_index.search(xq, 5)
print(I[0], D[0])

# Move back to CPU to save to disk
faiss.write_index(faiss.index_gpu_to_cpu(gpu_index), "vectors.index")

If get_num_gpus() returns 0 or StandardGpuResources is missing, Python is importing a CPU-only FAISS, usually a pip faiss-cpu wheel that shadows the source build. Check faiss.__file__ to see which copy was loaded.

5. Choosing an index type

Common FAISS index types IndexFlatL2 / IP exact search no training memory: full vectors GPU: yes IndexIVFFlat partitions into nlist cells train on a sample search nprobe cells GPU: yes IndexIVFPQ IVF + product quantization compressed vectors approximate distances GPU: yes IndexHNSWFlat graph-based no training high recall, more RAM GPU: CPU only Start with Flat for up to roughly a million vectors; move to IVF or IVF-PQ when memory or latency demands it.

Flat indexes compare the query with every stored vector, which on a GPU is fast enough for surprisingly large collections and gives exact results. IVF indexes cluster vectors with k-means and search only the closest nprobe clusters, trading recall for speed. Product quantization compresses each vector to a few bytes, letting very large collections fit in GPU memory. HNSW gives excellent recall-speed trade-offs on CPU but has no GPU implementation in FAISS itself.

6. Pitfalls