support pgvector v0.7.0

[jeffrey82221]

Hi,

Is it possible to upgrade the pgvector version to v0.7.0? I am building projects leveraging pgserver
but encounter difficulties in slow querying for embedding size > 2000.

BTW, here are some new capability of v0.7.0 I found:

pgvector Version Support for High-Dimensional Embedding Indexing

Introduction to Dimension Limitations in Vector Indexing

The pgvector extension for PostgreSQL historically imposed a 2,000-dimension limit for indexed vectors due to fundamental database architecture constraints. This restriction stemmed from PostgreSQL's 8KB page size and the memory requirements of 4-byte floating-point numbers. With the rise of advanced embedding models like OpenAI's text-embedding-3-large (3,072D) and Mistral derivatives (4,096D), this limitation became a critical barrier for AI/ML applications.

Breakthrough in pgvector 0.7.0

The 0.7.0 release (April 2024) introduced three technical innovations that overcome previous dimension constraints:

1. Half-Precision Vector Support (halfvec)

• Indexing Capacity: 4,000 dimensions
• Storage Optimization: 2-byte floats instead of 4-byte
• Memory Efficiency: 50% reduction in index memory footprint
• Usage:

  CREATE TABLE embeddings (
    id SERIAL PRIMARY KEY,
    vector halfvec(3072)
  );
  
  CREATE INDEX ON embeddings USING hnsw (vector halfvec_l2_ops);

2. Binary Quantization (bit)

• Dimension Support: Up to 64,000 dimensions
• Storage Efficiency: 1-bit per dimension
• Implementation:

  CREATE INDEX ON embeddings USING hnsw (
    binary_quantize(vector) bit_hamming_ops
  );

3. Sparse Vector Support (sparsevec)

• Effective Capacity: 1,000 non-zero elements
• Storage Optimization: 8 bytes per non-zero element
• Usage:

  CREATE TABLE sparse_embeds (
    id SERIAL PRIMARY KEY,
    vector sparsevec(16000)
  );

Technical Comparison of Indexing Methods

Feature	vector (pre-0.7.0)	halfvec (0.7.0+)	bit (0.7.0+)	sparsevec (0.7.0+)
Max Indexed Dimensions	2,000	4,000	64,000	1,000 non-zero
Storage per Dimension	4 bytes	2 bytes	1 bit	8 bytes/non-zero
Distance Metrics	L2, Cosine, IP	L2, Cosine, IP	Hamming, Jaccard	L2, Cosine
Index Size Reduction	-	50%	98%	90-99%
Recall Impact	-	<2% drop[8]	<5% drop[13]	Model-dependent

[adsharma]

pgvector-v0.8.0 wheels here

https://github.com/adsharma/pgserver/actions/runs/17194383449/job/48774327676

[adsharma]

https://github.com/Ladybug-Memory/pgembed is a fork with wheels available via https://pypi.org/project/pgembed/