# How to Use GPU Servers to Generate Embeddings for Weaviate Vector Search on Ubuntu 24.04

> URL: https://www.atlantic.net/gpu-server-hosting/how-to-use-gpu-servers-to-generate-embeddings-for-weaviate-vector-search-on-ubuntu-24-04/ | Published: 2025-07-05 | Updated: 2026-05-04 | Author: Hitesh Jethva

Weaviate is a powerful open-source vector database designed to perform semantic search using vector embeddings. When you integrate it with [GPU-powered servers](https://www.atlantic.net/gpu-server-hosting/), embedding generation becomes significantly faster, especially for large-scale or real-time applications.

In this guide, you’ll learn how to:

- Set up Weaviate using Docker
- Leverage a GPU for fast embedding generation
- Store custom embeddings in Weaviate
- Query and manage vectorized data using Python scripts

## Prerequisites

- An Ubuntu 24.04 server with an NVIDIA GPU.
- A non-root user or a user with sudo privileges.
- NVIDIA drivers are installed on your server.

## Step 1: Install Required Dependencies

First, install Python and the necessary tools.

```bash
apt install python3 python3-pip python3-venv -y
```

Restart Docker to ensure GPU support.

```bash
systemctl restart docker
```

Verify that the NVIDIA container runtime is available.

```bash
docker info | grep -i runtime
```

Output.

```bash
 Runtimes: io.containerd.runc.v2 nvidia runc
 Default Runtime: runc
```

## Step 2: Set Up Weaviate with Docker

Configure and launch Weaviate using Docker Compose, ensuring proper port exposure for both REST and gRPC connections that will be used for vector operations.

Create a **docker-compose.yml** file.

```bash
nano docker-compose.yml
```

Add the below configuration.

```bash
services:
  weaviate:
    image: semitechnologies/weaviate:latest
    ports:
      - "8080:8080"
      - "50051:50051"   # <-- required for gRPC
    environment:
      ENABLE_MODULES: ''
      DEFAULT_VECTORIZER_MODULE: 'none'
      QUERY_DEFAULTS_LIMIT: 25
      AUTHENTICATION_ANONYMOUS_ACCESS_ENABLED: 'true'
```

Start Weaviate container using the below command.

```bash
docker-compose up -d
```

Verify the running container.

```bash
docker ps
```

Output.

```bash
CONTAINER ID   IMAGE                              COMMAND                  CREATED          STATUS          PORTS                                                                                          NAMES
2166c9434c83   semitechnologies/weaviate:latest   "/bin/weaviate --hos…"   18 minutes ago   Up 18 minutes   0.0.0.0:8080->8080/tcp, [::]:8080->8080/tcp, 0.0.0.0:50051->50051/tcp, [::]:50051->50051/tcp   root-weaviate-1
```

## Step 3: Set Up a Python Virtual Environment

In this section, we will create an isolated Python environment to manage dependencies cleanly and verify that PyTorch can properly access your GPU hardware.

Create a Python virtual environment.

```bash
python3 -m venv venv
```

Activate the environment.

```bash
source venv/bin/activate
```

Update pip to the latest version.

```bash
pip install --upgrade pip
```

Install required Python packages.

```bash
pip install sentence-transformers weaviate-client torch torchvision
```

Verify GPU availability.

```bash
python3 -c "import torch; print(torch.cuda.is_available())"
```

Output.

```bash
True
```

## Step 4: Generate Embeddings Using GPU

In this section, we will create a Python script that leverages GPU acceleration to convert text into high-dimensional vectors using the sentence-transformers library.

Create **generate_embeddings.py.**

```bash
nano generate_embeddings.py
```

Add the following code.

```bash
from sentence_transformers import SentenceTransformer
import torch

device = "cuda" if torch.cuda.is_available() else "cpu"
model = SentenceTransformer('all-MiniLM-L6-v2', device=device)

def get_embedding(text):
    embedding = model.encode(text, convert_to_tensor=True)
    return embedding.cpu().numpy().tolist()
```

Run the script.

```bash
python3 generate_embeddings.py
```

## Step 5: Insert Data into Weaviate

In this section, we will create and populate a Weaviate collection with your text documents and their corresponding GPU-generated vector embeddings.

Create **insert_to_weaviate.py.**

```bash
nano insert_to_weaviate.py
```

Add the following code.

```bash
from weaviate.connect import ConnectionParams
from weaviate import WeaviateClient
from weaviate.collections.classes.config import DataType
from generate_embeddings import get_embedding

# Connect to Weaviate
connection_params = ConnectionParams.from_url("http://localhost:8080", 50051)
client = WeaviateClient(connection_params)
client.connect()

# Define class name
class_name = "Document"

# Create collection if not exists
if not client.collections.exists(class_name):
    client.collections.create(
        name=class_name,
        vectorizer_config=None,  # Use custom embeddings
        properties=[
            {"name": "content", "data_type": DataType.TEXT}  # ✅ Correct enum
        ]
    )

# Sample texts to insert
texts = [
    "Weaviate is a vector database.",
    "GPU servers are great for deep learning.",
    "Embeddings help convert text into searchable vectors."
]

collection = client.collections.get(class_name)

for text in texts:
    vector = get_embedding(text)
    collection.data.insert(
        properties={"content": text},
        vector=vector
    )

# Close the connection properly
client.close()
```

Run the script.

```bash
python3 insert_to_weaviate.py
```

## Step 6: Query Data in Weaviate

In this section, we will execute vector similarity searches against your Weaviate database to retrieve relevant documents based on semantic meaning rather than exact keyword matches.

Create **search_weaviate.py.**

```bash
nano search_weaviate.py
```

Add the following code.

```bash
from weaviate.connect import ConnectionParams
from weaviate import WeaviateClient
from generate_embeddings import get_embedding

# Connect to Weaviate
connection_params = ConnectionParams.from_url("http://localhost:8080", 50051)
client = WeaviateClient(connection_params)
client.connect()

collection = client.collections.get("Document")

# Query
query = "What is Weaviate?"
query_vector = get_embedding(query)

# Perform search
results = collection.query.near_vector(
    near_vector=query_vector,
    limit=3,
    return_properties=["content"]
)

# Print results
print("\nSearch Results:")
for result in results.objects:
    print(f"- {result.properties['content']}")

client.close()
```

Run the script.

```bash
python3 search_weaviate.py
```

You will see the following output.

```bash
Search Results:
- Weaviate is a vector database.
- Embeddings help convert text into searchable vectors.
- GPU servers are great for deep learning.
```

## Step 7: Manage Documents

In this section, we will learn additional operations for maintaining your vector database, including listing, updating, and removing documents as needed.

First, create a script to list all documents.

```bash
nano list_documents.py
```

Add the following code.

```bash
from weaviate.connect import ConnectionParams
from weaviate import WeaviateClient

connection_params = ConnectionParams.from_url("http://localhost:8080", 50051)
client = WeaviateClient(connection_params)
client.connect()

collection = client.collections.get("Document")

# Replace with actual UUID
doc_id = "REPLACE-WITH-UUID"

# Delete the document
success = collection.data.delete(uuid=doc_id)
print("Deleted successfully?" , success)

client.close()
```

Run the script.

```bash
python3 list_documents.py
```

You will see all documents in the output below.

```bash
Documents in 'Document' collection:
- ID: 2189ca0b-ebdb-437a-90f1-d2881bfbbbd7 | Content: Weaviate is a vector database.
- ID: 296db11b-b2bc-4554-a236-2fee50579c2d | Content: GPU servers are great for deep learning.
- ID: dde5a358-058b-46f3-9033-47967df64a01 | Content: Embeddings help convert text into searchable vectors.
```

Create an **update_document.py** script to update the existing document.

```bash
nano update_document.py
```

Add the below code.

```bash
from weaviate.connect import ConnectionParams
from weaviate import WeaviateClient
from weaviate.collections.classes.config import DataType
from generate_embeddings import get_embedding

# Connect to Weaviate
connection_params = ConnectionParams.from_url("http://localhost:8080", 50051)
client = WeaviateClient(connection_params)
client.connect()

try:
    collection = client.collections.get("Document")

    # UUID of the document to update (replace with actual ID)
    doc_id = "296db11b-b2bc-4554-a236-2fee50579c2d"

    # New content
    new_text = "Updated description about vector databases."

    # Generate new embedding
    new_vector = get_embedding(new_text)

    # Delete existing object by ID
    collection.data.delete_by_id(doc_id)

    # Re-insert object with same ID and new content + vector
    collection.data.insert(
        uuid=doc_id,
        properties={"content": new_text},
        vector=new_vector
    )

    print("✅ Document updated successfully.")

finally:
    client.close()
```

Replaced the id **“296db11b-b2bc-4554-a236-2fee50579c2d”** with the actual document.

Run the script.

```bash
python3 update_document.py
```

Output.

```bash
✅ Document updated successfully.
```

Create a **delete_document.py** to delete the existing document.

```bash
nano delete_document.py
```

Add the below code.

```bash
from weaviate.connect import ConnectionParams
from weaviate import WeaviateClient

# Connect to Weaviate
connection_params = ConnectionParams.from_url("http://localhost:8080", 50051)
client = WeaviateClient(connection_params)
client.connect()

try:
    # Get the 'Document' collection
    collection = client.collections.get("Document")

    # Replace this with the actual UUID you want to delete
    doc_id = "2189ca0b-ebdb-437a-90f1-d2881bfbbbd7"

    # Delete the object by UUID
    deleted = collection.data.delete_by_id(doc_id)

    if deleted:
        print(f"✅ Document with ID {doc_id} deleted successfully.")
    else:
        print(f"❌ Document with ID {doc_id} not found or could not be deleted.")

finally:
    client.close()
```

Run the script.

```bash
python3 delete_document.py
```

Output.

```bash
✅ Document with ID 2189ca0b-ebdb-437a-90f1-d2881bfbbbd7 deleted successfully.
```

## Conclusion

Congratulations! You’ve successfully created a powerful vector search system that combines Weaviate’s efficient similarity search capabilities with GPU-accelerated embedding generation. This setup demonstrates how modern hardware can dramatically improve the performance of semantic search applications.