# How to Deploy a Scalable Semantic Search Application with Deepset.ai on Atlantic.Net GPU Server

> URL: https://www.atlantic.net/gpu-server-hosting/how-to-deploy-a-scalable-semantic-search-application-with-deepset-ai-on-atlantic-net-gpu-server/ | Published: 2025-03-11 | Updated: 2026-05-04 | Author: Hitesh Jethva

Semantic search is a powerful technique that improves information retrieval by understanding the meaning behind queries and documents. With the help of Deepset.ai’s Haystack framework, you can build and deploy a scalable semantic search application on an Ubuntu GPU server.

This article will guide you through the process to deploy a Scalable Semantic Search Application with Deepset.ai’s Haystack framework on Ubuntu [GPU server](https://www.atlantic.net/gpu-server-hosting/).

## Prerequisites

Before proceeding, ensure you have the following:

- An Atlantic.Net Cloud GPU server running Ubuntu 24.04.
- CUDA Toolkit and cuDNN Installed.
- A root or sudo privileges.

## Step 1: Set Up the Environment

1. Install Python Dependencies:

```bash
apt install python3 python3-pip python3-venv
```

2. Create a virtual environment:

```bash
python3 -m venv haystack-env
source haystack-env/bin/activate
```

3. Install Haystack and other dependencies:

```bash
pip install farm-haystack[elasticsearch,gpu] sentence-transformers torch
```

4. Ensure PyTorch can detect the GPU:

```bash
python3
```

Write the following in Python shell:

```bash
>>> import torch
>>> print(torch.cuda.is_available())
```

Output.

```bash
True
```

Press CTRL+D to exit from the Python shell.

## Step 2: Install Docker and NVIDIA Container Toolkit

Docker simplifies deployment, and the NVIDIA Container Toolkit allows Docker containers to use GPU resources.

1. Install Docker.

```bash
apt install docker.io
```

2. Start and enable the Docker service.

```bash
systemctl start docker
systemctl enable docker
```

3. Install NVIDIA Container Toolkit.

```bash
distribution=$(. /etc/os-release;echo $ID$VERSION_ID)
curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | apt-key add -
curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.list | tee /etc/apt/sources.list.d/nvidia-docker.list
apt update && apt install -y nvidia-container-toolkit
systemctl restart docker
```

4. Haystack supports various document stores like Elasticsearch, FAISS, and Weaviate. For this tutorial, we’ll use Elasticsearch. Run Elasticsearch Docker container.

```bash
docker run -d -p 9200:9200 -e "discovery.type=single-node" elasticsearch:7.9.2
```

5. Verify the Elasticsearch.

```bash
curl -X GET "http://localhost:9200/"
```

Note: You may need to wait a short while for the container to start.

Output.

```bash
{
  "name" : "e42a0e2ac752",
  "cluster_name" : "docker-cluster",
  "cluster_uuid" : "Q76hITRvRu65Gzxo_gnYdg",
  "version" : {
    "number" : "7.9.2",
    "build_flavor" : "default",
    "build_type" : "docker",
    "build_hash" : "d34da0ea4a966c4e49417f2da2f244e3e97b4e6e",
    "build_date" : "2020-09-23T00:45:33.626720Z",
    "build_snapshot" : false,
    "lucene_version" : "8.6.2",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}
```

## Step 3: Delete the Existing Index (if it exists)

Before creating a new index, delete any existing index to avoid conflicts.

```bash
nano semantic_app.py
```

Add the below code:

```bash
from elasticsearch import Elasticsearch

# Connect to Elasticsearch
es = Elasticsearch(hosts=["http://localhost:9200"])
index_name = "document"  # Replace with your index name

# Delete the index if it exists
if es.indices.exists(index=index_name):
    es.indices.delete(index=index_name)
    print(f"Index '{index_name}' deleted successfully.")
else:
    print(f"Index '{index_name}' does not exist.")
```

## Step 4: Initialize the ElasticsearchDocumentStore

The ElasticsearchDocumentStore is a Haystack component that interacts with Elasticsearch to store and retrieve documents. Initialize it with the appropriate embedding dimension to match the output of your embedding model.

```bash
from haystack.document_stores import ElasticsearchDocumentStore

document_store = ElasticsearchDocumentStore(
    host="localhost",  # Replace with your Elasticsearch host
    username="",       # Replace with your Elasticsearch username (if applicable)
    password="",       # Replace with your Elasticsearch password (if applicable)
    index="document",  # Name of the index where documents will be stored
    embedding_dim=384  # Set embedding_dim to match the model's output
)
```

## Step 5: Prepare and Index Sample Documents

Add sample documents to the document store.

```bash
your_documents = [
    {
        "content": "The quick brown fox jumps over the lazy dog.",
        "meta": {"title": "Example Document 1", "author": "John Doe"}
    },
    {
        "content": "Artificial intelligence is transforming the world.",
        "meta": {"title": "Example Document 2", "author": "Jane Smith"}
    },
    {
        "content": "Semantic search improves information retrieval.",
        "meta": {"title": "Example Document 3", "author": "Alice Johnson"}
    }
]

# Write documents to the document store
document_store.write_documents(your_documents)
print("Documents have been successfully indexed!")
```

## Step 6: Initialize the Retriever

The retriever is responsible for fetching relevant documents based on the query.

```bash
from haystack.nodes import EmbeddingRetriever

retriever = EmbeddingRetriever(
    document_store=document_store,  # Pass the document_store here
    embedding_model="sentence-transformers/all-MiniLM-L6-v2"
)
```

## Step 7: Generate Embeddings for Documents

Generate embeddings for the indexed documents.

```bash
document_store.update_embeddings(retriever)
print("Document embeddings have been generated!")
```

## Step 8: Initialize the Reader

The reader extracts answers from the retrieved documents.

```bash
from haystack.nodes import FARMReader

reader = FARMReader(
    model_name_or_path="deepset/roberta-base-squad2",
    use_gpu=True  # Enable GPU acceleration
)
```

## Step 9: Create the Pipeline

Combine the retriever and reader into a pipeline.

```bash
from haystack.pipelines import ExtractiveQAPipeline

pipeline = ExtractiveQAPipeline(reader, retriever)
```

## Step 10: Query the Pipeline

Test the pipeline by running a query.

```bash
query = "What is semantic search?"
results = pipeline.run(query=query)

# Print the results
print("Search Results:")
for doc in results["documents"]:
    print(f"Content: {doc.content}")
    print(f"Meta: {doc.meta}")
    print("---")
```

## Step 11: Test the Application

To ensure the application is working correctly, run multiple test queries and verify the results.

```bash
test_queries = [
    "Who wrote about the quick brown fox?",
    "How is AI transforming the world?",
    "What improves information retrieval?"
]

for query in test_queries:
    print(f"Query: {query}")
    results = pipeline.run(query=query)
    for doc in results["documents"]:
        print(f"Content: {doc.content}")
        print(f"Meta: {doc.meta}")
        print("---")
```

## Step 12: Run the Application

Run the Python script:

```bash
python3 semantic_app.py
```

Output:

```bash
Documents have been successfully indexed!
Batches: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  3.45it/s]
Updating embeddings: 10000 Docs [00:00, 15556.09 Docs/s]                                                                                                                
Document embeddings have been generated!
Batches: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 148.67it/s]
Inferencing Samples: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  7.95 Batches/s]
Search Results:
Content: Semantic search improves information retrieval.
Meta: {'author': 'Alice Johnson', 'title': 'Example Document 3'}
---
Content: Artificial intelligence is transforming the world.
Meta: {'author': 'Jane Smith', 'title': 'Example Document 2'}
---
Content: The quick brown fox jumps over the lazy dog.
Meta: {'author': 'John Doe', 'title': 'Example Document 1'}
---
Query: Who wrote about the quick brown fox?
Batches: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 218.53it/s]
Inferencing Samples: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 74.42 Batches/s]
Content: The quick brown fox jumps over the lazy dog.
Meta: {'author': 'John Doe', 'title': 'Example Document 1'}
---
Content: Semantic search improves information retrieval.
Meta: {'author': 'Alice Johnson', 'title': 'Example Document 3'}
---
Content: Artificial intelligence is transforming the world.
Meta: {'author': 'Jane Smith', 'title': 'Example Document 2'}
---
Query: How is AI transforming the world?
Batches: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 256.44it/s]
Inferencing Samples: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 78.72 Batches/s]
Content: Artificial intelligence is transforming the world.
Meta: {'author': 'Jane Smith', 'title': 'Example Document 2'}
---
Content: Semantic search improves information retrieval.
Meta: {'author': 'Alice Johnson', 'title': 'Example Document 3'}
---
Content: The quick brown fox jumps over the lazy dog.
Meta: {'author': 'John Doe', 'title': 'Example Document 1'}
---
Query: What improves information retrieval?
Batches: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 263.43it/s]
Inferencing Samples: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 70.20 Batches/s]
Content: Semantic search improves information retrieval.
Meta: {'author': 'Alice Johnson', 'title': 'Example Document 3'}
---
Content: Artificial intelligence is transforming the world.
Meta: {'author': 'Jane Smith', 'title': 'Example Document 2'}
---
Content: The quick brown fox jumps over the lazy dog.
Meta: {'author': 'John Doe', 'title': 'Example Document 1'}
```

## Conclusion

By following these steps, you can deploy a scalable semantic search application using Deepset.ai’s Haystack framework on an Ubuntu GPU server. This setup leverages the power of Elasticsearch, sentence-transformers, and GPU acceleration to deliver fast and accurate search results.