# How to Use LLM CLI to Deploy the Deepseek Model on an Ubuntu GPU Server

> URL: https://www.atlantic.net/gpu-server-hosting/how-to-use-llm-cli-to-deploy-the-deepseek-model-on-an-ubuntu-gpu-server/ | Published: 2025-04-15 | Author: Hitesh Jethva

Deploying large language models (LLMs) has traditionally been complex, requiring significant effort and specialized knowledge. However, tools like LLM CLI now streamline this process, making it accessible even to those new to deploying advanced AI models. Deepseek, a powerful language model known for its remarkable performance in understanding and generating human-like text, can now easily be deployed using LLM CLI on Ubuntu GPU servers.

In this tutorial, we will explain how to use LLM CLI to deploy the Deepseek model on your Ubuntu GPU server.

## Prerequisites

- An Ubuntu 24.04 server with an NVIDIA GPU.
- A non-root user with sudo privileges.
- NVIDIA drivers installed.

## Step 1: Setup a Python Environment

Ubuntu’s default repositories include the Python 3.12 version, which may not support the latest LLM optimizations. We’ll use the deadsnakes PPA to install Python 3.11.

1. Add the deadsnakes PPA for Python 3.11.

```bash
add-apt-repository ppa:deadsnakes/ppa -y
```

2. Update package lists.

```bash
apt update
```

3. Install Python 3.11 along with essential tools.

```bash
apt install python3.11 python3.11-venv python3.11-distutils python3.11-dev -y
```

4. Create a virtual environment named ‘deepseek-env’.

```bash
python3.11 -m venv deepseek-env
```

5. Activate the environment.

```bash
source deepseek-env/bin/activate
```

## Step 2: Install PyTorch with CUDA 12.1 for GPU Acceleration

PyTorch with CUDA support is crucial for GPU-accelerated deep learning. We’ll install the latest stable version compatible with NVIDIA CUDA 12.1.

1. Install PyTorch with CUDA 12.1 support.

```bash
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121
```

2. Install Hugging Face libraries for model loading and quantization.

```bash
pip install transformers accelerate bitsandbytes auto-gptq optimum
```

3. Install the LLM CLI and Llama.cpp integration.

```bash
pip install llm llm-llama-cpp llama-cpp-python
```

## Step 3: Download the Deepseek GGUF Model

GGUF is the latest format for efficient GPU inference. We’ll download the Q5_K_M quantized version (good balance between quality & speed).

1. Create a directory to store models.

```bash
mkdir -p ~/models
```

2. Navigate into the models directory.

```bash
cd ~/models
```

3. Download the Deepseek-7B GGUF model.

```bash
wget https://huggingface.co/TheBloke/deepseek-llm-7B-chat-GGUF/resolve/main/deepseek-llm-7b-chat.Q5_K_M.gguf
```

4. Register the model with an alias ‘deepseek-chat’.

```bash
llm llama-cpp add-model ~/models/deepseek-llm-7b-chat.Q5_K_M.gguf --alias deepseek-chat
```

## Step 4: Run Inference with GPU Offloading

To maximize GPU utilization, we set LLAMA_CPP_GPU_LAYERS=35 (adjust based on VRAM).

Run the llm with your query.

```bash
LLAMA_CPP_GPU_LAYERS=35 llm -m deepseek-chat "What is Atlantic.Net Cloud GPU?"
```

Output.

```bash
llama_init_from_model: n_ctx_per_seq (4000) < n_ctx_train (4096) -- the full capacity of the model will not be utilized
Atlantic.Net Cloud GPU is a service that provides high-performance computing resources using Graphics Processing Units (GPUs) for running compute-intensive applications and workloads. GPU-based cloud computing offers significant performance gains in comparison to CPU-based systems, particularly for tasks that require a lot of graphical processing and data-intensive applications like machine learning, deep learning, data analysis, and scientific simulations.
```

## Conclusion

You’ve successfully deployed the Deepseek-7B-Chat model on an Ubuntu [GPU server](https://www.atlantic.net/gpu-server-hosting/) using the llm CLI. This setup allows for efficient, GPU-accelerated inference, making it ideal for AI-powered applications.