On-Premise Installation Guide

​

Prerequisites

​

Access Requirements

• Server Access:
  • Confirm access to deployment servers with sudo privileges.
  • Verify that all necessary network routes are established via a secure VPN or dedicated network configuration.
• User Permissions:
  • Ensure that the deployment team has been granted the necessary access rights.
  • Validate that any required two-factor authentication (2FA) or security tokens are in place.

​

System Requirements

• GPU (Required when deploying an LLM on the same machine as the Odin AI Platform platform.)
• SMALL: ~500 (100 Concurrent) users can be supported by a single VM 16 cores, 64gb ram, 1 TB
  • SSD these machines cost ~$600 per month. This is a minimum size deployment.
• MEDIUM ~ 2000 (500 Concurrent)  users can be supported by a larger machine 32 cores, 128gb ram, 2 TB
  • SSD these machines are around ~$1,200 per month
• LARGE: ~10000 (1000+ Concurrent)  users should be deployed across containers so that the infrastructure
  • can scale to support the needs, this typically costs a minimum of $2,000 per month it is
  • suggested to calculate $2,000 per month per additional 1,000 Active Users. This is a fully

​

Deployment Tools & Software Dependencies

• Deployment Scripts:
  • Verified and version-controlled Bash deployment scripts.
• Dependency Management Tools:
  • Python 3
  • Git (current version)
  • Docker (v27.1.1) or Podman
• Additional Tools (Optional):
  • Logging and monitoring tools (e.g., ELK Stack, Prometheus, Grafana) as per the client’s monitoring requirements.

​

Credentials and Keys

• Server Credentials:
  • Ensure that all server access credentials are up to date and securely stored.
• Application API Keys and Licenses:
  • Verify that all application-specific API keys and licenses (e.g., OpenAI, SERP, Sentry) are available and valid.

​

Pre-Deployment Checklist

​

Hardware and System Verification

# Check CPU Information

lscpu                         # Works on both Ubuntu and RHEL

cat /proc/cpuinfo             # More detailed CPU information


# Check Memory Usage

free -h                       # Human-readable format for memory usage

cat /proc/meminfo             # Detailed memory information


# Check Storage Usage

df -h                         # Displays disk usage

lsblk                         # Lists block devices and partitions

​

3.2 Software and Dependency Verification

# Check Docker Version

docker --version              # Applicable for both Ubuntu and RHEL


# Check Docker Compose Version

docker compose version        # For modern Docker setups

docker-compose --version      # For older installations

```

# Check Python Version

python3 --version             # Preferred check for Python 3

python --version              # Alternative if python3 is not available


# Check Git Version

git --version                 # Valid for both Ubuntu and RHEL

​

3.3 Network and Connectivity Check

# Check Internet Connection Availability

ping -c 4 google.com          # Ping Google to verify network connectivity

curl -Is https://www.google.com | head -n 1  # Alternative check using curl

​

3.4 Additional Verifications

Notes: All internal machines must be able to communicate freely with each other. We recommend using SSL (HTTPS) connections; HTTP is supported during PoCs.

​

General Architecture

​

Architecture Diagram – On-Premise

​

Deployment architecture flow for AWS Cloud deployment