Deploy AI models and apps to Kubernetes without hitting every pitfall.

Deploy the operator and chat with Hermes in 2 or 3 steps. Note that you need a NVIDIA GPU with 8GB of RAM.

1. Install the NVIDIA Operator

2. Install the Prem Operator

   $ helm install latest oci://registry-1.docker.io/premai/prem-operator-chart
   

3. Deploy the Hermes + Big-AGI example and forward the ports

   $ kubectl apply -f examples/big-agi.yaml
   $ kubectl port-forward services/big-agi-service 3000:3000
   

If you browse to localhost:3000 you'll be able to begin chatting as soon as LocalAI downloads the model.

Feel free to create an issue here or reach out to us on Prem's Discord on the "#operator" channel.

💖💖 We'd love to hear from you! 💖💖

• Quick Starts
  • 🦙+🖥️Llama 3 + TUI + CLI
  • 🌿+φElia TUI + Phi-2
• Guides
  • 🦜️🔗Langchain
  • 🧩Ingress
  • 🌐Managed Clusters (GCP, AWS)
  • 📜Getting started
  • 📦Deployment
  • 👩‍💻Developer
  • 🧪vLLM
• Topics
  • 🔍Frequently Asked Questions
  • 🤝Contributing
  • 🎗️Issues
  • 🔧Troubleshooting
• Blogs
  • 💡Introducing Prem-Operator

Prem-Operator is under Apache 2.0 license. See the LICENSE file for details.