Large Language Models (LLMs) have been making waves in the field of artificial intelligence (AI) for quite some time, and their popularity continues to soar. These advanced models have the remarkable ability to understand, generate, and respond to human language with unprecedented accuracy and depth. With this surge in interest comes the rise of open source solutions that enable individuals and organizations to host LLMs locally.

In this blog post we will explore how to turn your existing local computer/server into a simple ai server.

Kubernetes is an incredible tool for deploying, scaling, and managing containerized applications. One crucial aspect of kubernetes security is ensuring that communication between different entities is secure. By default, kubernetes management network is secure and pod network is handled by 3rd party plugin which mostly support encryption.

Today we will focus on properly securing outside-in web communication to our cluster with Cert-Manager and self-signed certificates. We assume that you have access to working kubernetes cluster with ingress controller.