Home › Software › LLMKube

LLMKube

Kubernetes operator for LLM inference with pluggable runtimes (llama.cpp, vLLM, TGI, Ollama, vllm-swift), multi-GPU sharding, NVIDIA CUDA + Apple Silicon Metal support, and OpenAI-compatible API. ([Source Code](https://github.com/defilantech/LLMKube))

Official Website

Go/Docker/K8S

Language

Apache-2.0

License

Similar Tools in Software

Stirling-PDF

★ 95,530

Local hosted web application that allows you to perform various operations on PDF files, such as merging, splitting, file conversions and OCR

TypeScript NOASSERTION Software

code-server

★ 79,719

VS Code in the browser, hosted on a remote server

Nodejs/Docker MIT Software

Vaultwarden

★ 69,553

Lightweight Bitwarden server API implementation written in Rust

Rust/Docker GPL-3.0 Software

Huginn

★ 50,258

Build agents that monitor and act on your behalf

Ruby MIT Software

copyparty

★ 47,216

Portable file server with accelerated resumable uploads, deduplication, WebDAV, FTP, zeroconf, media indexer, video thumbnails, audio transcoding, and write-only folders, in a single file with no mandatory dependencies. ([Demo](https://a.ocv.me/pub/demo/))

Python MIT Software

TriliumNext Notes

★ 38,519

Cross-platform hierarchical note taking application with focus on building large personal knowledge bases (fork of Trilium Notes)

Nodejs/Docker/K8S AGPL-3.0 Software