KoboldCpp

KoboldCpp is an AI text-generation software for GGML and GGUF models, inspired by KoboldAI. It is a single self-contained distributable that builds off llama.cpp and adds additional features. It runs on CPU or GPU, supports image generation via Stable Diffusion 1.5, SDXL, SD3, Flux, speech-to-text via Whisper, and text-to-speech via OuteTTS. It includes the KoboldAI Lite UI with multiple modes and provides API endpoints for KoboldCppApi, OpenAiApi, OllamaApi, A1111ForgeApi, ComfyUiApi, WhisperTranscribeApi, XttsApi, and OpenAiSpeechApi. It works on Windows, MacOS, Linux, Android (Termux), Docker, Colab, and cloud GPU providers like RunPod and Novita AI.