Proprietary software by InferApps Tech PVT LTD. The source code is private. This page describes the project and what we're building.
A small kit for running open AI models locally: one PowerShell script plus one profile per model. Any project on the machine can use the models without copying model files around, and nothing goes to the cloud.
-
Simple commands:
list,start,stop,status,test,devices. - One server per model profile, built on llama.cpp.
- OpenAI-compatible API, including replies forced to match a JSON schema.
- Locked down by default: servers listen on this machine only, and every request needs a per-user key, so a random web page in your browser can't reach your models.
- Chat, coding and embedding profiles, including a multimodal embedder for text, images, audio and video.
- Notes on sizing models to your GPU memory and on converting models to GGUF.
In build.
Questions? Open an issue in this repository. Part of Shanmuga Raja Portfolio.