AI Infrastructure
inference-platform
A platform for LLM inference serving: GPU scheduling, high-throughput serving with vLLM and KServe, and inference-aware autoscaling with KEDA on Kubernetes, with observability and cost/SLO discipline built in.
- Python
- Kubernetes
- KServe
- KEDA
- Envoy AI Gateway