kserve agent
FIPS 140-3Machine learning & AIKServe Agent is a component that manages model serving workloads in KServe.
KServe Agent is a component that manages model serving workloads in KServe.
KServe Agent is a lightweight sidecar proxy component that enhances machine learning model serving within Kubernetes environments. It runs alongside your model server container and provides advanced serving capabilities including request batching, model pulling, and observability features.
The agent operates as a transparent proxy that receives client requests, processes them through configurable middleware for batching and logging, and forwards them to your model server. This architecture enables you to add production-ready serving features to any model server without requiring code changes.
For more information about KServe, visit the upstream documentation at https://kserve.github.io/website/.