KServe configuration and inference requests

KServe configuration and inference requests A workflow diagram generated by Archify. 01 / Operator / client 02 / InferenceService 03 / Optional components 04 / Knative 05 / Standard Apply definition · operator / API · Operator / client Apply definition operator / API Inference request · authenticated endpoint route · Operator / client Inference request authenticated endpoint route Response to client · inference result · Operator / client Response to client inference result InferenceService · custom resource object · InferenceService InferenceService custom resource object Predictor spec · required · model server · InferenceService Predictor spec required · model server Deployment mode · Knative / Standard · InferenceService Deployment mode Knative / Standard Ready model server · model load / inference · InferenceService Ready model server model load / inference Transformer · optional · pre/post-processing · Optional components Transformer optional · pre/post-processing Explainer · optional · model explanations · Optional components Explainer optional · model explanations Knative Service · zero requires configuration · Knative Knative Service zero requires configuration Deployment / scaler · HPA / KEDA / external · Standard Deployment / scaler HPA / KEDA / external HTTP / gRPC Knative Standard response optional optional Legend KServe component Deployment-mode decision Runtime unit (pods) Client

Control and request paths

  • • Controllers process InferenceService specs; model servers run inference.
  • • Transformer/explainer are optional and require protocol compatibility.

Knative

  • • minReplicas defaults to 1; zero needs additional conditions.
  • • canaryTrafficPercent configures Knative revision traffic.

Standard

  • • Default HPA minimum 1; KEDA zero needs suitable activation signals
  • • Startup/model-loading latency and EC2 charges can remain.