Logos serves locally operated large language models (LLMs) through vLLM, a library for LLM inference and serving. For each local provider-model pair, Logos searches for a runnable configuration. The procedure can fail or require administrator overrides, and it does not compare the efficiency of runnable configurations. Administrators therefore still rely on trial-and-error to select a configuration.
This thesis develops an automated workflow for testing and ranking vLLM configurations. The benchmark varies parameters for parallelization, batching, prefix caching, context length, and graphics processing unit (GPU) memory utilization, then measures latency, throughput, and energy consumption. Logos tests each configuration with the same workload from the Grade School Math 8K (GSM8K) dataset. The resulting ranking identifies the configuration that best matches the selected metric weights. The Logos provider management interface lets administrators and application developers inspect trade-offs and apply the recommendation, while normal users automatically use the applied configuration.