The growing use of Large Language Models (LLMs) presents challenges in selecting the most suitable model for a given prompt. Each LLM has distinct strengths and limitations, making model selection a crucial factor in optimizing response quality, processing time and computational efficiency. Currently, prompts are often processed without prior classification, leading to one overall model choice that increases costs or reduces output quality.
This project introduces Logos (Leveraging Orchestrated Governance for Optimized Sustainability), a system for centralized LLM configuration, request routing, and efficient resource utilization.
It develops classification techniques, implements a scheduling mechanism and evaluates the achieved performance. A comparative analysis will assess improvements over direct prompt submission, aiming for higher response quality, reduced costs, and optimized resource utilization when running LLMs.