LLM performance

7 posts about LLM performance by 4 contributors.

Know LLM performance? Share what you know with developers who are learning it.

Write an article Ask a question

+ Follow LLM performanceBe the first to follow this

Articles: 7
Contributors: 4

Everything about LLM performance

Article · AI Agents
AI Coding Agent Cache Efficiency: Measuring Context Compaction and Model Switching
Measure AI coding agent cache efficiency by analyzing context compaction, model switching, cache reuse, latency, token usage, and performance across agent workloads.
Aarav PatelAug 205801,000
Article · LLMs
Designing Cost-Aware Multi-Model Routing for Enterprise AI APIs

Learn how to design cost-aware multi-model routing for enterprise AI APIs using model quality, latency, token costs, availability, fallback policies, and workload-specific routing strategies.

Aarav PatelAug 192541,000

Article · LLMs

Benchmarking Model Routing by Cost, Latency, and Answer Quality

Benchmark LLM model-routing strategies by comparing inference cost, response latency, answer quality, reliability, and workload-specific model selection across multiple models.

Riya PatelAug 163271,000

Article · AI Agents

Prompt Optimization Agent Capability: What We Can Build and Why It Matters

Build a prompt optimization capability for AI systems. Improve accuracy, efficiency, and governance by systematically evaluating and refining prompts.

John GodelApr 225591,000

Article · AI

AI LLM Reaches 1M TPS: The Next Leap in Inference Speed

Microsoft Azure’s ND GB300 v6 virtual machines powered by NVIDIA GB300 Blackwell GPUs have broken the one million tokens per second barrier for large language model inference. This article explains how this record was achieved and what it means for the future of AI infrastructure.

Mahesh ChandNov 4, 20252,3791,000

Article · Prompt Engineering

Prompt Engineering: Cost & Speed: Engineering for $/Accepted Output and Predictable Latency — Part 8

Master prompt engineering for cost and speed! Learn to design efficient prompts by setting budgets upfront, using sectioned generation with hard stops, caching effectively, and routing intelligently. Optimize for $/accepted output and predictable latency, turning them into controllable levers. Discover practical techniques for significant savings and improved performance in your LLM applications. This guide provides actionable strategies and metrics to ensure your AI scales affordably and reliably.

John GodelOct 16, 20259841,000

Article · LLMs

How to Evaluate LLMs?

Unlock the power of Large Language Models (LLMs) with our comprehensive guide! Learn how to evaluate LLMs for accuracy, safety, speed, cost-effectiveness, and more. Discover essential metrics and testing methods to ensure your chosen LLM is reliable, ethical, and perfectly aligned with your business goals. Make informed decisions and build trustworthy AI systems.

Riya PatelSep 15, 20251,1491,000