Llm Inference

Use when "LLM inference", "serving LLM", "vLLM", "llama.cpp", "GGUF", "text generation", "model s…

by eyadsibai

The skill

Use when "LLM inference", "serving LLM", "vLLM", "llama.cpp", "GGUF", "text generation", "model serving", "inference optimization", "KV cache", "continuous batching", "speculative decoding", "local LLM", "CPU inference"

This skill's licence does not allow redistribution, so the store does not show its text. Read it at the source.

Use this skill

Connect your agent to the Skill Store MCP server and ask for it by ref: eyadsibai/llm-inference-19277044961.

Markdown version

Details

Publisher: eyadsibai
Category: Data & Analytics › Machine learning & training
Updated: 2026-01-15

References

Related skills

Ai Llm Inference LLM serving for latency, batching, caching, quantization, and routing.
Llm Integration LLM integration patterns for function calling, streaming responses, local inference with Ollama,…
Serving Llms Vllm Serves LLMs with high throughput using vLLM's PagedAttention and continuous batching.
Llm Training Use when "training LLM", "finetuning", "RLHF", "distributed training", "DeepSpeed", "Accelerate",…
Inference Serving Topology LLM/model inference serving architecture: the engine → serving → orchestration layering (vLLM/SGL…