AI tools for “llm”

Set up GenAI tracing for an LLM or agent application in Domino by checking the Domino SDK and MLflow requirements (the DSE needs nothing installed), then adding a tracing_setup.py helper, span evaluators, a config.yaml,…

Migrate a GenAI/LLM app off Traceloop/OpenLLMetry onto OpenInference with its OTel gen_ai.* semconv option enabled (`enable_genai_semconv` / `OPENINFERENCE_ENABLE_GENAI_SEMCONV`). Covers detecting the existing Traceloop…

DeepEval evaluation workflow for AI agents and LLM applications. TRIGGER when the user wants to evaluate or improve an AI agent, tool-using workflow, multi-turn chatbot, RAG pipeline, or LLM app; add evals; generate…

…decision process for instrumenting a GenAI/LLM app (Python or TypeScript, any provider/agent SDK) so it emits OTel GenAI semantic-convention (gen_ai.*) spans queryable in Dynatrace. Four steps: detect,…

Free LLM VRAM calculator that runs in your browser. Pick a model, quant and context length to see how much GPU memory it needs and if your card has room.

Release dates of the newest Claude, GPT, Gemini, Grok, DeepSeek, Qwen, Kimi, Mistral and Meta models, plus free LLM API limits, deprecations and dev tools.

LangSmith Observability gives complete visibility into agent behavior with tracing, real-time monitoring, alerting, and high-level insights into usage.

Benchmark Jetson LLM/VLM serving performance across vLLM, llama.cpp, and Ollama with structured JSON output.

Checklist for code that calls a language model or exposes a system to one. Use when writing or reviewing code that calls a model API (Anthropic, OpenAI, Bedrock, Gemini, Perplexity), a prompt or prompt template,…

Jump to content Main menu Main menu move to sidebar hide Navigation Main pageCommunity portalProject chatCreate a new ItemRecent changesRandom ItemQuery ServiceNearbyHelpSpecial pages Lexicographical data Create a new…

Investigate AI observability clusters — understand usage patterns in AI/LLM traffic, compare cluster behavior, compute cost/latency metrics, and drill into individual traces within clusters.

…evaluations — `hog` (deterministic code-based), `llm_judge` (LLM-prompt-based), and `sentiment` (user-message sentiment). Find existing evaluations, inspect their configuration, run them against specific…

Investigate LLM spend in PostHog — total cost over time, cost by model, provider, user, trace, or custom dimension, token and cache-hit economics, and cost regressions. Use when the user asks "how much are we spending…

Add PostHog AI Observability to trace AI model usage. Use after implementing LLM features or reviewing PRs to ensure all generations are captured with token counts, latency, and costs. Also handles initial PostHog SDK…

Set up an LLM-judge evaluation that extracts canonical use cases for a PostHog feature at scale and streams the results to a Slack channel as a live feed. Use when someone wants to understand how users are actually…

Debug and inspect LLM/AI agent traces using PostHog's MCP tools. Use when the user pastes a trace or session URL (e.g. /ai-observability/traces/<id> or /ai-observability/sessions/<id>), asks to debug a trace, figure out…

One API and one credential for frontier and open-source LLMs and embedding models, built into your Neon branch and powered by Databricks. Use when a user wants to call an LLM, generate embeddings, add AI/chat/an agent…

Meet OneLLM Office for everyday work on Mac, and OneLLM for iPhone and iPad. Create reports, choose local or online AI, and take your work with you.

Test results for 160 AI image, video and chat models: which deliver NSFW requests, how they score in a blind quality test, and what each one costs to run.

A curated directory of DeepSeek Harness (DSH) plugins: find, compare, and install community plugins for DeepSeek’s open-source agent framework.