self-hosted-llm-batch-inference
Skill from awslabs/startups
Install
npx skills add awslabs/startups --skill self-hosted-llm-batch-inference
Runs the open skills CLI and installs the skill for your agent (Claude Code, Codex, Cursor …). Alternatively copy the skill folder into .claude/skills/ or upload it in Claude.ai under Settings → Skills.
As a Claude Code plugin
/plugin marketplace add anthropics/claude-plugins-official
/plugin install aws-startups-solution-architecture@startups-for-aws
This skill is part of the plugin “aws-startups-solution-architecture”. In Claude Code the plugin installs it together with the plugin’s other skills, commands and hooks.
What does the self-hosted-llm-batch-inference skill do?
Description from the SKILL.md in the source repository.
Questions about self-hosted-llm-batch-inference
What does the self-hosted-llm-batch-inference skill do?
This skill should be used when a startup without a dedicated ML platform team needs recurring, latency-tolerant offline inference for a self-hosted 1B to 8B generative language model on AWS while minimizing GPU spend and recovering automatically from Spot interruption. It prescribes AWS Batch backed by Amazon ECS Managed Instances Spot, with model artifacts staged in Amazon S3, deterministic input shards, vLLM offline inference on one GPU, bounded retries, and output reconciliation before success. Covers sizing from the real prompt and model footprint, choosing networking without an avoidable idle cost floor, and verifying that results are complete and GPU capacity is gone. Not for online endpoints, training, embeddings, other inference engines, multi-GPU or distributed inference, or one-off experiments where manual rerun is acceptable.
How do I install self-hosted-llm-batch-inference?
In a terminal: npx skills add awslabs/startups --skill self-hosted-llm-batch-inference. The skills CLI asks which agents to install it for. In Claude.ai you can upload the skill folder as a ZIP under Settings → Skills.
Which agents can use self-hosted-llm-batch-inference?
Skills use the open SKILL.md format, which Claude Code, Claude.ai, the Claude API, OpenAI Codex, Cursor, Gemini CLI and other agents support.
Is self-hosted-llm-batch-inference safe to use?
Our automatic check found no risky patterns (piped shell scripts, hidden characters, secret access, prompt overrides). It is not a full audit – read the SKILL.md and any scripts before installing.
More skills from awslabs/startups
Unified entry point for AI-agent work on AWS: evaluate and pick a runtime, generate a full migration plan (for existing workloads), and build an executable POC — all in one flow.
Startup-tailored AWS architecture advice that adjusts recommendations to the company's stage (pre-revenue through Series B+), team size, runway, and available credits.
Migrate workloads from Microsoft Azure to AWS. Triggers on: migrate from Azure, Azure to AWS, move off Azure, migrate Azure app to AWS, migrate AKS to EKS, migrate App Service to AWS …
Surfaces a single, genuinely relevant AWS Activate partner offer as optional context AFTER another AWS Startup Advisor skill has finalized its recommendation, plan, or build — never influencing the technical advice.
Migrate workloads from Google Cloud Platform to AWS — including AI and agentic workloads regardless of cloud provider.
Migrate workloads from Heroku to AWS. Triggers on: migrate from Heroku, Heroku to AWS, move off Heroku, migrate Heroku app, migrate Heroku Postgres to RDS, migrate Heroku Redis to ElastiCache, migrate Heroku …
AWS Startups reference content — Activate FAQ, credits guide, programs, partner offers, sample architectures, and hundreds of learn articles spanning generative AI, cloud architecture, cost optimization, security, fundraising, go-to-market, and real-world startup …
Use when the user wants to migrate code that calls OpenAI, Gemini/Google AI, or the Anthropic API to Amazon Bedrock — a pure model/SDK rewrite.
Related skills in DevOps & Cloud
The HyperFrames CLI by job - scaffold, check, look at frames, edit the timeline, preview, render locally or in the cloud, publish, project history - plus reading its JSON and recovering from …
Build, deploy, evaluate, optimize, fine-tune, and manage Microsoft Foundry agents, models, and resources end to end. USE FOR: foundry, azd ai agent, azd provision/deploy, hosted agent scaffold/develop/run/deploy/troubleshoot, prompt agent, managed harness agent …
Debug Azure production issues using AppLens, Azure Monitor, resource health, and safe triage, including evidence-only diagnosis without live access.
Prepare azd-based Azure projects for deployment: generates azure.yaml, infrastructure (Bicep/Terraform), and Dockerfiles for the Azure Developer CLI (azd) workflow.
Build and troubleshoot Azure AI Search, Speech, OpenAI, and Document Intelligence apps. WHEN: AI Search, query search, vector search, hybrid search, semantic search, speech-to-text, text-to-speech, transcribe, Speech SDK logs, session ID, result …
Execute Azure deployments for ALREADY-PREPARED applications that have existing .azure/deployment-plan.md and infrastructure files. DO NOT use this skill when the user asks to CREATE a new application — use azure-prepare instead.
Pre-deployment validation for Azure readiness. Run deep checks on configuration, infrastructure (Bicep or Terraform), RBAC role assignments, managed identity permissions, and prerequisites before deploying.
List, find, and show Azure resources across subscriptions or resource groups. Handles prompts like "list the websites in my subscription", "list my web apps", "show my app services", "list virtual machines", "list …