Skip to content
mcp/skillhub

Agent Skill

Pytorch Fsdp2 skill

pytorch-fsdp2Orchestra-Research/AI-Research-SKILLs08-distributed-training/pytorch-fsdp2/SKILL.md

Adds PyTorch FSDP2 (fully_shard) to training scripts with correct init, sharding, mixed precision/offload config, and distributed checkpointing. Use when models exceed single-GPU memory or when you need DTensor-based sharding with DeviceMesh.

14kMIT

always on

61 tokname + description

once used

2,544 tokSKILL.md body

bundled files

1213 KB

repo stars

14kGitHub

Install the Pytorch Fsdp2 skill

Skills CLI (Claude Code, Codex, Cursor and others)

npx skills add https://github.com/Orchestra-Research/AI-Research-SKILLs --skill pytorch-fsdp2

Claude Code, by hand (all projects)

git clone --depth 1 https://github.com/Orchestra-Research/AI-Research-SKILLs.git /tmp/AI-Research-SKILLs
mkdir -p ~/.claude/skills
cp -R /tmp/AI-Research-SKILLs/08-distributed-training/pytorch-fsdp2 ~/.claude/skills/pytorch-fsdp2

For one project only, copy the folder into .claude/skills/ in that project instead. The agent picks it up when a task matches its description.

What it costs your context

Agent Skills load in three steps. Tokens counted with the o200k tokenizer; your model's count will differ a little.

WhenWhat loadsCost
Every sessionName and description61 tok
When the agent uses itThe SKILL.md instructions2,544 tok
Only if opened12 bundled files (13 KB)varies

Until it is used, pytorch-fsdp2 costs about 22× less than the median MCP server we measured (1,362 tokens on every turn). Installing all 98 skills from Orchestra-Research/AI-Research-SKILLs would load 6,174 tokens of descriptions. Skills vs MCP study

Checks

Our checks found nothing to flag: it follows the Agent Skills specification's naming and size guidelines and doesn't pipe downloads into a shell.

Checked 2026-10-22 · method

What's inside

The headings of its SKILL.md. Read the full file on GitHub.

  • Skill: Use PyTorch FSDP2 (fullyshard) correctly in a training script
  • When to use this skill
  • Alternatives (when FSDP2 is not the best fit)
  • Contract the agent must follow
  • Step-by-step procedure
  • 0) Version & environment sanity
  • 1) Initialize distributed and set device
  • 2) Build model on meta device (recommended for very large models)
  • 3) Apply fullyshard() bottom-up (wrapping policy = “apply where needed”)
  • 4) Configure reshardafterforward for memory/perf trade-offs
  • 5) Mixed precision & offload (optional but common)
  • 6) Optimizer, gradient clipping, accumulation
  • 7) Checkpointing: prefer DCP or distributed state dict helpers
  • Workflow checklists (copy-paste friendly)
  • Workflow A: Retrofit FSDP2 into an existing training script
  • Workflow B: Add DCP save/load (minimal pattern)
  • Debug checklist (what the agent should check first)
  • Common issues and fixes
  • Minimal reference implementation outline (agent-friendly)
  • References

More skills from Orchestra-Research/AI-Research-SKILLs

All skills in this repo
  1. academic-plottingOrchestra-Research/AI-Research-SKILLs

    Generates publication-quality figures for ML papers from research context. Given a paper section or description, extracts system components and relationships to generate architecture diagrams via Gemini. Given experiment results or data, auto-selects chart type and generates data-driven figures via matplotlib/seaborn. Use when creating any figure for a conference paper.

    14k67 tok idle · 5,308 in use
  2. ara-compilerOrchestra-Research/AI-Research-SKILLs

    Compiles any research input — PDF papers, GitHub repositories, experiment logs, code directories, or raw notes — into a complete Agent-Native Research Artifact (ARA) with cognitive layer (claims, concepts, heuristics), physical layer (configs, code stubs), exploration graph, and grounded evidence. Use when ingesting a paper or codebase into a structured, machine-executable knowledge package, building an ARA from scratch, or converting research outputs into a falsifiable, agent-traversable form.

    14k106 tok idle · 3,117 in use
  3. ara-research-managerOrchestra-Research/AI-Research-SKILLs

    Records research provenance as a post-task epilogue, scanning conversation history at the end of a coding or research session to extract decisions, experiments, dead ends, claims, heuristics, and pivots, and writing them into the ara/ directory with user-vs-AI provenance tags. Use as a session epilogue — never during execution — to maintain a faithful, auditable trace of how a research project actually evolved.

    14k91 tok idle · 2,841 in use
  4. ara-rigor-reviewerOrchestra-Research/AI-Research-SKILLs

    Performs ARA Seal Level 2 semantic epistemic review on Agent-Native Research Artifacts, scoring six dimensions (evidence relevance, falsifiability, scope calibration, argument coherence, exploration integrity, methodological rigor) and producing a constructive, severity-ranked report with a Strong Accept-to-Reject recommendation. Use after Level 1 structural validation passes, when an ARA needs an objective epistemic critique before publication or release.

    14k90 tok idle · 4,013 in use
  5. audiocraft-audio-generationOrchestra-Research/AI-Research-SKILLs

    PyTorch library for audio generation including text-to-music (MusicGen) and text-to-sound (AudioGen). Use when you need to generate music from text descriptions, create sound effects, or perform melody-conditioned music generation.

    14k54 tok idle · 3,633 in use
  6. autogpt-agentsOrchestra-Research/AI-Research-SKILLs

    Autonomous AI agent platform for building and deploying continuous agents. Use when creating visual workflow agents, deploying persistent autonomous agents, or building complex multi-step AI automation systems.

    14k38 tok idle · 2,137 in use
  7. autoresearchOrchestra-Research/AI-Research-SKILLs

    Orchestrates end-to-end autonomous AI research projects using a two-loop architecture. The inner loop runs rapid experiment iterations with clear optimization targets. The outer loop synthesizes results, identifies patterns, and steers research direction. Routes to domain-specific skills for execution, supports continuous agent operation via Claude Code /loop and OpenClaw heartbeat, and produces research presentations and papers. Use when starting a research project, running autonomous experiments, or managing a multi-hypothesis research effort.

    14k98 tok idle · 5,151 in use
  8. awq-quantizationOrchestra-Research/AI-Research-SKILLs

    Activation-aware weight quantization for 4-bit LLM compression with 3x speedup and minimal accuracy loss. Use when deploying large models (7B-70B) on limited GPU memory, when you need faster inference than GPTQ with better accuracy preservation, or for instruction-tuned and multimodal models. MLSys 2024 Best Paper Award winner.

    14k79 tok idle · 2,318 in use