SEO Technical: Crawlability
Guides crawlability improvements: robots, X-Robots-Tag, site structure, and internal linking.
When invoking: On first use, if helpful, open with 1–2 sentences on what this skill covers and why it matters, then provide the main output. On subsequent use or when the user asks to skip, go directly to the main output.
Scope (Technical SEO)
- Redirect chains & loops: Fix multi-hop redirects; point directly to final URL
- Broken links (4xx): Fix broken internal/external links; 301 or remove
- Site architecture: Logical hierarchy; pages within 3–4 clicks from homepage
- Orphan pages: Add internal links to pages with no incoming links
- Pagination: Prefer pagination over infinite scroll for crawlability
Initial Assessment
Check for product marketing context first: If .claude/product-marketing-context.md or .cursor/product-marketing-context.md exists, read it for site structure.
Identify:
- Site structure: Flat vs. deep hierarchy
- Framework: Next.js, static, SPA, etc.
- Key paths: Sitemap, robots.txt, API, static assets
Best Practices
Redirect Chains & Loops
- Fix multi-hop redirects; point directly to final URL
- Loops: URLs redirecting back to themselves; break the cycle
Broken Links (4xx)
- Fix broken internal/external links; 301 or remove
- Audit regularly; update or remove broken links
Site Architecture
| Principle | Guideline |
|---|---|
| Depth | Important pages within 3–4 clicks from homepage |
| Orphan pages | Add internal links to pages with no incoming links |
| Hierarchy | Logical structure; hub pages link to content |
Pagination
- Prefer pagination over infinite scroll for crawlability
- Reference links to next/previous pages; avoid dynamic-only loading
Common Issues
| Issue | Check |
|---|---|
| Redirect chains | Update links to point directly to final URL |
| Broken links | 301 or remove; audit internal and external |
| Orphan pages | Add internal links from hub or navigation |
| Infinite scroll | Replace with pagination for key content |
Output Format
- Redirect audit: Chains and loops to fix
- Broken link audit: 4xx links to fix
- Site structure: Orphan pages, hierarchy
- Pagination: Implementation for crawlable content
Related Skills
- seo-technical-robots: robots.txt configuration
- seo-technical-sitemap: URL discovery
- analytics-google-search-console: Index status, Coverage report
- seo-technical-indexing: Fix indexing issues
- seo-on-page-internal-links: Internal linking best practices