Can it extract data from private repositories?

Yes, provided the authenticated user or provided token has the necessary permissions to access those repositories.

How does it handle large README files?

It uses an intelligent chunking strategy that breaks documents down by Markdown headings (H1-H3) to preserve context for LLM ingestion.

Does this skill require a GitHub token?

Yes, while some public data can be accessed without one, an authenticated GITHUB_TOKEN or GitHub CLI login is recommended to avoid rate limits.

What languages does it support for code mining?

It includes built-in detection and extraction for Python, JavaScript, TypeScript, Go, Rust, Java, Ruby, and Bash.

Can I harvest issue comments as well?

Yes, the harvester recursively fetches issue bodies along with their associated comments and labels for complete discussion context.

GitHub Harvester

Name: GitHub Harvester
Author: mindmorass

bymindmorass

Web Scraping & Data Collection

Extracts and processes comprehensive data from GitHub repositories for ingestion into RAG pipelines and LLM knowledge bases.

About

GitHub Harvester is a specialized utility designed to bridge the gap between GitHub's vast repositories and AI-powered development. It provides structured methods for mining documentation, code examples, issues, and discussions, transforming raw repo data into digestible formats for Retrieval-Augmented Generation (RAG). By automating the extraction of READMEs, changelogs, and language-specific code blocks, it enables developers to quickly build context-aware AI tools and documentation assistants directly from source control.

Key Features

0 GitHub stars
Multi-method extraction using GitHub CLI and PyGithub API
Automated documentation mining from READMEs and .github folders
Metadata extraction including stars, topics, and release history
Intelligent chunking strategies for READMEs, code, and issues
Recursive code example harvesting with language detection

Use Cases

Collecting training data for code-specific LLM context windows
Automating documentation audits and changelog aggregations
Building RAG-based AI assistants for specific open-source libraries

About

Key Features

0 GitHub stars
Multi-method extraction using GitHub CLI and PyGithub API
Automated documentation mining from READMEs and .github folders
Metadata extraction including stars, topics, and release history
Intelligent chunking strategies for READMEs, code, and issues
Recursive code example harvesting with language detection

Use Cases

Collecting training data for code-specific LLM context windows
Automating documentation audits and changelog aggregations
Building RAG-based AI assistants for specific open-source libraries