Skip to content
#

ai-training-data

Star

Here are 5 public repositories matching this topic...

NanoNets / llm-data-converter

Star 3

Convert any document format into LLM-ready data format (markdown) with advanced intelligent document processing capabilities powered by pre-trained models.

markdown document-conversion html-to-markdown text-extraction excel-to-markdown layout-detection structured-data-extraction pdf-to-markdown ppt-to-markdown document-to-markdown ai-training-data word-to-markdown docling-alternative llm-ready-data local-document-processing marker-alternative markitdown-alternative mineru-alternative offline-document-converter unstructured-alternative
  • Updated Jul 28, 2025
  • Python

shua-ie / QuarryCore

Star 2

[CURRENT WIP] Production-grade AI training data mining system. Hardware-adaptive architecture scales from Raspberry Pi to GPU workstations. Features advanced web scraping, multi-level deduplication, domain-specific extractors, and enterprise monitoring.

distributed-systems async-python web-scraping gpu-acceleration enterprise-software data-pipeline python-asyncio mlops production-grade machine-learning-pipeline ai-training-data hardware-adaptive
  • Updated Jul 21, 2025
  • Python

gokhaneraslan / llm-qa-dataset-pipeline

Sponsor
Star 0

🤖 Automated Q&A Dataset Generation Pipeline powered by LLMs. Multi-stage pipeline that searches, filters, extracts and transforms web content into high-quality question-answer datasets for LLM training. Supports multiple LLM providers (Groq, Mistral, Ollama) and search engines.

nlp machine-learning natural-language-processing web-scraping question-answering dataset-generation content-extraction mistral document-processing qa-dataset groq automated-pipeline llm llama-index trafilatura ollama semantic-chunking crawl4ai ai-training-data
  • Updated Jun 7, 2025
  • Python

vinsblack / The-Stach-Processed-v2

Sponsor
Star 0

Sample edition of The Stack Enriched: annotated, secure, and optimized code dataset, this is a sample version

machine-learning dataset programming-languages code-generation code-quality code-completion machine-learning-dataset bigcode ml-training huggingface-datasets commercial-license ai-code-generation llm-training-data premium-dataset ai-training-data commercial-dataset dataset-licensing rust-dataset
  • Updated Jul 19, 2025
  • Python

Improve this page

Add a description, image, and links to the ai-training-data topic page so that developers can more easily learn about it.

Curate this topic

Add this topic to your repo

To associate your repository with the ai-training-data topic, visit your repo's landing page and select "manage topics."

Learn more