7 Data Sources
Covers GitHub, Hacker News, Reddit, YouTube, Product Hunt, X/Twitter, and 6 Chinese tech platforms in a single run.
Plugin OpenClaw Ecosystem
Your AI Never Falls Behind
AI models freeze at their training cutoff. Info Pipeline gives your agents a live intelligence feed β GitHub, Hacker News, Reddit, and 4 Chinese platforms, updated daily.
Overview
Info Pipeline aggregates 7 data sources across English and Chinese tech platforms every day. Each item passes through keyword filtering, relevance scoring, and deduplication before being serialized into a unified JSON schema β ready for your AI researcher agents to consume.
7
Data Sources
2
Languages (EN + ZH)
1 cmd
python main.py
1 schema
Unified Output
Live Feed
LIVE
Today's Intelligence Feed Data from Feb 25, 2026 Β· Updated weekly
Updated daily. Ingested by your agents before you've had your morning coffee.
Capabilities
Key Features
Keyword Filtering + Relevance Scoring
Each item is scored against a configurable keyword list. Only high-signal content passes β no noise.
Deduplication
Cross-source deduplication ensures the same story from multiple platforms appears only once in your report.
Unified Schema
All 7 collectors output the same JSON schema (title, url, source, score, summary, tags) for easy downstream processing.
Config-Driven
Everything in config.yaml β keywords, per-source limits, score thresholds, enabled platforms. Change behavior without touching code.
Markdown Reports for Agents
Pipeline outputs structured Markdown reports consumed directly by AI researcher agents for further analysis.
Architecture
How It Works
7 Sources β Collectors β Filter + Score β Dedup β Unified JSON β Markdown Report β Agent
1. Collect Each collector runs independently and fetches raw items from its platform using the parameters in config.yaml. 2. Filter & Score Items are matched against global keywords and scored for relevance. Low-signal content is discarded before it reaches storage. 3. Deduplicate URL-based and title-similarity deduplication removes duplicates across sources β the same story won't appear twice. 4. Unify Schema All surviving items are normalized into a single JSON structure: title, url, source, score, published_at, summary, tags. 5. Report A structured Markdown report is written to reports/ β directly consumable by AI researcher agents for further analysis. CLI Usage
# Run all 7 sources
python main.py
# Run a single source
python main.py --source github
python main.py --source hackernews
python main.py --source reddit
# List available sources
python main.py --list
Output Schema (per item)
{
"title": "...",
"url": "https://...",
"source": "github",
"score": 85,
"published_at": "2026-02-24T...",
"summary": "...",
"tags": ["llm", "open-source"]
}Data Sources
7 Platforms, 2 Languages
English mainstream tech + Chinese ecosystem β both covered in a single pipeline run.
GitHub Trending
ENHot repositories by topic (LLM, AI agent, RAG, MCP, diffusion) filtered by stars and recency.
Hacker News
ENTop stories filtered by score threshold β surface what the tech community is talking about today.
Multi-subreddit coverage: r/LocalLLaMA, r/MachineLearning, r/artificial, r/ChatGPT, r/ClaudeAI and more.
YouTube
ENLatest uploads from top AI channels β Karpathy, Yannic Kilcher, Two Minute Papers, 3Blue1Brown, Fireship.
Product Hunt
ENDaily new products in AI, Developer Tools, and Productivity β filtered by votes and topic.
X / Twitter
ENKeyword search for AI/LLM discussions from the English-language tech community.
Chinese Platforms
ZHη₯δΉ Β· 36ζ°ͺ Β· ζι Β· ε°ζ°ζ΄Ύ Β· InfoQ Β· Bη«η§ζεΊ β via trends-hub MCP integration.
Why It Matters
Why It Matters
Training Cutoffs Are Real
GPT-4, Claude, Gemini β all frozen at a date in the past. Ask about a library released last month and they'll hallucinate.
Daily Fresh Context
Info Pipeline runs on a cron schedule. Your agents wake up with a briefing that includes last night's top GitHub repos and this morning's HN frontpage.
Compound Intelligence
Every day your agents read the latest research, the more precise their answers become. The gap between your AI and others widens over time.
Coverage
What Gets Monitored
Every content type that matters for AI & tech research β all in one daily run.
GitHub Repos
Stars, forks, topics, recency
HN Discussions
Score, comments, domain
Reddit Posts
Multi-subreddit, upvote filtered
YouTube Videos
Selected AI creator channels
Product Launches
Daily PH feed, vote threshold
Tweets
Keyword search, recent timeline
Chinese Tech News
η₯δΉ / 36ζ°ͺ / ζι / Bη« / InfoQ / ε°ζ°ζ΄Ύ
Coming Soon
Join the Waitlist
Be the first to know when this plugin launches.