🔍Rendered Output

🏗️ Multi-Agent Crawler: High-Level Architecture (HLD)

1. System Overview

The Multi-Agent Harvester (MAS Crawler) is an asynchronous, AI-native pipeline designed to ingest, filter, structure, and deduplicate practice problems from open-source repositories and web sources.


2. Component Architecture


3. Agent Roles & Specifications

AgentResponsibilityModel ProfileTarget Latency
🕵️‍♂️ ScoutBinary relevance filterUltra-fast (Haiku / Flash-8B)< 200ms
⛏️ ExtractorSemantic schema & SQL parsingHigh-context (Flash 2.5)~ 1.2s
⚖️ CuratorQuality assurance & deduplicationHigh-reasoning (Sonnet 3.5)~ 2.5s
💡Tip

Paste your own markdown with ```mermaid diagram blocks into the Source editor on the left to see instant live rendering.