ReconPipeline
I built ReconPipeline as an open-source Go-based reconnaissance automation tool for Linux. It brings together subdomain discovery, HTTP probing, crawling, historical URL collection and pattern-based security analysis into a unified workflow.
Problem & Purpose
During initial security assessments and penetration testing engagements, security engineers frequently run disjointed tools for subdomain discovery, port probing, crawler extraction, and URL normalization. Executing these steps manually is repetitive, time-consuming, and prone to incomplete coverage.
I designed ReconPipeline to solve this problem by providing a streamlined Go pipeline built specifically for Linux environments. It binds together critical discovery stages into a cohesive, deterministic execution flow.
What Was Built & Pipeline Stages
A modular CLI pipeline built for concurrency, repeatability, and ease of use:
- Automated asset discovery gathering subdomains across open-source feeds.
- Concurrent HTTP probing to quickly filter live and responsive hosts.
- Deep crawling combining web crawlers and historical archive URL endpoints.
- Pattern-based security analysis detecting exposed parameters and high-value endpoints.
- Clean, structured CLI outputs easily piped into downstream audit tools.
- Minimal system dependencies for quick installation on Linux cloud and auditing nodes.
Technical Implementation
Written in Go, utilizing Go's native concurrency primitives (goroutines and channels) for asynchronous network I/O and low memory usage during scanning: