Desearch
A Decentralized Search Engine for Real Time Information
Abstract
Modern software systems increasingly rely on fresh and reliable information from the open internet. Applications, autonomous agents, and large language models require continuous access to current data in order to operate correctly. However, most existing search systems depend on centralized infrastructure, static indexing, or opaque ranking mechanisms that are poorly suited for real time usage.
Desearch introduces a Decentralized search engine built on Bittensor Subnet 22. The system separates search execution from evaluation through a validator miner architecture. Miners perform real time search and data extraction, while validators generate queries, evaluate responses, and distribute incentives based on relevance, structure, freshness, and performance.
Desearch is implemented as an open source, machine first search layer designed to be consumed programmatically by applications, agents, and LLM based systems.
1. Introduction
Search has become a foundational dependency for modern software. Applications rely on live data for monitoring and discovery. Agents depend on real time information to make autonomous decisions. LLM based systems require up to date sources to ground generation in reality.
Despite this dependency, most search infrastructure remains centralized and index driven. These systems struggle with freshness, impose trust assumptions, and limit transparency. Ranking logic is typically closed, making it difficult to audit correctness or bias.
These limitations become critical when search results are consumed directly by automated systems. Incorrect, outdated, or poorly structured data can propagate downstream and cause system level failures.
Desearch addresses this problem by treating search as a decentralized and continuously evaluated process rather than a static service.
2. Design Goals
Desearch is built around the following principles.
- Real time search without reliance on long lived indexes
- Separation of execution and evaluation
- Continuous quality measurement through incentives
- Open source implementation and transparent scoring
- Machine first interfaces suitable for applications, agents, and LLMs
The system prioritizes correctness, structure, and timeliness over static ranking or keyword optimization.
3. System Overview
Desearch operates as a network of independent participants coordinated through Bittensor Subnet 22.
The system consists of three primary components.
- Desearch API
- Validators
- Miners
External systems submit search requests through the Desearch API. These requests may target web content, social platforms, or structured knowledge sources.
Validators manage query generation, miner evaluation, scoring, and weight updates. Miners execute real time search and return structured responses. No single participant controls indexing, ranking, or data collection.
4. Miner Role
Miners are responsible for executing search and data extraction tasks in real time.
Typical miner responsibilities include.
- Querying open web sources
- Searching social platforms
- Scraping documents and posts
- Producing summaries grounded in source content
Miner performance is evaluated across multiple dimensions.
- Source relevance
- Data freshness
- Response structure
- Execution latency
Miners that consistently deliver high quality results receive higher weight and increased rewards.
5. Validator Role
Validators act as evaluators rather than data providers.
Their responsibilities include.
- Generating synthetic queries
- Collecting organic user queries
- Scoring miner responses
- Updating miner weights
- Applying penalties
Validators evaluate correctness, structure, and consistency. LLMs may be used as evaluation tools, not as authoritative sources. Validators continuously adjust weights based on observed miner behavior rather than fixed rules.
6. Scoring and Incentives
Desearch uses a performance based incentive mechanism.
Rewards are assigned based on.
- Correct source selection
- Freshness of returned data
- Structural correctness of responses
- Summary alignment with original content
- Execution speed
Penalties are applied for.
- Irrelevant or misleading sources
- Hallucinated summaries
- Streaming failures
- Timeouts or delayed responses
This mechanism ensures that miners compete on measurable quality rather than volume or reputation.
7. System Model
The Desearch network consists of a set of validators V and a set of miners M.
Validators generate search queries and evaluate miner outputs. Miners execute search tasks and return structured results. At each iteration, validators assign scores and update miner weights based on observed performance.
Over time, the network converges toward miners that provide accurate, timely, and well structured data for specific query types.
8. Threat Model and Reliability
Desearch is designed to operate in adversarial and unreliable environments.
Potential failure cases include.
- Low quality or spam responses
- Stale or outdated data
- Slow or unreliable miners
- Incorrect summaries
These behaviors are mitigated through continuous scoring, penalties, and weight reduction. Miners that fail to meet performance requirements lose influence over time.
This feedback loop increases reliability for downstream applications and automated systems.
9. Open Source Implementation
Desearch is developed as an open source subnet on the Bittensor network.
The full implementation, including validator logic, scoring mechanisms, and miner interfaces, is publicly available.
https://github.com/Desearch-ai/subnet-22-desearch
Open source development enables auditability, community contribution, and reduced trust assumptions for integrators.
10. Applications and Usage
Desearch is designed as a general purpose search layer.
Example use cases include.
- Real time retrieval augmented generation
- Market and narrative monitoring
- Research automation
- Social signal analysis
- Knowledge discovery for LLM systems
- Backend search for applications
The system provides structured outputs suitable for direct machine consumption.
11. Conclusion
Desearch introduces a Decentralized search engine optimized for real time information access. By separating execution from evaluation and aligning incentives with measurable quality, the system provides reliable and transparent search infrastructure for applications, agents, and LLM based systems.
As automated systems increasingly depend on live data, decentralized and open search infrastructure becomes a foundational requirement. Desearch provides this foundation through Bittensor Subnet 22.
References
Bittensor Whitepaper