🌐Product Architecture

There are five layers of Adot’s search engine architecture:

• Hybrid data fetcher layer. The goal of this layer is to flow all Web3 data into the engine regardless of where the data is stored. Adot invented a hybrid system combining different methods to efficiently fetch data from different sources. The Web3 crawler works best for data stored on centralized servers like Twitter or Medium. Users and developers can contribute to this layer by uploading data from their database, hosting a crawler node, or sending their personal user data.

• Structure parser layer. The fetched data is originally stored in many different protocols, e.g., HTML, text, Lens protocol, Twitter format, etc. This layer parses the structure of the original data, builds their connections, and transforms them into the index protocol. The unified data structure is easy to be used by developers and read by users. Developers can contribute by annotating the original structure of their data and generating parsed content based on the index protocol.

• AI modeling layer. The well-structured data contains valuable knowledge. This layer applies AI technologies to mine deep knowledge from data and generate extra signals. For example, an embedding of each image is created to make the image searchable. Developers can contribute by running their own AI models to generate labels. Users can manually label the content and users based on their knowledge.

• Structured index layer. Each parsed content, generated label, or signal is generated offline and cannot be directly used to serve results to users in real-time. An index bridges offline data and online computing. Adot invented a structured index for not only regular content but also content structures and knowledge graphs to improve search precision.

• Search algorithm layer. This layer implements sophisticated algorithms to process input search queries of users, search relevant data from the index, run some real-time computation, and return ranked results. Developers and users can contribute their own search algorithms or define rules to customize their search-engine experiences. These algorithms are plugged into the overall search infra and return customized results to developers and users.