RAG with LangChain x Desearch

Welcome to the comprehensive guide on integrating Desearch with LangChain to enable advanced Retrieval-Augmented Generation (RAG) and agent-style workflows. Desearch is a cutting-edge, privacy-first search API that excels in delivering relevant, real-time web and Twitter content, specifically optimized for use in Large Language Model (LLM) applications.

🚀 What You’ll Learn

In this guide, you will learn:

Setting up Desearch Tools: How to configure and initialize Desearch tools within the LangChain framework.
Content Retrieval: Techniques to retrieve and utilize web and Twitter content for enhancing LLM capabilities.
RAG Pipelines and Agent Workflows: Building sophisticated RAG pipelines and agent workflows to automate and streamline information retrieval and processing.
Testing and Extending: Methods to test the integration and extend its capabilities to suit your specific needs.

📦 Installation

To get started with Desearch's LangChain integration, follow these steps:

Install the Integration: Use pip to install the necessary package.
```
pip install langchain-desearch
```
Configure API Keys: Ensure your Desearch API key is securely stored in a .env file. This key is essential for authenticating your requests to the Desearch API.
```
.env
DESEARCH_API_KEY=your_api_key_here
```

🛠️ Tools Overview

Desearch provides a suite of modular tools compatible with LangChain, each designed for specific use cases:

🔎 Search Tools

DesearchTool: A comprehensive tool for searching across web, AI, and Twitter posts.
BasicWebSearchTool: A lightweight tool focused solely on web searches.
BasicTwitterSearchTool: A specialized tool for searching tweets with advanced filtering options.

🐦 Twitter Tools

These tools are designed to interact with Twitter data:

FetchTweetsByUrlsTool: Retrieve tweets using their URLs.
FetchTweetsByIdTool: Fetch tweets by their unique IDs.
FetchLatestTweetsTool: Access the latest tweets from a specified user or topic.
FetchTweetsAndRepliesByUserTool: Gather tweets and their replies from a specific user.
FetchRepliesByPostTool: Obtain replies to a specific tweet.
FetchRetweetsByPostTool: Collect retweets of a particular tweet.
FetchTwitterUserTool: Retrieve detailed information about a Twitter user.

🧪 Quick Examples

Basic Usage

Here's a simple example to demonstrate how to use the DesearchTool for web searches:

from langchain_desearch.tools import DesearchTool

# Initialize the tool
tool = DesearchTool()

# Run a search query
response = tool._run(
    prompt="Bittensor network activity",
    tool=["web"],
    model="NOVA",
    date_filter="PAST_24_HOURS"
)

# Print the response
print(response)

Search Twitter

To search Twitter for specific topics, use the BasicTwitterSearchTool:

from langchain_desearch.tools import BasicTwitterSearchTool

# Initialize the Twitter search tool
tool = BasicTwitterSearchTool()

# Execute a search query
response = tool._run(query="AI governance", sort="Top", count=5)

# Display the results
print(response)

🤖 Building a RAG Chain

Creating a RAG chain involves integrating Desearch with LangChain's prompt and LLM capabilities:

from langchain_desearch.tools import DesearchTool
from langchain_core.prompts import PromptTemplate, ChatPromptTemplate
from langchain_deepseek import ChatDeepSeek
from langchain_core.runnables import RunnableParallel, RunnablePassthrough
from langchain_core.output_parsers import StrOutputParser

# Initialize the Desearch tool
tool = DesearchTool()

# Define a function to fetch context
def fetch_context(query):
    return tool._run(prompt=query, tool="desearch_web", model="NOVA")

# Create a prompt template
prompt = ChatPromptTemplate.from_messages([
    ("system", "You are a research assistant."),
    ("human", "Answer this using the provided context:\n\n<context>{context}</context>")
])

# Set up the LLM
llm = ChatDeepSeek(model="deepseek-chat", temperature=0.7)
parser = StrOutputParser()

# Build the RAG chain
chain = RunnableParallel({
    "query": RunnablePassthrough(),
    "context": lambda q: fetch_context(q["query"]),
}) | prompt | llm | parser

# Invoke the chain with a query
result = chain.invoke("What is the latest update on the EU AI Act?")
print(result)

🧠 Using Desearch in a LangChain Agent

Integrate Desearch into a LangChain agent to automate complex workflows:

from langchain_desearch.agent import create_search_agent
from langchain_deepseek import ChatDeepSeek

# Initialize the LLM
llm = ChatDeepSeek(model="deepseek-chat", temperature=0.7)

# Create a search agent
agent = create_search_agent(llm=llm)

# Invoke the agent with an input message
response = agent.invoke({"input_message": "Summarize current AI regulation trends"})
print(response["output"])

✅ Testing

Unit Tests (Mocked)

To ensure your integration works as expected, run unit tests:

pytest tests/test_tools.py

Live API Tests

For testing with live data, ensure your API key is set and run:

pytest tests/test_tools_real.py

⚠️ Note: The DESEARCH_API_KEY must be set in your .env file to execute live tests.

🤝 Contributing

We welcome contributions! To contribute:

Fork the Repository: Create your own fork of the project.
Create a Branch: Develop your feature or fix in a new branch.
```
git checkout -b feature/my-feature
```
Make Changes: Implement your changes and commit them.
Submit a Pull Request: Open a pull request to merge your changes.

📜 License

This project is licensed under the MIT License, allowing for open collaboration and sharing.

📬 Contact

For questions or feedback, feel free to reach out at [[email protected]] or open an issue on the repository.