Langchain news tools extract old news articles even specify a date range.

[jiagaoxiang]

I am using duckduckgo and yahoo finance news tools to extract news articles about something, for example, news articles about Google after 10/13/2024. But the returned articles are very old articles like published on 6/18/2024, but the date field shows very recent date like 10/14/2024. What's the problem here? How to fix? It seems these tools don't have access to the most recent results.

[dosubot[bot]]

Hey @jiagaoxiang! I'm here to help you out with any bugs, questions, or contributions you might have. Let's tackle this issue together!

To address the issue of retrieving old news articles despite specifying a date range, you can adjust the configuration settings in LangChain to improve the accuracy of date filtering. For instance, if you are using the AskNews integration, you can set the start_timestamp and end_timestamp parameters in the AskNewsRetriever to define the time range for filtering news articles. This can help ensure that the articles retrieved fall within the desired date range.

Here's an example of how you can configure these parameters:

from datetime import datetime, timedelta

start = (datetime.now() - timedelta(days=7)).timestamp()
end = datetime.now().timestamp()

retriever = AskNewsRetriever(
    k=3,
    categories=["Business", "Technology"],
    start_timestamp=int(start),  # defaults to 48 hours ago
    end_timestamp=int(end),  # defaults to now
    method="kw",  # defaults to "nl", natural language, can also be "kw" for keyword search
    offset=10,  # allows you to paginate results
)

retriever.invoke("Google")

Additionally, if you are using the Exa Search integration, you can use the start_published_date and end_published_date parameters in the search_and_contents tool to restrict the search to documents published within a specific date range:

@tool
def search_and_contents(
    query: str,
    start_published_date: str = None,
    end_published_date: str = None,
):
    return exa.search_and_contents(
        query,
        start_published_date=start_published_date,
        end_published_date=end_published_date,
        text=True,
        highlights=True,
    )

These configurations should help improve the relevance and accuracy of the retrieved articles ^[1][2].

To continue talking to Dosu, mention @dosu.

[jiagaoxiang]

@dosu what if I am using these tools: GoogleFinanceQueryRun(api_wrapper=GoogleFinanceAPIWrapper()),YahooFinanceNewsTool(), DuckDuckGoSearchRun(), ExecPython(), WikipediaQueryRun(api_wrapper=api_wrapper)

[dosubot[bot]]

For the tools you're using, here's how you can handle date filtering:

1. DuckDuckGoSearchRun: You can specify a date range using the time parameter with options like d (day), w (week), m (month), and y (year) to filter search results by recency ^[1].

2. GoogleFinanceQueryRun and YahooFinanceNewsTool: Unfortunately, I couldn't find specific information on date filtering for these tools in the LangChain repository. You might need to check the documentation or source code for these tools to see if they support date filtering.

3. ExecPython and WikipediaQueryRun: Similarly, there isn't specific information available on date-specific queries for these tools in the repository. You may need to explore their documentation or source code for more details.

If these tools don't natively support date filtering, you might need to implement additional logic to filter the results based on the publication date after retrieving them.

To continue talking to Dosu, mention @dosu.