## [v9.0]() (2023-03-11)

**Added**
- added use of the typing module. All parameters in the method calls use typing support to make it easier to understand what type is expected.
- added autosuggest methods `suggestEventTypes`, `suggestIndustries`, `getSdgUris`, `getSasbUris` - all to be used only when querying mentions
-

**Updated**
- `QueryArticles` class. Added filters `authorsFilter`, `videosFilter`, `linksFilter`
- `QueryMentions` class. Added several filters: `industryUri`, `sdgUri`, `sasbUri`, `esgUri`, `minSentenceIndex`, `maxSentenceIndex`, `showDuplicates`
- updated several code example files

Author: gregorleban

CHANGELOG.md

@@ -1,5 +1,18 @@
 # Change Log
 
+## [v9.0]() (2023-05-15)
+
+**Added**
+- added use of the typing module. All parameters in the method calls use typing support to make it easier to understand what type is expected.
+- added autosuggest methods `suggestEventTypes`, `suggestIndustries`, `getSdgUris`, `getSasbUris` - all to be used only when querying mentions
+- 
+
+**Updated**
+- `QueryArticles` class. Added filters `authorsFilter`, `videosFilter`, `linksFilter`
+- `QueryMentions` class. Added several filters: `industryUri`, `sdgUri`, `sasbUri`, `esgUri`, `minSentenceIndex`, `maxSentenceIndex`, `showDuplicates`
+- updated several code example files
+
+
 ## [v8.12]() (2022-03-11)
 
 **Updated**

README.md

@@ -1,18 +1,12 @@
-## Accessing Event Registry's News API through Python
-
-This library contains classes and methods that allow one to obtain from Event Registry (http://eventregistry.org) all available data, such as news articles, events, trends, etc.
-
-The detailed documentation on how to use the library is available at the [project's wiki page](https://github.com/EventRegistry/event-registry-python/wiki). Examples of use are in the [Examples folder in the repository](https://github.com/EventRegistry/event-registry-python/tree/master/eventregistry/examples).
-
-Changes introduced in the different versions of the module are described in the [CHANGELOG.md](https://github.com/EventRegistry/event-registry-python/blob/master/CHANGELOG.md) as well as on the [Releases](https://github.com/EventRegistry/event-registry-python/releases) page.
+Event Registry is a Python package that can be used to easily access the news data available in [Event Registry](http://eventregistry.org/) through the API. The package can be used to query for articles or events by filtering using a large set of filters, like keywords, concepts, topics, sources, sentiment, date, etc. Details about the News API are available on the [landing page of the product](https://newsapi.ai/).
 
 ## Installation
 
 Event Registry package can be installed using Python's pip installer. In the command line, simply type:
 
     pip install eventregistry
 
-and the package should be installed. Alternatively, you can also clone the package from the GitHub repository at https://github.com/EventRegistry/event-registry-python. After cloning it, open the command line and run:
+and the package should be installed. Alternatively, you can also clone the package from the [GitHub repository](https://github.com/EventRegistry/event-registry-python). After cloning it, open the command line and run:
 
     python setup.py install
 
@@ -24,7 +18,7 @@ To ensure the package has been properly installed run python and type:
 import eventregistry
 ```
 
-If you don't get any error messages then your installation has been successful.
+If you don't get any error messages, then your installation has been successful.
 
 ### Updating the package
 
@@ -34,51 +28,112 @@ As features are added to the package you will need at some point to update it. I
 
 ### Authentication and API key
 
-When making queries to Event Registry you will have to use an API key that you can obtain for free. The details how to obtain and use the key are described in the [Authorization](../../wiki/EventRegistry-class#authorization) section.
+When making queries to Event Registry you will have to use an API key that you can obtain for free. The details on how to obtain and use the key are described in the [Authorization](../../wiki/EventRegistry-class#authorization) section.
 
-## Three simple examples to make you interested
+## Four simple examples to get you interested
 
-**Find news articles that mention Tesla in the article title**
+**Print a list of recently articles or blog posts from *US based sources* *with positive sentiment* mentioning phrases *"George Clooney"* or *"Sandra Bullock"***
 
 ```python
 from eventregistry import *
 er = EventRegistry(apiKey = YOUR_API_KEY)
-# print at most 500 articles
-MAX_ITEMS = 500
-q = QueryArticlesIter(keywords = "tesla", keywordsLoc="title")
-for art in q.execQuery(er, sortBy = "date", maxItems = MAX_ITEMS):
+
+# get the USA URI
+usUri = er.getLocationUri("USA")    # = http://en.wikipedia.org/wiki/United_States
+
+q = QueryArticlesIter(
+    keywords = QueryItems.OR(["George Clooney", "Sandra Bullock"]),
+    minSentiment = 0.4,
+    sourceLocationUri = usUri,
+    dataType = ["news", "blog"])
+
+# obtain at most 500 newest articles or blog posts, remove maxItems to get all
+for art in q.execQuery(er, sortBy = "date", maxItems = 500):
     print(art)
 ```
 
-**Print a list of recently added articles mentioning George Clooney**
+**Print a list of most relevant *business* articles from the last month related to *Microsoft* or *Google*. The articles should be in any language (including Chinese, Arabic, ...)**
 
 ```python
 from eventregistry import *
-er = EventRegistry(apiKey = YOUR_API_KEY)
-q = QueryArticlesIter(conceptUri = er.getConceptUri("George Clooney"))
-for art in q.execQuery(er, sortBy = "date"):
-    print art
+# allowUseOfArchive=False will allow us to search only over the last month of data
+er = EventRegistry(apiKey = YOUR_API_KEY, allowUseOfArchive=False)
+
+# get the URIs for the companies and the category
+microsoftUri = er.getConceptUri("Microsoft")    # = http://en.wikipedia.org/wiki/Microsoft
+googleUri = er.getConceptUri("Google")          # = http://en.wikipedia.org/wiki/Google
+businessUri = er.getCategoryUri("news business")    # = news/Business
+
+q = QueryArticlesIter(
+    conceptUri = QueryItems.OR([microsoftUri, googleUri]),
+    categoryUri = businessUri)
+
+# obtain at most 500 newest articles, remove maxItems to get all
+for art in q.execQuery(er, sortBy = "date", maxItems = 500):
+    print(art)
 ```
 
+
 **Search for latest events related to Star Wars**
 
 ```python
 from eventregistry import *
 er = EventRegistry(apiKey = YOUR_API_KEY)
-q = QueryEvents(conceptUri = er.getConceptUri("Star Wars"))
-q.setRequestedResult(RequestEventsInfo(sortBy = "date", count=10))   # return event details for last 10 events
-print er.execQuery(q)
+
+q = QueryEvents(keywords = "Star Wars")
+q.setRequestedResult(RequestEventsInfo(sortBy = "date", count = 50))   # request event details for latest 50 events
+
+# get the full list of 50 events at once
+print(er.execQuery(q))
 ```
 
-## Run a Jupyter notebook
+**Search for articles that (a) mention immigration, (b) are related to business, and (c) were published by news sources located in New York City**
 
-We've also prepared an interactive Jupyter notebook where we demonstrate how you can use the SDK. You can run it online and modify the individual examples.
+```python
+from eventregistry import *
+er = EventRegistry(apiKey = YOUR_API_KEY)
 
-**[Run Jupyter notebook with examples](https://mybinder.org/v2/gh/EventRegistry/event-registry-python-intro/master)**
+q = QueryArticlesIter(
+    # here we don't use keywords so we will also get articles that mention immigration using various synonyms
+    conceptUri = er.getConceptUri("immigration"),
+    categoryUri = er.getCategoryUri("business"),
+    sourceLocationUri = er.getLocationUri("New York City"))
 
-## Where to next?
+# obtain 500 articles that have were shared the most on social media
+for art in q.execQuery(er, sortBy = "socialScore", maxItems = 500):
+    print(art)
+```
 
-Depending on your interest and existing knowledge of the `eventregistry` package you can check different things:
+**What are the currently trending topics**
+
+```python
+from eventregistry import *
+er = EventRegistry(apiKey = YOUR_API_KEY)
+
+# top 10 trending concepts in the news
+q = GetTrendingConcepts(source = "news", count = 10)
+print(er.execQuery(q))
+```
+
+## Learning from examples
+
+We believe that it's easiest to learn how to use our service by looking at examples. For this reason, we have prepared examples of various most used features. View the examples grouped by main search actions:
+
+[View examples of searching for articles](https://github.com/EventRegistry/event-registry-python/blob/master/eventregistry/examples/QueryArticlesExamples.py)
+
+[View examples of searching for events](https://github.com/EventRegistry/event-registry-python/blob/master/eventregistry/examples/QueryEventsExamples.py)
+
+[View examples of obtaining information about an individual event](https://github.com/EventRegistry/event-registry-python/blob/master/eventregistry/examples/QueryEventExamples.py)
+
+[Examples of how to obtain the full feed of articles](https://github.com/EventRegistry/event-registry-python/blob/master/eventregistry/examples/FeedOfNewArticlesExamples.py)
+
+[Examples of how to obtain the full feed of events](https://github.com/EventRegistry/event-registry-python/blob/master/eventregistry/examples/FeedOfNewEventsExamples.py)
+
+## Play with interactive Jupyter notebook
+
+To interactively learn about how to use the SDK, see examples of use, see how to get extra meta-data properties, and more, please open [this Binder](https://mybinder.org/v2/gh/EventRegistry/event-registry-python-intro/master). You'll be able to view and modify the examples.
+
+## Where to next?
 
 **[Terminology](../../wiki/Terminology)**. There are numerous terms in the Event Registry that you will constantly see. If you don't know what we mean by an *event*, *story*, *concept* or *category*, you should definitely check this page first.
 
@@ -94,10 +149,6 @@ Depending on your interest and existing knowledge of the `eventregistry` package
 
 **[Articles and events shared the most on social media](../../wiki/Social-shares)**. Do you want to get the list of articles that have been shared the most on Facebook and Twitter on a particular date? What about the most relevant event based on shares on social media?
 
-**[Daily mentions and sentiment of concepts and categories](../../wiki/Number-of-mentions-in-news-or-social-media)**. Are you interested in knowing how often was a particular concept or category mentioned in the news in the previous two years? How about the sentiment expressed on social media about your favorite politician?
-
-**[Correlations of concepts](../../wiki/Correlations)**. Do you have some time series of daily measurements? Why not find the concepts that correlate the most with it based on the number of mentions in the news.
-
 ## Data access and usage restrictions
 
-Event Registry is a commercial service but it allows also unsubscribed users to perform a certain number of operations. Free users are not allowed to use the obtained data for any commercial purposes (see the details on our [Terms of Service page](https://newsapi.ai/terms)). In order to avoid these restrictions please contact us about the [available plans](https://newsapi.ai/plans).
+Event Registry is a commercial service but it allows also unsubscribed users to perform a certain number of operations. Non-paying users are not allowed to use the obtained data for any commercial purposes (see the details on our [Terms of Service page](http://newsapi.ai/terms)) and have access to only last 30 days of content. In order to avoid these restrictions please contact us about the [available plans](http://newsapi.ai/plans).

eventregistry/Analytics.py

@@ -11,18 +11,20 @@
 """
 
 import json
+from typing import Union, List
+from eventregistry.EventRegistry import EventRegistry
 from eventregistry.Base import *
 from eventregistry.ReturnInfo import *
 
 class Analytics:
-    def __init__(self, eventRegistry):
+    def __init__(self, eventRegistry: EventRegistry):
         """
         @param eventRegistry: instance of EventRegistry class
         """
         self._er = eventRegistry
 
 
-    def annotate(self, text, lang = None, customParams = None):
+    def annotate(self, text: str, lang: str = None, customParams: dict = None):
         """
         identify the list of entities and nonentities mentioned in the text
         @param text: input text to annotate
@@ -36,18 +38,21 @@ def annotate(self, text, lang = None, customParams = None):
         return self._er.jsonRequestAnalytics("/api/v1/annotate", params)
 
 
-    def categorize(self, text, taxonomy = "dmoz"):
+    def categorize(self, text: str, taxonomy: str = "dmoz", concepts: List[str] = None):
         """
         determine the set of up to 5 categories the text is about. Currently, only English text can be categorized!
         @param text: input text to categorize
         @param taxonomy: which taxonomy use for categorization. Options "dmoz" (over 5000 categories in 3 levels, English language only)
             or "news" (general news categorization, 9 categories, any langauge)
         @returns: dict
         """
-        return self._er.jsonRequestAnalytics("/api/v1/categorize", { "text": text, "taxonomy": taxonomy })
+        params = { "text": text, "taxonomy": taxonomy }
+        if isinstance(concepts, list) and len(concepts) > 0:
+            params["concepts"] = concepts
+        return self._er.jsonRequestAnalytics("/api/v1/categorize", params)
 
 
-    def sentiment(self, text, method = "vocabulary", sentencesToAnalyze = 10, returnSentences = True):
+    def sentiment(self, text: str, method: str = "vocabulary", sentencesToAnalyze: int = 10, returnSentences: bool = True):
         """
         determine the sentiment of the provided text in English language
         @param text: input text to categorize
@@ -61,7 +66,7 @@ def sentiment(self, text, method = "vocabulary", sentencesToAnalyze = 10, return
         return self._er.jsonRequestAnalytics("/api/v1/sentiment", { "text": text, "method": method, "sentences": sentencesToAnalyze, "returnSentences": returnSentences })
 
 
-    def semanticSimilarity(self, text1, text2, distanceMeasure = "cosine"):
+    def semanticSimilarity(self, text1: str, text2: str, distanceMeasure: str = "cosine"):
         """
         determine the semantic similarity of the two provided documents
         @param text1: first document to analyze
@@ -72,7 +77,7 @@ def semanticSimilarity(self, text1, text2, distanceMeasure = "cosine"):
         return self._er.jsonRequestAnalytics("/api/v1/semanticSimilarity", { "text1": text1, "text2": text2, "distanceMeasure": distanceMeasure })
 
 
-    def detectLanguage(self, text):
+    def detectLanguage(self, text: str):
         """
         determine the language of the given text
         @param text: input text to analyze
@@ -81,7 +86,7 @@ def detectLanguage(self, text):
         return self._er.jsonRequestAnalytics("/api/v1/detectLanguage", { "text": text })
 
 
-    def extractArticleInfo(self, url, proxyUrl = None, headers = None, cookies = None):
+    def extractArticleInfo(self, url: str, proxyUrl: str = None, headers: Union[str, dict] = None, cookies: Union[dict, str] = None):
         """
         extract all available information about an article available at url `url`. Returned information will include
         article title, body, authors, links in the articles, ...
@@ -105,7 +110,7 @@ def extractArticleInfo(self, url, proxyUrl = None, headers = None, cookies = Non
         return self._er.jsonRequestAnalytics("/api/v1/extractArticleInfo", params)
 
 
-    def ner(self, text):
+    def ner(self, text: str):
         """
         extract named entities from the provided text. Supported languages are English, German, Spanish and Chinese.
         @param text: text on wich to extract named entities
@@ -114,9 +119,9 @@ def ner(self, text):
         return self._er.jsonRequestAnalytics("/api/v1/ner", {"text": text})
 
 
-    def trainTopicOnTweets(self, twitterQuery, useTweetText=True, useIdfNormalization=True,
-            normalization="linear", maxTweets=2000, maxUsedLinks=500, ignoreConceptTypes=[],
-            maxConcepts = 20, maxCategories = 10, notifyEmailAddress = None):
+    def trainTopicOnTweets(self, twitterQuery: str, useTweetText: bool = True, useIdfNormalization: bool = True,
+            normalization: bool = "linear", maxTweets: int = 2000, maxUsedLinks: int = 500, ignoreConceptTypes: Union[str, List[str]] = [],
+            maxConcepts: int = 20, maxCategories: int = 10, notifyEmailAddress: str = None):
         """
         create a new topic and train it using the tweets that match the twitterQuery
         @param twitterQuery: string containing the content to search for. It can be a Twitter user account (using "@" prefix or user's Twitter url),
@@ -145,23 +150,23 @@ def trainTopicOnTweets(self, twitterQuery, useTweetText=True, useIdfNormalizatio
         return self._er.jsonRequestAnalytics("/api/v1/trainTopicOnTwitter", params)
 
 
-    def trainTopicCreateTopic(self, name):
+    def trainTopicCreateTopic(self, name: str):
         """
         create a new topic to train. The user should remember the "uri" parameter returned in the result
         @returns object containing the "uri" property that should be used in the follow-up call to trainTopic* methods
         """
         return self._er.jsonRequestAnalytics("/api/v1/trainTopic", { "action": "createTopic", "name": name})
 
 
-    def trainTopicClearTopic(self, uri):
+    def trainTopicClearTopic(self, uri: str):
         """
         if the topic is already existing, clear the definition of the topic. Use this if you want to retrain an existing topic
         @param uri: uri of the topic (obtained by calling trainTopicCreateTopic method) to clear
         """
         return self._er.jsonRequestAnalytics("/api/v1/trainTopic", { "action": "clearTopic", "uri": uri })
 
 
-    def trainTopicAddDocument(self, uri, text):
+    def trainTopicAddDocument(self, uri: str, text: str):
         """
         add the information extracted from the provided "text" to the topic with uri "uri"
         @param uri: uri of the topic (obtained by calling trainTopicCreateTopic method)
@@ -170,8 +175,8 @@ def trainTopicAddDocument(self, uri, text):
         return self._er.jsonRequestAnalytics("/api/v1/trainTopic", { "action": "addDocument", "uri": uri, "text": text})
 
 
-    def trainTopicGetTrainedTopic(self, uri, maxConcepts = 20, maxCategories = 10,
-            ignoreConceptTypes=[], idfNormalization = True):
+    def trainTopicGetTrainedTopic(self, uri: str, maxConcepts: int = 20, maxCategories: int = 10,
+            ignoreConceptTypes: Union[str, List[str]] = [], idfNormalization: bool = True):
         """
         retrieve topic for the topic for which you have already finished training
         @param uri: uri of the topic (obtained by calling trainTopicCreateTopic method)