From 3a46e726811fb0a076fd285b859b34777de632e0 Mon Sep 17 00:00:00 2001 From: semantic-release-bot Date: Sat, 20 Jul 2024 18:22:26 +0000 Subject: [PATCH 1/7] ci(release): 1.10.0 [skip ci] ## [1.10.0](https://github.com/ScrapeGraphAI/Scrapegraph-ai/compare/v1.9.2...v1.10.0) (2024-07-20) ### Features * add gpt4o omni ([431edb7](https://github.com/ScrapeGraphAI/Scrapegraph-ai/commit/431edb7bb2504f4c1335c3ae3ce2f91867fa7222)) * add searchngx integration ([5c92186](https://github.com/ScrapeGraphAI/Scrapegraph-ai/commit/5c9218608140bf694fbfd96aa90276bc438bb475)) * refactoring_to_md function ([602dd00](https://github.com/ScrapeGraphAI/Scrapegraph-ai/commit/602dd00209ee1d72a1223fc4793759450921fcf9)) ### Bug Fixes * add gpt o mini for azure ([77777c8](https://github.com/ScrapeGraphAI/Scrapegraph-ai/commit/77777c898d1fad40f340b06c5b36d35b65409ea6)) * parse_node ([07f1e23](https://github.com/ScrapeGraphAI/Scrapegraph-ai/commit/07f1e23d235db1a0db2cb155f10b73b0bf882269)) * search link node ([cf3ab55](https://github.com/ScrapeGraphAI/Scrapegraph-ai/commit/cf3ab5564ae5c415c63d1771b32ea68f5169ca82)) ### chore * correct search engine name ([7ba2f6a](https://github.com/ScrapeGraphAI/Scrapegraph-ai/commit/7ba2f6ae0b9d2e9336e973e1f57ab8355c739e27)) * remove unused import ([fd1b7cb](https://github.com/ScrapeGraphAI/Scrapegraph-ai/commit/fd1b7cb24a7c252277607abde35826e3c58e34ef)) * **ci:** upgrade lockfiles ([c7b05a4](https://github.com/ScrapeGraphAI/Scrapegraph-ai/commit/c7b05a4993df14d6ed4848121a3cd209571232f7)) * upgrade tiktoken ([7314bc3](https://github.com/ScrapeGraphAI/Scrapegraph-ai/commit/7314bc383068db590662bf7e512f799529308991)) ### Docs * **gpt-4o-mini:** added new gpt, fixed chromium lazy loading, ([99dc849](https://github.com/ScrapeGraphAI/Scrapegraph-ai/commit/99dc8497d85289759286a973e4aecc3f924d3ada)) ### CI * **release:** 1.10.0-beta.1 [skip ci] ([8f619de](https://github.com/ScrapeGraphAI/Scrapegraph-ai/commit/8f619de23540216934b53bcf3426702e56c48f31)) * **release:** 1.10.0-beta.2 [skip ci] ([aa7d4f0](https://github.com/ScrapeGraphAI/Scrapegraph-ai/commit/aa7d4f0ebfc2623a51ce1e4887ff26c9906b0a95)) * **release:** 1.10.0-beta.3 [skip ci] ([bf0a2f3](https://github.com/ScrapeGraphAI/Scrapegraph-ai/commit/bf0a2f386f38cbe81d1e5ea3e05357f8ecabcab2)) * **release:** 1.10.0-beta.4 [skip ci] ([a91807a](https://github.com/ScrapeGraphAI/Scrapegraph-ai/commit/a91807a20cc07b15feb1ddd5cf7a1c323ff32b46)) * **release:** 1.10.0-beta.5 [skip ci] ([0d5f925](https://github.com/ScrapeGraphAI/Scrapegraph-ai/commit/0d5f9259d8fb148de7c95cf6f67f9562c5d2c880)) * **release:** 1.9.0-beta.3 [skip ci] ([d3e63d9](https://github.com/ScrapeGraphAI/Scrapegraph-ai/commit/d3e63d91be79f74e8a3fdb00e692d546c24cead5)) * **release:** 1.9.0-beta.4 [skip ci] ([2fa04b5](https://github.com/ScrapeGraphAI/Scrapegraph-ai/commit/2fa04b58159abf7af890ebc0768fe23d51bf177f)) * **release:** 1.9.0-beta.5 [skip ci] ([bb62439](https://github.com/ScrapeGraphAI/Scrapegraph-ai/commit/bb624399cfc3924825892dd48697fc298ad3b002)) * **release:** 1.9.0-beta.6 [skip ci] ([54a69de](https://github.com/ScrapeGraphAI/Scrapegraph-ai/commit/54a69de69e8077e02fd5584783ca62cc2e0ec5bb)) --- CHANGELOG.md | 42 ++++++++++++++++++++++++++++++++++++++++++ pyproject.toml | 2 +- 2 files changed, 43 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 01fb0c3a..b79488b7 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,3 +1,45 @@ +## [1.10.0](https://github.com/ScrapeGraphAI/Scrapegraph-ai/compare/v1.9.2...v1.10.0) (2024-07-20) + + +### Features + +* add gpt4o omni ([431edb7](https://github.com/ScrapeGraphAI/Scrapegraph-ai/commit/431edb7bb2504f4c1335c3ae3ce2f91867fa7222)) +* add searchngx integration ([5c92186](https://github.com/ScrapeGraphAI/Scrapegraph-ai/commit/5c9218608140bf694fbfd96aa90276bc438bb475)) +* refactoring_to_md function ([602dd00](https://github.com/ScrapeGraphAI/Scrapegraph-ai/commit/602dd00209ee1d72a1223fc4793759450921fcf9)) + + +### Bug Fixes + +* add gpt o mini for azure ([77777c8](https://github.com/ScrapeGraphAI/Scrapegraph-ai/commit/77777c898d1fad40f340b06c5b36d35b65409ea6)) +* parse_node ([07f1e23](https://github.com/ScrapeGraphAI/Scrapegraph-ai/commit/07f1e23d235db1a0db2cb155f10b73b0bf882269)) +* search link node ([cf3ab55](https://github.com/ScrapeGraphAI/Scrapegraph-ai/commit/cf3ab5564ae5c415c63d1771b32ea68f5169ca82)) + + +### chore + +* correct search engine name ([7ba2f6a](https://github.com/ScrapeGraphAI/Scrapegraph-ai/commit/7ba2f6ae0b9d2e9336e973e1f57ab8355c739e27)) +* remove unused import ([fd1b7cb](https://github.com/ScrapeGraphAI/Scrapegraph-ai/commit/fd1b7cb24a7c252277607abde35826e3c58e34ef)) +* **ci:** upgrade lockfiles ([c7b05a4](https://github.com/ScrapeGraphAI/Scrapegraph-ai/commit/c7b05a4993df14d6ed4848121a3cd209571232f7)) +* upgrade tiktoken ([7314bc3](https://github.com/ScrapeGraphAI/Scrapegraph-ai/commit/7314bc383068db590662bf7e512f799529308991)) + + +### Docs + +* **gpt-4o-mini:** added new gpt, fixed chromium lazy loading, ([99dc849](https://github.com/ScrapeGraphAI/Scrapegraph-ai/commit/99dc8497d85289759286a973e4aecc3f924d3ada)) + + +### CI + +* **release:** 1.10.0-beta.1 [skip ci] ([8f619de](https://github.com/ScrapeGraphAI/Scrapegraph-ai/commit/8f619de23540216934b53bcf3426702e56c48f31)) +* **release:** 1.10.0-beta.2 [skip ci] ([aa7d4f0](https://github.com/ScrapeGraphAI/Scrapegraph-ai/commit/aa7d4f0ebfc2623a51ce1e4887ff26c9906b0a95)) +* **release:** 1.10.0-beta.3 [skip ci] ([bf0a2f3](https://github.com/ScrapeGraphAI/Scrapegraph-ai/commit/bf0a2f386f38cbe81d1e5ea3e05357f8ecabcab2)) +* **release:** 1.10.0-beta.4 [skip ci] ([a91807a](https://github.com/ScrapeGraphAI/Scrapegraph-ai/commit/a91807a20cc07b15feb1ddd5cf7a1c323ff32b46)) +* **release:** 1.10.0-beta.5 [skip ci] ([0d5f925](https://github.com/ScrapeGraphAI/Scrapegraph-ai/commit/0d5f9259d8fb148de7c95cf6f67f9562c5d2c880)) +* **release:** 1.9.0-beta.3 [skip ci] ([d3e63d9](https://github.com/ScrapeGraphAI/Scrapegraph-ai/commit/d3e63d91be79f74e8a3fdb00e692d546c24cead5)) +* **release:** 1.9.0-beta.4 [skip ci] ([2fa04b5](https://github.com/ScrapeGraphAI/Scrapegraph-ai/commit/2fa04b58159abf7af890ebc0768fe23d51bf177f)) +* **release:** 1.9.0-beta.5 [skip ci] ([bb62439](https://github.com/ScrapeGraphAI/Scrapegraph-ai/commit/bb624399cfc3924825892dd48697fc298ad3b002)) +* **release:** 1.9.0-beta.6 [skip ci] ([54a69de](https://github.com/ScrapeGraphAI/Scrapegraph-ai/commit/54a69de69e8077e02fd5584783ca62cc2e0ec5bb)) + ## [1.10.0-beta.5](https://github.com/ScrapeGraphAI/Scrapegraph-ai/compare/v1.10.0-beta.4...v1.10.0-beta.5) (2024-07-20) diff --git a/pyproject.toml b/pyproject.toml index c42bf33b..daa040c5 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -2,7 +2,7 @@ name = "scrapegraphai" -version = "1.10.0b5" +version = "1.10.0" From 208ab267ceda30b4527222d9dfd61e5c5ed243c3 Mon Sep 17 00:00:00 2001 From: Marco Vinciguerra Date: Sat, 20 Jul 2024 21:05:31 +0200 Subject: [PATCH 2/7] Create requirements.txt --- requirements.txt | 25 +++++++++++++++++++++++++ 1 file changed, 25 insertions(+) create mode 100644 requirements.txt diff --git a/requirements.txt b/requirements.txt new file mode 100644 index 00000000..502e9df5 --- /dev/null +++ b/requirements.txt @@ -0,0 +1,25 @@ +beautifulsoup4==4.12.3 +burr==0.24.0 +free_proxy==1.1.1 +google==3.0.0 +graphviz==0.20.3 +html2text==2024.2.26 +langchain==0.2.10 +langchain_anthropic==0.1.20 +langchain_aws==0.1.11 +langchain_community==0.2.9 +langchain_core==0.2.22 +langchain_fireworks==0.1.5 +langchain_google_genai==1.0.7 +langchain_google_vertexai==1.0.6 +langchain_groq==0.1.6 +langchain_openai==0.1.17 +minify_html==0.15.0 +pandas==2.2.2 +playwright==1.43.0 +pydantic==2.8.2 +Requests==2.32.3 +semchunk==2.2.0 +tiktoken==0.7.0 +tqdm==4.66.4 +undetected_playwright==0.3.0 From ce6be37fbc1095afe4df6a2fc206923e477190e5 Mon Sep 17 00:00:00 2001 From: Marco Vinciguerra Date: Sun, 21 Jul 2024 18:37:15 +0200 Subject: [PATCH 3/7] fix: abstract_graph moel token bug --- scrapegraphai/graphs/abstract_graph.py | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/scrapegraphai/graphs/abstract_graph.py b/scrapegraphai/graphs/abstract_graph.py index 7f8ec4ea..ca2b2e81 100644 --- a/scrapegraphai/graphs/abstract_graph.py +++ b/scrapegraphai/graphs/abstract_graph.py @@ -148,6 +148,10 @@ def _create_llm(self, llm_config: dict, chat=False) -> object: # If model instance is passed directly instead of the model details if "model_instance" in llm_params: + try: + self.model_token = llm_params["model_tokens"] + except KeyError as exc: + raise KeyError("model_tokens not specified") from exc return llm_params["model_instance"] # Instantiate the language model based on the model name From 03f528a34e385f16fed2070ddf64717de0036fb2 Mon Sep 17 00:00:00 2001 From: semantic-release-bot Date: Sun, 21 Jul 2024 16:38:44 +0000 Subject: [PATCH 4/7] ci(release): 1.10.1 [skip ci] ## [1.10.1](https://github.com/ScrapeGraphAI/Scrapegraph-ai/compare/v1.10.0...v1.10.1) (2024-07-21) ### Bug Fixes * abstract_graph moel token bug ([ce6be37](https://github.com/ScrapeGraphAI/Scrapegraph-ai/commit/ce6be37fbc1095afe4df6a2fc206923e477190e5)) --- CHANGELOG.md | 7 +++++++ pyproject.toml | 2 +- 2 files changed, 8 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index b79488b7..a942a57d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,3 +1,10 @@ +## [1.10.1](https://github.com/ScrapeGraphAI/Scrapegraph-ai/compare/v1.10.0...v1.10.1) (2024-07-21) + + +### Bug Fixes + +* abstract_graph moel token bug ([ce6be37](https://github.com/ScrapeGraphAI/Scrapegraph-ai/commit/ce6be37fbc1095afe4df6a2fc206923e477190e5)) + ## [1.10.0](https://github.com/ScrapeGraphAI/Scrapegraph-ai/compare/v1.9.2...v1.10.0) (2024-07-20) diff --git a/pyproject.toml b/pyproject.toml index daa040c5..60fe1ef9 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -2,7 +2,7 @@ name = "scrapegraphai" -version = "1.10.0" +version = "1.10.1" From b49f9866fd86f0fc873227a4f7eecd8248c7b80f Mon Sep 17 00:00:00 2001 From: Marco Vinciguerra Date: Sun, 21 Jul 2024 19:20:19 +0200 Subject: [PATCH 5/7] fixed telemetry version --- scrapegraphai/telemetry/telemetry.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scrapegraphai/telemetry/telemetry.py b/scrapegraphai/telemetry/telemetry.py index d1c8a367..3bf6d90f 100644 --- a/scrapegraphai/telemetry/telemetry.py +++ b/scrapegraphai/telemetry/telemetry.py @@ -91,7 +91,7 @@ def _check_config_and_environ_for_telemetry_flag( "python_version": f"{platform.python_version()}/{platform.python_implementation()}", "distinct_id": g_anonymous_id, "scrapegraphai_version": VERSION, - "telemetry_version": "0.0.1", + "telemetry_version": "0.0.2", } From b0418b679cf45e1e680d2daadcc47e6e4f585575 Mon Sep 17 00:00:00 2001 From: Marco Vinciguerra Date: Sun, 21 Jul 2024 19:23:40 +0200 Subject: [PATCH 6/7] fix: telemetry version From da451e5f1618e70eb1bd48a4ff3ad65ba31b0a84 Mon Sep 17 00:00:00 2001 From: semantic-release-bot Date: Sun, 21 Jul 2024 17:25:05 +0000 Subject: [PATCH 7/7] ci(release): 1.10.2 [skip ci] ## [1.10.2](https://github.com/ScrapeGraphAI/Scrapegraph-ai/compare/v1.10.1...v1.10.2) (2024-07-21) ### Bug Fixes * telemetry version ([b0418b6](https://github.com/ScrapeGraphAI/Scrapegraph-ai/commit/b0418b679cf45e1e680d2daadcc47e6e4f585575)) --- CHANGELOG.md | 7 +++++++ pyproject.toml | 2 +- 2 files changed, 8 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index a942a57d..0b35e2c2 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,3 +1,10 @@ +## [1.10.2](https://github.com/ScrapeGraphAI/Scrapegraph-ai/compare/v1.10.1...v1.10.2) (2024-07-21) + + +### Bug Fixes + +* telemetry version ([b0418b6](https://github.com/ScrapeGraphAI/Scrapegraph-ai/commit/b0418b679cf45e1e680d2daadcc47e6e4f585575)) + ## [1.10.1](https://github.com/ScrapeGraphAI/Scrapegraph-ai/compare/v1.10.0...v1.10.1) (2024-07-21) diff --git a/pyproject.toml b/pyproject.toml index 60fe1ef9..02c8d0bc 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -2,7 +2,7 @@ name = "scrapegraphai" -version = "1.10.1" +version = "1.10.2"