Merge pull request #2694 from Drakkar-Software/dev

Dev merge

Author: GuillaumeDSM

CHANGELOG.md

@@ -6,6 +6,19 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 *It is strongly advised to perform an update of your tentacles after updating OctoBot. (start.py tentacles --install --all)*
 
+## [2.0.3] - 2024-08-03
+### Added
+- [IndexTradingMode]: Default profile, intra-day, real time update option and custom content
+### Updated
+- [Trades] Greatly increase maximum trades in history
+- [Updaters] Restore binary updater
+- [WebInterface] Improve candlesticks display
+- [WebInterface] Improve dark theme UI
+### Fixed
+- [WebInterface] Ignore archived bots in community tab
+- [Kucoin] Websocket issues when using more than 100 feeds
+- [Notifications] Now display real trades PNL or no PNL at all
+
 ## [2.0.2] - 2024-07-17
 ### Updated
 - [WebInterface] Improve light & dark themes display

README.md

@@ -1,4 +1,4 @@
-# OctoBot [2.0.2](https://github.com/Drakkar-Software/OctoBot/blob/master/CHANGELOG.md)
+# OctoBot [2.0.3](https://github.com/Drakkar-Software/OctoBot/blob/master/CHANGELOG.md)
 [![PyPI](https://img.shields.io/pypi/v/OctoBot.svg?logo=pypi)](https://pypi.org/project/OctoBot)
 [![Downloads](https://pepy.tech/badge/octobot/month)](https://pepy.tech/project/octobot)
 [![Dockerhub](https://img.shields.io/docker/pulls/drakkarsoftware/octobot.svg?logo=docker)](https://hub.docker.com/r/drakkarsoftware/octobot)
@@ -12,15 +12,6 @@
 [![Twitter](https://img.shields.io/twitter/follow/DrakkarsOctobot.svg?label=twitter&style=social)](https://x.com/DrakkarsOctoBot)
 [![YouTube](https://img.shields.io/youtube/channel/views/UC2YAaBeWY8y_Olqs79b_X8A?label=youtube&style=social)](https://www.youtube.com/@octobot1134)
 
-## 🎉 OctoBot is launching on Product Hunt on July 10th
-
-<p align="center">
-    <a href="https://www.producthunt.com/posts/octobot-open-source?embed=true&utm_source=badge-featured&utm_medium=badge&utm_souce=badge-octobot&#0045;open&#0045;source" target="_blank"><img src="https://api.producthunt.com/widgets/embed-image/v1/featured.svg?post_id=463219&theme=light" alt="OctoBot&#0032;open&#0032;source - Your&#0032;open&#0032;source&#0032;investment&#0032;strategy&#0032;builder | Product Hunt" style="width: 250px; height: 54px;" width="250" height="54" /></a>
-</p>
-<p align="center">
-    👆 Follow the launch to get your exclusive discount!
-</p>
-
 <p align="center">
   <img src="../assets/illustration.png" alt="Octobot automating trades of its user while the user is relaxing on his couch">
 </p>
@@ -67,15 +58,7 @@ To install OctoBot, you can either:
     Your OctoBot will be accessible on [http://localhost](http://localhost).
 
 ## Exchanges
-[![Binance supported exchange partnership](../assets/binance-logo.png)](https://www.octobot.cloud/en/guides/octobot-partner-exchanges/binance?utm_source=github&utm_medium=dk&utm_campaign=regular_open_source_content&utm_content=binance)
-[![Okx supported exchange partnership](../assets/okex-logo.png)](https://www.octobot.cloud/en/guides/octobot-partner-exchanges/okx?utm_source=github&utm_medium=dk&utm_campaign=regular_open_source_content&utm_content=okx)
-[![Kucoin supported exchange partnership](../assets/kucoin-logo.png)](https://www.octobot.cloud/en/guides/octobot-partner-exchanges/kucoin?utm_source=github&utm_medium=dk&utm_campaign=regular_open_source_content&utm_content=kucoin)
-[![Crypto.com supported exchange partnership](../assets/cryptocom-logo.png)](https://www.octobot.cloud/en/guides/octobot-partner-exchanges/crypto-com?utm_source=github&utm_medium=dk&utm_campaign=regular_open_source_content&utm_content=crypto-com)
-[![Huobi supported exchange partnership](../assets/huobi-logo.png)](https://www.octobot.cloud/en/guides/octobot-partner-exchanges/huobi?utm_source=github&utm_medium=dk&utm_campaign=regular_open_source_content&utm_content=huobi)
-[![Hollaex supported exchange partnership](../assets/hollaex-logo.png)](https://www.octobot.cloud/en/guides/octobot-partner-exchanges/hollaex?utm_source=github&utm_medium=dk&utm_campaign=regular_open_source_content&utm_content=hollaex)
-[![Coinbase supported exchange](../assets/coinbasepro-logo.png)](https://www.octobot.cloud/en/guides/octobot-supported-exchanges/coinbase?utm_source=github&utm_medium=dk&utm_campaign=regular_open_source_content&utm_content=coinbase)
-[![GateIO supported exchange partnership](../assets/gateio-logo.png)](https://www.octobot.cloud/en/guides/octobot-partner-exchanges/gateio?utm_source=github&utm_medium=dk&utm_campaign=regular_open_source_content&utm_content=gateio)
-[![Ascendex supported exchange partnership](../assets/ascendex-logo.png)](https://www.octobot.cloud/en/guides/octobot-partner-exchanges/ascendex?utm_source=github&utm_medium=dk&utm_campaign=regular_open_source_content&utm_content=ascendex)
+[![All OctoBot supported exchanges](../assets/exchange_logo.png)](https://www.octobot.cloud/en/guides/exchanges?utm_source=github&utm_medium=dk&utm_campaign=regular_open_source_content&utm_content=binance)
 
 Octobot supports many [exchanges](https://www.octobot.cloud/en/guides/exchanges?utm_source=github&utm_medium=dk&utm_campaign=regular_open_source_content&utm_content=exchanges) thanks to the [ccxt library](https://github.com/ccxt/ccxt). 
 To activate trading on an exchange, just configure OctoBot with your API keys as described [on the exchange setup guides](https://www.octobot.cloud/en/guides/octobot-configuration/exchanges?utm_source=github&utm_medium=dk&utm_campaign=regular_open_source_content&utm_content=exchanges_setup_guides).

octobot/__init__.py

@@ -16,5 +16,5 @@
 
 PROJECT_NAME = "OctoBot"
 AUTHOR = "Drakkar-Software"
-VERSION = "2.0.2"  # major.minor.revision
+VERSION = "2.0.3"  # major.minor.revision
 LONG_VERSION = f"{VERSION}"

octobot/commands.py

@@ -127,8 +127,8 @@ async def update_or_repair_tentacles_if_necessary(community_auth, selected_profi
 
     to_install_urls, to_remove_tentacles, force_refresh_tentacles_setup_config = \
         community_tentacles_packages.get_to_install_and_remove_tentacles(
-            community_auth, selected_profile_tentacles_setup_config
-        )
+            community_auth, selected_profile_tentacles_setup_config, constants.LONG_VERSION
+        ) if community_auth else ([], [], False)
     if to_remove_tentacles:
         await community_tentacles_packages.uninstall_tentacles(to_remove_tentacles)
     elif force_refresh_tentacles_setup_config:
@@ -167,7 +167,8 @@ async def install_or_update_tentacles(
 
 
 async def install_all_tentacles(
-    tentacles_url=None, additional_tentacles_package_urls: typing.Optional[list] = None, only_additional: bool = False
+    tentacles_url=None, additional_tentacles_package_urls: typing.Optional[list] = None, only_additional: bool = False,
+    bot_version: str = constants.LONG_VERSION
 ):
     if tentacles_url is None:
         tentacles_url = configuration_manager.get_default_tentacles_url()
@@ -185,7 +186,7 @@ async def install_all_tentacles(
             if url is None:
                 continue
             hide_url = url in additional_tentacles_package_urls
-            url = community_tentacles_packages.adapt_url_to_bot_version(url)
+            url = community_tentacles_packages.adapt_url_to_bot_version(url, bot_version)
             await tentacles_manager_api.install_all_tentacles(
                 url,
                 aiohttp_session=aiohttp_session,

octobot/community/authentication.py

@@ -427,7 +427,7 @@ def _get_self_hosted_bots(self, bots):
         return [
             bot
             for bot in bots
-            if self.user_account.is_self_hosted(bot)
+            if self.user_account.is_self_hosted(bot) and not self.user_account.is_archived(bot)
         ]
 
     async def on_new_bot_select(self):

octobot/community/models/community_user_account.py

@@ -74,6 +74,11 @@ def is_self_hosted(self, bot):
             backend_enums.BotDeploymentKeys.TYPE.value, backend_enums.DeploymentTypes.SELF_HOSTED.value
         ) == backend_enums.DeploymentTypes.SELF_HOSTED.value
 
+    def is_archived(self, bot):
+        return self._get_bot_deployment(bot).get(
+            backend_enums.BotDeploymentKeys.STATUS.value
+        ) == backend_enums.BotDeploymentStatus.ARCHIVED.value
+
     def get_selected_bot_deployment_id(self):
         return self.get_bot_deployment_value(backend_enums.BotDeploymentKeys.ID)
 

octobot/community/supabase_backend/enums.py

@@ -77,6 +77,7 @@ class BotDeploymentStatus(enum.Enum):
     STOPPED = "stopped"
     PENDING = "pending"
     UNKNOWN = "unknown"
+    ARCHIVED = "archived"
 
 
 class ProductSubscriptionDesiredStatus(enum.Enum):

octobot/community/tentacles_packages.py

@@ -23,7 +23,7 @@ async def has_tentacles_to_install_and_uninstall_tentacles_if_necessary(communit
         community_auth.config.get_tentacles_config_path()
     )
     to_install, to_remove_tentacles, force_refresh_tentacles_setup_config = get_to_install_and_remove_tentacles(
-        community_auth, tentacles_setup_config
+        community_auth, tentacles_setup_config, constants.LONG_VERSION
     )
     if to_remove_tentacles:
         logging.get_logger(__name__).debug(
@@ -36,29 +36,29 @@ async def has_tentacles_to_install_and_uninstall_tentacles_if_necessary(communit
     return bool(to_install)
 
 
-def adapt_url_to_bot_version(package_url: str) -> str:
+def adapt_url_to_bot_version(package_url: str, bot_version: str) -> str:
     if constants.VERSION_PLACEHOLDER in package_url:
-        package_url = package_url.replace(constants.VERSION_PLACEHOLDER, constants.LONG_VERSION)
+        package_url = package_url.replace(constants.VERSION_PLACEHOLDER, bot_version)
     return package_url
 
 
 def get_to_install_and_remove_tentacles(
-    community_auth, selected_profile_tentacles_setup_config
+    community_auth, selected_profile_tentacles_setup_config, bot_version: str
 ):
     installed_community_package_urls = [
-        adapt_url_to_bot_version(package_url)
+        adapt_url_to_bot_version(package_url, bot_version)
         for package_url in tentacles_manager_api.get_all_installed_package_urls(
             selected_profile_tentacles_setup_config
         )
         if is_community_tentacle_url(package_url)
     ]
     additional_tentacles_package_urls = [
-        adapt_url_to_bot_version(package_url)
+        adapt_url_to_bot_version(package_url, bot_version)
         for package_url in community_auth.get_saved_package_urls()
     ] if community_auth.is_logged_in() else []
     to_keep_tentacles = set(
         additional_tentacles_package_urls + [
-            adapt_url_to_bot_version(package_url)
+            adapt_url_to_bot_version(package_url, bot_version)
             for package_url in get_env_variable_tentacles_urls()
         ] + get_env_variable_tentacles_urls()
     )

octobot/task_manager.py

@@ -26,6 +26,9 @@
 import octobot.constants as constants
 
 
+ASYNC_IGNORED_ERROR_MESSAGES = ["'message': 'Unclosed client session'"]
+
+
 class TaskManager:
     """TaskManager class:
         - Create, manage and stop octobot tasks
@@ -136,11 +139,19 @@ def create_pool_executor(self, workers=2):
     def _loop_exception_handler(self, loop, context):
         loop_str = "bot main async" if loop is self.async_loop else {loop}
         message = f"Error in {loop_str} loop: {context}"
+        function = self.logger.debug if self._should_use_debug_for_message(message) else self.logger.warning
         exception = context.get('exception')
         if exception is not None:
             formatted_traceback = "\n".join(traceback.format_tb(exception.__traceback__))
             message = f"{message}:\n{formatted_traceback}"
-        self.logger.warning(message)
+        function(message)
+
+    @staticmethod
+    def _should_use_debug_for_message(message: str) -> bool:
+        for ignored_message in ASYNC_IGNORED_ERROR_MESSAGES:
+            if ignored_message in message:
+                return True
+        return False
 
     def _create_new_asyncio_main_loop(self):
         self.async_loop = asyncio.new_event_loop()

octobot/updater/binary_updater.py

@@ -56,10 +56,6 @@ async def get_latest_version(self):
         return self._parse_latest_version(await self._get_latest_release_data())
 
     async def update_impl(self) -> bool:
-        # TODO fix binary updater to use release endpoint (assents can't be downloaded from gh directly)
-        self.logger.error(f"Please manually update your OctoBot executable from this page: "
-                          f"{self._get_latest_release_url(False)}")
-        return False
         new_binary_file = await self._download_binary()
         if new_binary_file is not None:
             self._give_execution_rights(new_binary_file)
@@ -73,10 +69,11 @@ def _get_latest_release_url(self, use_api_url):
 
     async def _get_latest_release_data(self):
         try:
-            async with aiohttp.ClientSession().get(self._get_latest_release_url(True)) as resp:
-                text = await resp.text()
-                if resp.status == 200:
-                    return json.loads(text)
+            async with aiohttp.ClientSession() as session:
+                async with session.get(self._get_latest_release_url(True)) as resp:
+                    text = await resp.text()
+                    if resp.status == 200:
+                        return json.loads(text)
             return None
         except Exception as e:
             self.logger.debug(f"Error when fetching latest binary data : {e}")
@@ -137,10 +134,11 @@ async def _download_binary(self):
             return None
         self.logger.info(f"Start downloading OctoBot update at {new_binary_file_url}")
         async with aiofiles.open(new_binary_file, 'wb+') as downloaded_file:
-            await aiohttp_util.download_stream_file(output_file=downloaded_file,
-                                                    file_url=new_binary_file_url,
-                                                    aiohttp_session=aiohttp.ClientSession(),
-                                                    is_aiofiles_output_file=True)
+            async with aiohttp.ClientSession() as session:
+                await aiohttp_util.download_stream_file(output_file=downloaded_file,
+                                                        file_url=new_binary_file_url,
+                                                        aiohttp_session=session,
+                                                        is_aiofiles_output_file=True)
         self.logger.info(f"OctoBot update downloaded successfully")
         return new_binary_file
 

octobot/updater/python_updater.py

@@ -49,10 +49,11 @@ def _get_latest_pypi_release_url(self):
 
     async def _get_latest_pypi_version_data(self):
         try:
-            async with aiohttp.ClientSession().get(self._get_latest_pypi_release_url()) as resp:
-                text = await resp.text()
-                if resp.status == 200:
-                    return json.loads(text)
+            async with aiohttp.ClientSession() as session:
+                async with session.get(self._get_latest_pypi_release_url()) as resp:
+                    text = await resp.text()
+                    if resp.status == 200:
+                        return json.loads(text)
             return None
         except Exception as e:
             self.logger.debug(f"Error when fetching latest pypi package data : {e}")

octobot/updater/updater.py

@@ -45,11 +45,13 @@ async def update_impl(self) -> bool:
         raise NotImplementedError("update_impl is not implemented")
 
     async def update_tentacles(self):
+        bot_version = await self.get_latest_version()
         authenticator = authentication.Authenticator.instance()
         additional_tentacles_package_urls = authenticator.get_saved_package_urls()
         await commands.install_all_tentacles(
-            tentacles_url=configuration_manager.get_default_tentacles_url(version=await self.get_latest_version()),
-            additional_tentacles_package_urls=additional_tentacles_package_urls
+            tentacles_url=configuration_manager.get_default_tentacles_url(version=bot_version),
+            additional_tentacles_package_urls=additional_tentacles_package_urls,
+            bot_version=bot_version
         )
 
     async def post_update(self):

requirements.txt

@@ -1,12 +1,12 @@
 # Drakkar-Software requirements
-OctoBot-Commons==1.9.50
-OctoBot-Trading==2.4.92
+OctoBot-Commons==1.9.51
+OctoBot-Trading==2.4.95
 OctoBot-Evaluators==1.9.5
 OctoBot-Tentacles-Manager==2.9.15
-OctoBot-Services==1.6.15
+OctoBot-Services==1.6.16
 OctoBot-Backtesting==1.9.7
 Async-Channel==2.2.1
-trading-backend==1.2.25
+trading-backend==1.2.26
 
 ## Others
 colorlog==6.8.0
@@ -27,6 +27,10 @@ supafunc==0.2.3   # Supabase functions calls (required by supabase and enforced
 postgrest==0.10.8 # Supabase posgres calls (required by supabase and enforced to allow direct import)
 realtime==1.0.0   # Supabase realtime lib (required by supabase and enforced to allow direct import)
 
+# async http requests
+# avoid v3.10.0 as it is failing CI (<class 'RuntimeError'>: aiodns needs a SelectorEventLoop on Windows. See more: https://github.com/saghul/aiodns/issues/86)
+aiohttp==3.9.5
+
 # Experimental to prevent httpx.PoolTimeout
 httpcore==0.17.3  # to up to at least 1.0.2 (prevent default version when installing httpx which creates httpx.PoolTimeout)
 anyio==4.0.0      #