Version3 for gpt4free model

1. Add the gpt4free model for app, user can choose use gpt4free or openai model.

2. In the gpt4free model, we'll use HuggingFace embedding for document which is no need openai api key.

3. Clean and refactor code.

Author: Lin-jun-xiang

Pipfile

@@ -14,6 +14,8 @@ faiss-cpu = "==1.7.4"
 tiktoken = "==0.4.0"
 tenacity = "==8.1.0"
 google-search-results = "==2.4.2"
+google-cloud-aiplatform = ">=1.26.0"
+sentence-transformers = "*"
 
 [dev-packages]
 

README.md

@@ -18,7 +18,11 @@
 
 ### docGPT-V2 vs docGPT-V3
 
-In docGPT-V3, we introduced the usage of `gpt4free`, allowing users to use the application **for free without entering any API key or making payments** (For more information about [`gpt4free`](https://github.com/xtekky/gpt4free), please refer to the source project).
+In docGPT-V3, we introduced the usage of `gpt4free`, allowing users to use the application **for free without entering any API key or making payments**.
+
+If you want to use the `gpt4free` free model, you need to select a `Provider` (default is `g4f.provider.ChatgptAi`).
+
+(For more information about [`gpt4free`](https://github.com/xtekky/gpt4free), please refer to the source project).
 
 * Version2
   * Utilizes the **`openai` model**
@@ -33,7 +37,7 @@ In docGPT-V3, we introduced the usage of `gpt4free`, allowing users to use the a
     * `openai`: Stable access to the `openai` model by providing an API key
 
 <p align="center">
-<img src="img/2023-08-29-11-51-13.png" width="70%">
+<img src="img/2023-08-29-13-39-00.png" width="70%">
 </p>
 
 If you like this project, please give it a ⭐`Star` to support the developers~
@@ -97,8 +101,8 @@ With Langchain, we can create our own ChatGPT model that can be general-purpose
 
 * Visit the [application](https://docgpt-app.streamlit.app/).
 
-* Enter your API keys:
-    * `OpenAI API Key`: Required.
+* Enter your API keys: (This step is optional in version V3, you can choose to skip it and use the `gpt4free` free model)
+    * `OpenAI API Key`: Make sure you still have usage left
     * `SERPAPI API Key`: Optional. If you want to ask questions about content not appearing in the PDF document, you need this key.
 
 * Upload a PDF file from your local machine.

README.zh-TW.md

@@ -17,7 +17,10 @@
 
 ### docGPT-V2 vs docGPT-V3
 
-我們在 docGPT-V3 引入 `gpt4free` 的調用，讓使用者可以在不輸入任何 api key、付費的情況下，**免費使用**該應用程序
+我們在 docGPT-V3 引入 `gpt4free` 的調用，讓使用者可以在不輸入任何 api key、付費的情況下，**免費使用**該應用程序。
+
+如果您要使用 `gpt4free` 免費模型，您需要選擇 `Provider` (預設是 `g4f.provider.ChatgptAi`)。
+
 (更多 [`gpt4free`](https://github.com/xtekky/gpt4free) 的資訊請參考源專案)
 
 * Version2
@@ -33,7 +36,7 @@
     * `openai`: 帶入 api key，穩定調用 `openai` 模型
 
 <p align="center">
-<img src="img/2023-08-29-11-51-13.png" width="70%">
+<img src="img/2023-08-29-13-39-00.png" width="70%">
 </p>
 
 如果您喜歡這個專案，請給予⭐`Star`以支持開發者~
@@ -100,8 +103,8 @@
 
 * 前往[應用程序](https://docgpt-app.streamlit.app/)
 
-* 輸入您的 `API_KEY`:
-    * `OpenAI API KEY`: 必須設定
+* 輸入您的 `API_KEY` (此步驟在V3版本，可以選擇忽略，使用 `gpt4free` 免費模型):
+    * `OpenAI API KEY`: 務必確認是否還有使用量
     * `SERPAPI API KEY`: 根據您需求，如果您要問**PDF文檔沒有出現**的內容，您就需要用此 KEY
 
 * 上傳來自本地的 PDF 檔案

agent/__init__.py

@@ -8,35 +8,46 @@
 import langchain
 import streamlit as st
 from langchain.cache import InMemoryCache
-from langchain.chat_models import ChatOpenAI
 from streamlit import logger
 from streamlit_chat import message
 
-from agent import AgentHelper
-from docGPT import DocGPT, OpenAiAPI, SerpAPI, GPT4Free
+from docGPT import GPT4Free, create_doc_gpt
 from model import PDFLoader
-import g4f
-from tenacity import retry, stop_after_attempt
 
 langchain.llm_cache = InMemoryCache()
 
 OPENAI_API_KEY = ''
 SERPAPI_API_KEY = ''
-agent_ = None
+model = None
 
 st.session_state.openai_api_key = None
 st.session_state.serpapi_api_key = None
 st.session_state.g4f_provider = None
 app_logger = logger.get_logger(__name__)
 
 
-def theme():
+def theme() -> None:
     st.set_page_config(page_title="DocGPT")
     icon, title = st.columns([3, 20])
     with icon:
         st.image('./img/chatbot.png')
     with title:
         st.title('PDF Chatbot')
+    
+    with st.sidebar:
+
+        with st.expander(':orange[How to use?]'):
+            st.markdown(
+                """
+                1. Enter your API keys: (You can choose to skip it and use the `gpt4free` free model)
+                    * `OpenAI API Key`: Make sure you still have usage left
+                    * `SERPAPI API Key`: Optional. If you want to ask questions about content not appearing in the PDF document, you need this key.
+                2. Upload a PDF file from your local machine.
+                3. Start asking questions!
+                4. More details.(https://github.com/Lin-jun-xiang/docGPT-streamlit)
+                5. If you have any questions, feel free to leave comments and engage in discussions.(https://github.com/Lin-jun-xiang/docGPT-streamlit/issues)
+                """
+            )
 
 
 def load_api_key() -> None:
@@ -80,7 +91,7 @@ def load_api_key() -> None:
         )
 
 
-def upload_and_process_pdf():
+def upload_and_process_pdf() -> list:
     upload_file = st.file_uploader('#### Upload a PDF file:', type='pdf')
     if upload_file:
         temp_file = tempfile.NamedTemporaryFile(delete=False)
@@ -98,79 +109,30 @@ def upload_and_process_pdf():
 
 
 @lru_cache(maxsize=20)
-def get_response(query: str):
+def get_response(query: str) -> str:
     try:
-        if agent_.agent_ is not None:
-            response = agent_.query(query)
+        if model is not None:
+            response = model.run(query)
             return response
     except Exception as e:
-        app_logger.info(e)
+        app_logger.info(f'{__file__}: {e}')
+        return (
+            'Something wrong in docGPT...\n'
+            '1. If you are using gpt4free model, '
+            'try to select the different provider.\n'
+            '2. If you are using openai model, '
+            'check your usage for openai api key.'
+        )
 
 
 theme()
 load_api_key()
 
 doc_container = st.container()
 
-
 with doc_container:
     docs = upload_and_process_pdf()
-
-    if docs:
-        docGPT = DocGPT(docs=docs)
-        docGPT_tool, calculate_tool, search_tool, llm_tool = [None]*4
-        agent_ = AgentHelper()
-
-        if OpenAiAPI.is_valid():
-            # Use openai llm model
-            docGPT.llm = ChatOpenAI(
-                temperature=0.2,
-                max_tokens=6000,
-                model_name='gpt-3.5-turbo-16k'
-            )
-            agent_.llm = ChatOpenAI(
-                temperature=0.2,
-                max_tokens=6000,
-                model_name='gpt-3.5-turbo-16k'
-            )
-            docGPT.create_qa_chain(
-                chain_type='refine',
-            )
-
-            docGPT_tool = agent_.create_doc_chat(docGPT)
-            calculate_tool = agent_.get_calculate_chain
-            llm_tool = agent_.create_llm_chain()
-
-            if SerpAPI.is_valid():
-                search_tool = agent_.get_searp_chain
-        else:
-            # Use gpt4free llm model
-            docGPT.llm = GPT4Free(
-                provider=GPT4Free().PROVIDER_MAPPING[
-                    st.session_state.g4f_provider
-                ]
-            )
-            agent_.llm = GPT4Free(
-                provider=GPT4Free().PROVIDER_MAPPING[
-                    st.session_state.g4f_provider
-                ]
-            )
-            docGPT.create_qa_chain(
-                chain_type='refine',
-            )
-            docGPT_tool = agent_.create_doc_chat(docGPT)
-        try:
-            tools = [
-                docGPT_tool,
-                search_tool,
-                # llm_tool, # This will cause agent confuse
-                calculate_tool
-            ]
-            agent_.initialize(tools)
-        except Exception as e:
-            app_logger.info(e)
-
-
+    model = create_doc_gpt(docs)
     st.write('---')
 
 if 'response' not in st.session_state:

app.py

@@ -1,2 +1,69 @@
-from .docGPT import DocGPT, GPT4Free
+import os
+
+import openai
+import streamlit as st
+from langchain.chat_models import ChatOpenAI
+from streamlit import logger
+
+from .agent import AgentHelper
 from .check_api_key import OpenAiAPI, SerpAPI
+from .docGPT import DocGPT, GPT4Free
+
+openai.api_key = os.getenv('OPENAI_API_KEY')
+os.environ['SERPAPI_API_KEY'] = os.getenv('SERPAPI_API_KEY')
+module_logger = logger.get_logger(__name__)
+
+
+def create_doc_gpt(docs):
+    if not docs:
+        return
+
+    docGPT = DocGPT(docs=docs)
+
+    try:
+        if OpenAiAPI.is_valid():
+            # Use openai llm model with agent
+            docGPT_tool, calculate_tool, search_tool, llm_tool = [None] * 4
+            agent_ = AgentHelper()
+
+            llm_model = ChatOpenAI(
+                temperature=0.2,
+                max_tokens=6000,
+                model_name='gpt-3.5-turbo-16k'
+            )
+            docGPT.llm = llm_model
+            agent_.llm = llm_model
+            with st.spinner('Running...'):
+                docGPT.create_qa_chain(
+                    chain_type='refine',
+                )
+            docGPT_tool = agent_.create_doc_chat(docGPT)
+            calculate_tool = agent_.get_calculate_chain
+            llm_tool = agent_.create_llm_chain()
+
+            if SerpAPI.is_valid():
+                search_tool = agent_.get_searp_chain
+
+            tools = [
+                docGPT_tool,
+                search_tool,
+                # llm_tool, # This will cause agent confuse
+                calculate_tool
+            ]
+            agent_.initialize(tools)
+            return agent_ if agent_ is not None else None
+        else:
+            # Use gpt4free llm model without agent
+            llm_model = GPT4Free(
+                provider=GPT4Free().PROVIDER_MAPPING[
+                    st.session_state.g4f_provider
+                ]
+            )
+            docGPT.llm = llm_model
+            with st.spinner('Running...(free model will take more time)'):
+                docGPT.create_qa_chain(
+                    chain_type='refine',
+                )
+            return docGPT
+    except Exception as e:
+        module_logger.info(f'{__file__}: {e}')

docGPT/__init__.py

@@ -6,7 +6,6 @@
 from langchain.agents import AgentType, Tool, initialize_agent
 from langchain.callbacks import get_openai_callback
 from langchain.chains import LLMChain
-from langchain.chat_models import ChatOpenAI
 from langchain.prompts import PromptTemplate
 
 openai.api_key = os.getenv('OPENAI_API_KEY')
@@ -86,12 +85,15 @@ def initialize(self, tools):
             verbose=True
         )
 
-    def query(self, query: str) -> Optional[str]:
+    def run(self, query: str) -> Optional[str]:
         response = None
         with get_openai_callback() as callback:
-            # TODO: The true result will hide in 'Observation'
-            # https://github.com/hwchase17/langchain/issues/4916
-            # https://python.langchain.com/docs/modules/agents/how_to/intermediate_steps
-            response = self.agent_.run(query)
+            try:
+                response = self.agent_.run(query)
+            except ValueError as e:
+                response = 'Something wrong in agent: ' + str(e)
+                if not response.startswith("Could not parse LLM output: `"):
+                    raise e
+
             print(callback)
         return response

agent/agent.py renamed to docGPT/agent.py

@@ -20,7 +20,7 @@ class OpenAiAPI(ApiKey):
     @classmethod
     def is_valid(cls) -> str:
         if not st.session_state['openai_api_key']:
-            st.error('⚠️ :red[You have not pass OpenAI API key.] Use default key')
+            st.error('⚠️ :red[You have not pass OpenAI API key.] Use default model')
             return
 
         openai.api_key = os.getenv('OPENAI_API_KEY')
@@ -57,4 +57,3 @@ def is_valid(cls) -> str:
                 '[Check your usage](https://serpapi.com/dashboard)'
             )
             print(f'Test error\n{e}')
-