Merge pull request #1972 from expectedparrot/docs_0523

Added note about reasoning models and example nb

Author: rbyh

docs/credits.rst

@@ -43,7 +43,6 @@ To purchase credits, navigate to the `Credits <https://www.expectedparrot.com/ho
 
 *Note:*
 Payments are processed by Stripe. 
-You may be charged payment processing fees when purchasing credits.
 
 
 Using credits

docs/humanize.rst

@@ -0,0 +1,91 @@
+.. _humanize:
+
+Humanize
+========
+
+EDSL comes with built-in methods for generating web-based versions of your surveys and collecting and analyzing human responses.
+
+*Note:* This page provides information about EDSL code methods for launching surveys, gathering human responses, and analyzing results from your workspace. 
+For more information about building surveys interactively, see the :ref:`survey_builder` page.
+
+
+How it works
+------------
+
+1. Create a `Survey` (see the :ref:`surveys` section for details and examples).
+2. Use the `run` method to launch your survey with AI :ref:`agents` and :ref:`language_models`. 
+3. Use the `humanize` method to generate a web-based version of your survey, with a link that can be shared with human respondents and another link for the admin page at your Coop account where you can access responses interactively. *Note:* You may want to modify your LLM-based survey to add or change questions specific to human respondents before calling the `humanize` method, e.g., to add screening questions for aligning human respondents and AI agent personas.
+4. Share the web survey link with human respondents.
+5. Use the `Coop().get_project_human_responses` method to collect the responses in a `Results` object.
+6. Analyze the results together with your LLM results.
+
+
+Example
+-------
+
+Code and results for the example below are also accessible at this downloadable `notebook at Coop <https://www.expectedparrot.com/content/RobinHorton/human-results-example-notebook>`_.
+
+.. code-block:: python
+
+    # Import modules from EDSL
+    from edsl import (
+        QuestionYesNo,
+        QuestionNumerical,
+        QuestionLinearScale,
+        Survey,
+        Agent,
+        Model,
+        Coop
+    )
+
+    # Create a survey with different question types
+    q1 = QuestionYesNo(
+        question_name="drive", 
+        question_text="Do you drive?"
+    )
+
+    q2 = QuestionNumerical(
+        question_name="count",
+        question_text="How many vehicles do you currently own or lease?",
+    )
+
+    q3 = QuestionLinearScale(
+        question_name="enjoy",
+        question_text="On a scale from 1 to 10, how much do you enjoy driving?",
+        question_options=[1, 2, 3, 4, 5, 6, 7, 8, 9, 10],
+        option_labels={1: "Hate it", 10: "Love it"},
+    )
+
+    # Create a survey with the questions
+    survey = Survey(questions=[q1, q2, q3])
+
+    # Create an AI agent to respond to the survey
+    agent = Agent(
+        traits={
+            "persona": "You are a middle-aged mom working on a software startup.",
+            "location": "Massachusetts",
+        }
+    )
+
+    # Select a language model to generate the responses
+    model = Model("gemini-1.5-pro", service_name="google")
+
+    # Run the survey with the AI agent and model
+    results = survey.by(agent).by(model).run()
+
+    # Generate a web-based version of the survey for human respondents
+    web_survey_info = survey.humanize()
+
+    # Create a Coop instance
+    coop = Coop()
+
+    # Get human responses from Coop
+    human_responses = coop.get_project_human_responses(web_survey_info["uuid"])
+
+    # Combine results (you can add Results objects for the same survey)
+    combined_results = results + human_results
+
+
+*We are continually adding features for launching hybrid LLM and human surveys, so check back for updates!*
+*If you are interested in testing new features please reach out at anytime for credits and access.*
+

docs/index.rst

@@ -5,11 +5,11 @@ Expected Parrot: Tools for AI-Powered Research
 
 Expected Parrot delivers powerful tools for conducting research with human and artificial intelligences.
 
-This page provides documentation for **Expected Parrot Domain-Specific Language (EDSL)**, a Python package for performing research with AI agents and language models,
-and **Coop**, a platform for creating, storing and sharing AI-based research projects.
+This page provides documentation for **Expected Parrot Domain-Specific Language (EDSL)**, an open-source Python package for performing research with AI agents and language models,
+and **Coop**, a platform for creating, storing and sharing AI research projects.
 
 * EDSL is available to download from `PyPI <https://pypi.org/project/edsl/>`_ (run `pip install edsl`). The source code is available at `GitHub <https://github.com/expectedparrot/edsl>`_.
-* `Create an account <https://www.expectedparrot.com/login>`_ to post and share content, run surveys and store results at the Expected Parrot survey. Learn more about `how it works <https://docs.expectedparrot.com/en/latest/coop.html>`_ and start `exploring <https://www.expectedparrot.com/content/explore>`_.
+* `Create an account <https://www.expectedparrot.com/login>`_ to post and share content, run surveys with LLMs and humans, and store results at the Expected Parrot server. Learn more about `how it works <https://docs.expectedparrot.com/en/latest/coop.html>`_ and start `exploring <https://www.expectedparrot.com/content/explore>`_.
 
 
 Key features 
@@ -94,6 +94,10 @@ Please see the links in the steps below for more details:
    Read the :ref:`starter_tutorial` and `download a notebook <https://www.expectedparrot.com/content/179b3a78-2505-4568-acd9-c09d18953288>`_ to create a survey and run it.
    See examples for many use cases and `tips <https://docs.expectedparrot.com/en/latest/checklist.html>`_ on using EDSL effectively in the documentation.
 
+5. **Validate with real respondents.**
+
+   You can run surveys with real respondents using the Coop platform or at your workspace.
+   Learn about methods for generating web-based surveys and collecting responses in the :ref:`survey_builder` and :ref:`humanize` sections.
 
 Join our `Discord channel <https://discord.com/invite/mxAYkjfy9m>`_ to ask questions and chat with other users!
 
@@ -142,6 +146,7 @@ Coop
 It is fully integrated with EDSL and provides access to special features for working with AI agents and language models, free storage and collaboration tools, including:
 
 - :ref:`survey_builder`: A user-friendly no-code interface for creating surveys and gathering responses from humans and AI agents.
+- :ref:`humanize`: Generate web-based surveys and collect responses from human respondents.
 - :ref:`remote_inference`: Access all available language models and run surveys at the Expected Parrot server. 
 - :ref:`remote_caching`: Automatically store results and API calls at the Expected Parrot server. 
 - :ref:`notebooks` & :ref:`colab_notebooks`: Easily post and share `.ipynb` and `.py` files to the Coop and access with Colab. 
@@ -173,6 +178,7 @@ Examples of special methods and use cases for EDSL, including:
 - Conducting agent conversations
 - Converting surveys into EDSL
 - Cognitive testing 
+- Validating LLM answers with humans
 - Research methods
 
 
@@ -227,6 +233,7 @@ Links
    :hidden:
 
    results
+   humanize
    dataset
    data
    exceptions
@@ -285,6 +292,8 @@ Links
    :caption: Notebooks
    :hidden:
 
+   notebooks/data_labeling_validation_example.ipynb
+   notebooks/reasoning_model_example.ipynb
    notebooks/next_token_probs.ipynb
    notebooks/summarizing_transcripts.ipynb
    notebooks/analyze_evaluations.ipynb

docs/language_models.rst

@@ -56,6 +56,9 @@ Output:
    * - xai
 
 
+*Note*: We recently added support for OpenAI reasoning models. See an example notebook for usage `here <https://www.expectedparrot.com/content/RobinHorton/reasoning-model-example>`_.
+Use `service_name = "openai_v2"` when using these models. The `Results` that are generated with reasoning models include additional fields for reasoning summaries.
+
 .. Available models
 .. ----------------