Skip to content

Commit bcbb0af

Browse files
khuyentran1401khuyentran1401
committed
add pyspark join
1 parent 5e1d988 commit bcbb0af

File tree

12 files changed

+383
-27798
lines changed

12 files changed

+383
-27798
lines changed

‎Chapter5/spark.ipynb

Lines changed: 129 additions & 9188 deletions
Large diffs are not rendered by default.

‎Chapter5/testing.ipynb

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4176,6 +4176,14 @@
41764176
"pytest-postgres provides fixtures that manage the setup and cleanup of test databases, ensuring repeatable tests. Additionally, each test runs in isolation, preventing any impact on the production database from testing changes."
41774177
]
41784178
},
4179+
{
4180+
"cell_type": "markdown",
4181+
"id": "737b7abf",
4182+
"metadata": {},
4183+
"source": [
4184+
"To see how pytest-postgres works, let's create a test function that sets up a test table in a PostgreSQL database, inserts some test data, and then verifies that the query results match the expected data."
4185+
]
4186+
},
41794187
{
41804188
"cell_type": "code",
41814189
"execution_count": null,

‎Chapter5/time_series.ipynb

Lines changed: 12 additions & 86 deletions
Original file line numberDiff line numberDiff line change
@@ -3359,101 +3359,27 @@
33593359
},
33603360
{
33613361
"cell_type": "code",
3362-
"execution_count": 1,
3362+
"execution_count": 7,
33633363
"id": "4f410242",
33643364
"metadata": {},
33653365
"outputs": [
33663366
{
3367-
"data": {
3368-
"text/html": [
3369-
"<div>\n",
3370-
"<style scoped>\n",
3371-
" .dataframe tbody tr th:only-of-type {\n",
3372-
" vertical-align: middle;\n",
3373-
" }\n",
3374-
"\n",
3375-
" .dataframe tbody tr th {\n",
3376-
" vertical-align: top;\n",
3377-
" }\n",
3378-
"\n",
3379-
" .dataframe thead th {\n",
3380-
" text-align: right;\n",
3381-
" }\n",
3382-
"</style>\n",
3383-
"<table border=\"1\" class=\"dataframe\">\n",
3384-
" <thead>\n",
3385-
" <tr style=\"text-align: right;\">\n",
3386-
" <th></th>\n",
3387-
" <th>Open</th>\n",
3388-
" <th>High</th>\n",
3389-
" <th>Low</th>\n",
3390-
" <th>Close</th>\n",
3391-
" <th>Volume</th>\n",
3392-
" </tr>\n",
3393-
" </thead>\n",
3394-
" <tbody>\n",
3395-
" <tr>\n",
3396-
" <th>2013-02-25</th>\n",
3397-
" <td>802.3</td>\n",
3398-
" <td>808.41</td>\n",
3399-
" <td>790.49</td>\n",
3400-
" <td>790.77</td>\n",
3401-
" <td>2303900</td>\n",
3402-
" </tr>\n",
3403-
" <tr>\n",
3404-
" <th>2013-02-26</th>\n",
3405-
" <td>795.0</td>\n",
3406-
" <td>795.95</td>\n",
3407-
" <td>784.40</td>\n",
3408-
" <td>790.13</td>\n",
3409-
" <td>2202500</td>\n",
3410-
" </tr>\n",
3411-
" <tr>\n",
3412-
" <th>2013-02-27</th>\n",
3413-
" <td>794.8</td>\n",
3414-
" <td>804.75</td>\n",
3415-
" <td>791.11</td>\n",
3416-
" <td>799.78</td>\n",
3417-
" <td>2026100</td>\n",
3418-
" </tr>\n",
3419-
" <tr>\n",
3420-
" <th>2013-02-28</th>\n",
3421-
" <td>801.1</td>\n",
3422-
" <td>806.99</td>\n",
3423-
" <td>801.03</td>\n",
3424-
" <td>801.20</td>\n",
3425-
" <td>2265800</td>\n",
3426-
" </tr>\n",
3427-
" <tr>\n",
3428-
" <th>2013-03-01</th>\n",
3429-
" <td>797.8</td>\n",
3430-
" <td>807.14</td>\n",
3431-
" <td>796.15</td>\n",
3432-
" <td>806.19</td>\n",
3433-
" <td>2175400</td>\n",
3434-
" </tr>\n",
3435-
" </tbody>\n",
3436-
"</table>\n",
3437-
"</div>"
3438-
],
3439-
"text/plain": [
3440-
" Open High Low Close Volume\n",
3441-
"2013-02-25 802.3 808.41 790.49 790.77 2303900\n",
3442-
"2013-02-26 795.0 795.95 784.40 790.13 2202500\n",
3443-
"2013-02-27 794.8 804.75 791.11 799.78 2026100\n",
3444-
"2013-02-28 801.1 806.99 801.03 801.20 2265800\n",
3445-
"2013-03-01 797.8 807.14 796.15 806.19 2175400"
3446-
]
3447-
},
3448-
"execution_count": 1,
3449-
"metadata": {},
3450-
"output_type": "execute_result"
3367+
"name": "stdout",
3368+
"output_type": "stream",
3369+
"text": [
3370+
" Open High Low Close Volume\n",
3371+
"2013-02-25 802.3 808.41 790.49 790.77 2303900\n",
3372+
"2013-02-26 795.0 795.95 784.40 790.13 2202500\n",
3373+
"2013-02-27 794.8 804.75 791.11 799.78 2026100\n",
3374+
"2013-02-28 801.1 806.99 801.03 801.20 2265800\n",
3375+
"2013-03-01 797.8 807.14 796.15 806.19 2175400\n"
3376+
]
34513377
}
34523378
],
34533379
"source": [
34543380
"from backtesting.test import GOOG\n",
34553381
"\n",
3456-
"GOOG.tail()"
3382+
"print(GOOG.tail())"
34573383
]
34583384
},
34593385
{

‎docs/Chapter5/spark.html

Lines changed: 72 additions & 9179 deletions
Large diffs are not rendered by default.

‎docs/Chapter5/testing.html

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -234,6 +234,7 @@
234234
<li class="toctree-l2"><a class="reference internal" href="../Chapter2/dataclasses.html">3.7. Data Classes</a></li>
235235
<li class="toctree-l2"><a class="reference internal" href="../Chapter2/typing.html">3.8. Typing</a></li>
236236
<li class="toctree-l2"><a class="reference internal" href="../Chapter2/pathlib.html">3.9. pathlib</a></li>
237+
<li class="toctree-l2"><a class="reference internal" href="../Chapter2/pydantic.html">3.10. Pydantic</a></li>
237238
</ul>
238239
</li>
239240
<li class="toctree-l1 has-children"><a class="reference internal" href="../Chapter3/Chapter3.html">4. Pandas</a><input class="toctree-checkbox" id="toctree-checkbox-4" name="toctree-checkbox-4" type="checkbox"/><label class="toctree-toggle" for="toctree-checkbox-4"><i class="fa-solid fa-chevron-down"></i></label><ul>
@@ -2906,6 +2907,7 @@ <h2><span class="section-number">6.13.31. </span>pytest-postgresql: Incorporate
29062907
</div>
29072908
<p>If you want to incorporate database testing seamlessly within your pytest test suite, use pytest-postgresql.</p>
29082909
<p>pytest-postgres provides fixtures that manage the setup and cleanup of test databases, ensuring repeatable tests. Additionally, each test runs in isolation, preventing any impact on the production database from testing changes.</p>
2910+
<p>To see how pytest-postgres works, let’s create a test function that sets up a test table in a PostgreSQL database, inserts some test data, and then verifies that the query results match the expected data.</p>
29092911
<div class="cell docutils container">
29102912
<div class="cell_input docutils container">
29112913
<div class="highlight-ipython3 notranslate"><div class="highlight"><pre><span></span><span class="o">%%writefile</span> test_postgres.py

‎docs/Chapter5/time_series.html

Lines changed: 10 additions & 70 deletions
Original file line numberDiff line numberDiff line change
@@ -2573,80 +2573,20 @@ <h2><span class="section-number">6.7.21. </span>Backtesting: Assess Trading Stra
25732573
<div class="cell_input docutils container">
25742574
<div class="highlight-ipython3 notranslate"><div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">backtesting.test</span> <span class="kn">import</span> <span class="n">GOOG</span>
25752575

2576-
<span class="n">GOOG</span><span class="o">.</span><span class="n">tail</span><span class="p">()</span>
2576+
<span class="nb">print</span><span class="p">(</span><span class="n">GOOG</span><span class="o">.</span><span class="n">tail</span><span class="p">())</span>
25772577
</pre></div>
25782578
</div>
25792579
</div>
25802580
<div class="cell_output docutils container">
2581-
<div class="output text_html"><div>
2582-
<style scoped>
2583-
.dataframe tbody tr th:only-of-type {
2584-
vertical-align: middle;
2585-
}
2586-
2587-
.dataframe tbody tr th {
2588-
vertical-align: top;
2589-
}
2590-
2591-
.dataframe thead th {
2592-
text-align: right;
2593-
}
2594-
</style>
2595-
<table border="1" class="dataframe">
2596-
<thead>
2597-
<tr style="text-align: right;">
2598-
<th></th>
2599-
<th>Open</th>
2600-
<th>High</th>
2601-
<th>Low</th>
2602-
<th>Close</th>
2603-
<th>Volume</th>
2604-
</tr>
2605-
</thead>
2606-
<tbody>
2607-
<tr>
2608-
<th>2013-02-25</th>
2609-
<td>802.3</td>
2610-
<td>808.41</td>
2611-
<td>790.49</td>
2612-
<td>790.77</td>
2613-
<td>2303900</td>
2614-
</tr>
2615-
<tr>
2616-
<th>2013-02-26</th>
2617-
<td>795.0</td>
2618-
<td>795.95</td>
2619-
<td>784.40</td>
2620-
<td>790.13</td>
2621-
<td>2202500</td>
2622-
</tr>
2623-
<tr>
2624-
<th>2013-02-27</th>
2625-
<td>794.8</td>
2626-
<td>804.75</td>
2627-
<td>791.11</td>
2628-
<td>799.78</td>
2629-
<td>2026100</td>
2630-
</tr>
2631-
<tr>
2632-
<th>2013-02-28</th>
2633-
<td>801.1</td>
2634-
<td>806.99</td>
2635-
<td>801.03</td>
2636-
<td>801.20</td>
2637-
<td>2265800</td>
2638-
</tr>
2639-
<tr>
2640-
<th>2013-03-01</th>
2641-
<td>797.8</td>
2642-
<td>807.14</td>
2643-
<td>796.15</td>
2644-
<td>806.19</td>
2645-
<td>2175400</td>
2646-
</tr>
2647-
</tbody>
2648-
</table>
2649-
</div></div></div>
2581+
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> Open High Low Close Volume
2582+
2013-02-25 802.3 808.41 790.49 790.77 2303900
2583+
2013-02-26 795.0 795.95 784.40 790.13 2202500
2584+
2013-02-27 794.8 804.75 791.11 799.78 2026100
2585+
2013-02-28 801.1 806.99 801.03 801.20 2265800
2586+
2013-03-01 797.8 807.14 796.15 806.19 2175400
2587+
</pre></div>
2588+
</div>
2589+
</div>
26502590
</div>
26512591
<div class="cell docutils container">
26522592
<div class="cell_input docutils container">

‎docs/_images/shuffle_join_4.png

34.9 KB
Loading

0 commit comments

Comments
 (0)