Skip to content

Commit dde22e0

Browse files
KshitizLohiaKshitizLohia
committed
added release notes
1 parent 4328335 commit dde22e0

File tree

5 files changed

+330
-5
lines changed

5 files changed

+330
-5
lines changed

‎ads/feature_store/docs/source/conf.py

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@
99

1010
sys.path.insert(0, os.path.abspath("../../"))
1111

12-
version = "1.0"
12+
version = "1.2"
1313
release = version
1414

1515

@@ -54,7 +54,7 @@
5454

5555
# Get version
5656

57-
version = "1.0"
57+
version = "1.2"
5858
release = version
5959

6060
# Unless we want to expose real buckets and namespaces

‎ads/feature_store/docs/source/feature_validation.rst

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
.. _Feature Validation:
22

33
Feature Validation
4-
*************
4+
******************
55

66
Feature validation is the process of checking the quality and accuracy of the features used in a machine learning model. This is important because features that are not accurate or reliable can lead to poor model performance.
77
Feature store allows you to define expectation on the data which is being materialized into feature group and dataset. This is achieved using open source library Great Expectations.
@@ -43,7 +43,7 @@ An Expectation is a verifiable assertion about your data. You can define expecta
4343
)
4444
4545
Expectations Suite
46-
============
46+
===================
4747

4848
Expectation Suite is a collection of verifiable assertions i.e. expectations about your data. You can define expectation suite as below:
4949

‎ads/feature_store/docs/source/release_notes.rst

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,35 @@
33
=============
44
Release Notes
55
=============
6+
1.2
7+
---
8+
.. note::
9+
10+
.. list-table::
11+
:header-rows: 1
12+
13+
* - Package Name
14+
- Latest Version
15+
- Notes
16+
* - Conda pack
17+
- `https://objectstorage.us-ashburn-1.oraclecloud.com/n/bigdatadatasciencelarge/b/service-conda-packs-fs/o/service_pack/cpu/PySpark_3.2_and_Feature_Store/1.0/fspyspark32_p38_cpu_v1#conda`
18+
-
19+
* - SERVICE_VERSION
20+
- 0.1.218.master
21+
-
22+
* - Terraform Stack
23+
- `link <https://objectstorage.us-ashburn-1.oraclecloud.com/p/vZogtXWwHqbkGLeqyKiqBmVxdbR4MK4nyOBqDsJNVE4sHGUY5KFi4T3mOFGA3FOy/n/idogsu2ylimg/b/oci-feature-store/o/beta/terraform/feature-store-terraform.zip>`__
24+
- Par link expires Jan 5, 2026
25+
26+
Release notes: July 21, 2023
27+
28+
* [FEATURE] Supporting for deployment in ``us-ashburn`` and ``uk-london`` region.
29+
* [FEATURE] For ``ValidationOutput`` instance, addition of ``to_summary()`` method for validation summary details.
30+
* [FEATURE] For ``ValidationOutput`` instance, addition of ``to_pandas()`` method for validation detailed report.
31+
* [FIX] Fixed unit test integration to support the merging of ADS into the main branch.
32+
* [DOCS] For ``ValidationOutput`` instance, addition of ``to_summary()`` method for validation summary details.
33+
* [DOCS] For ``ValidationOutput`` instance, addition of ``to_pandas()`` method for validation detailed report.
34+
635
1.1
736
---
837

‎ads/feature_store/docs/source/terraform.rst

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -90,6 +90,7 @@ Steps
9090
Refer :ref:`Release Notes` for getting the latest conda pack and ``SERVICE_VERSION``. Remember to replace the values within angle brackets ("<>" symbols) in the command above with the relevant values for your environment. Also, Refer :ref:`User Policies` to create feature store stack for non admin users. No policies are explicitly required for admin user.
9191

9292
1. Run the shell command.
93+
9394
.. code-block:: shell
9495
9596
rm -f feature-store-terraform.zip \
@@ -116,7 +117,7 @@ Steps
116117
--wait-for-state FAILED
117118
118119
Update Feature Store Stack with the Latest using OCI CLI
119-
==============================
120+
========================================================
120121

121122
Prerequisites
122123
#############
@@ -139,6 +140,7 @@ Steps
139140
Refer :ref:`Release Notes` for getting the latest conda pack and ``SERVICE_VERSION``. Remember to replace the values within angle brackets ("<>" symbols) in the command above with the relevant values for your environment. Also, Refer :ref:`User Policies` to create feature store stack for non admin users. No policies are explicitly required for admin user.
140141

141142
1. Run the shell command.
143+
142144
.. code-block:: shell
143145
144146
rm -f feature-store-terraform.zip \

‎ads/feature_store/test.ipynb

Lines changed: 294 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,294 @@
1+
{
2+
"cells": [
3+
{
4+
"cell_type": "code",
5+
"execution_count": 1,
6+
"metadata": {
7+
"collapsed": true,
8+
"ExecuteTime": {
9+
"end_time": "2023-07-07T06:46:31.889241Z",
10+
"start_time": "2023-07-07T06:46:23.997958Z"
11+
}
12+
},
13+
"outputs": [
14+
{
15+
"name": "stdout",
16+
"output_type": "stream",
17+
"text": [
18+
" name age address.street address.city address.state \\\n",
19+
"0 John 30 123 Main St New York NY \n",
20+
"\n",
21+
" address.contacts street city \\\n",
22+
"0 [{'type': 'email', 'value': 'john@example.com'... 123 Main St New York \n",
23+
"\n",
24+
" state contacts \n",
25+
"0 NY [{'type': 'email', 'value': 'john@example.com'... \n"
26+
]
27+
}
28+
],
29+
"source": [
30+
"import pandas as pd\n",
31+
"import json\n",
32+
"\n",
33+
"data = {\n",
34+
" \"name\": \"John\",\n",
35+
" \"age\": 30,\n",
36+
" \"address\": {\n",
37+
" \"street\": \"123 Main St\",\n",
38+
" \"city\": \"New York\",\n",
39+
" \"state\": \"NY\",\n",
40+
" \"contacts\": [\n",
41+
" {\n",
42+
" \"type\": \"email\",\n",
43+
" \"value\": \"john@example.com\"\n",
44+
" },\n",
45+
" {\n",
46+
" \"type\": \"phone\",\n",
47+
" \"value\": \"123-456-7890\"\n",
48+
" }\n",
49+
" ]\n",
50+
" }\n",
51+
"}\n",
52+
"\n",
53+
"# Flatten the outer level\n",
54+
"df_outer = pd.json_normalize(data)\n",
55+
"\n",
56+
"# Flatten the \"address\" subtable\n",
57+
"df_address = pd.json_normalize(data['address'], max_level=1)\n",
58+
"\n",
59+
"# Combine the outer and subtable dataframes\n",
60+
"df_combined = pd.concat([df_outer, df_address], axis=1)\n",
61+
"\n",
62+
"print(df_combined)\n"
63+
]
64+
},
65+
{
66+
"cell_type": "code",
67+
"execution_count": 2,
68+
"outputs": [
69+
{
70+
"data": {
71+
"text/plain": " name age address.street address.city address.state \\\n0 John 30 123 Main St New York NY \n\n address.contacts street city \\\n0 [{'type': 'email', 'value': 'john@example.com'... 123 Main St New York \n\n state contacts \n0 NY [{'type': 'email', 'value': 'john@example.com'... ",
72+
"text/html": "<div>\n<style scoped>\n .dataframe tbody tr th:only-of-type {\n vertical-align: middle;\n }\n\n .dataframe tbody tr th {\n vertical-align: top;\n }\n\n .dataframe thead th {\n text-align: right;\n }\n</style>\n<table border=\"1\" class=\"dataframe\">\n <thead>\n <tr style=\"text-align: right;\">\n <th></th>\n <th>name</th>\n <th>age</th>\n <th>address.street</th>\n <th>address.city</th>\n <th>address.state</th>\n <th>address.contacts</th>\n <th>street</th>\n <th>city</th>\n <th>state</th>\n <th>contacts</th>\n </tr>\n </thead>\n <tbody>\n <tr>\n <th>0</th>\n <td>John</td>\n <td>30</td>\n <td>123 Main St</td>\n <td>New York</td>\n <td>NY</td>\n <td>[{'type': 'email', 'value': 'john@example.com'...</td>\n <td>123 Main St</td>\n <td>New York</td>\n <td>NY</td>\n <td>[{'type': 'email', 'value': 'john@example.com'...</td>\n </tr>\n </tbody>\n</table>\n</div>"
73+
},
74+
"execution_count": 2,
75+
"metadata": {},
76+
"output_type": "execute_result"
77+
}
78+
],
79+
"source": [
80+
"df_combined"
81+
],
82+
"metadata": {
83+
"collapsed": false,
84+
"ExecuteTime": {
85+
"end_time": "2023-07-07T06:46:38.652495Z",
86+
"start_time": "2023-07-07T06:46:38.619414Z"
87+
}
88+
}
89+
},
90+
{
91+
"cell_type": "code",
92+
"execution_count": 3,
93+
"outputs": [
94+
{
95+
"name": "stdout",
96+
"output_type": "stream",
97+
"text": [
98+
" name age address.street address.city address.state \\\n",
99+
"0 John 30 123 Main St New York NY \n",
100+
"\n",
101+
" address.contacts \n",
102+
"0 [{\"type\": \"email\", \"value\": \"john@example.com\"... \n"
103+
]
104+
}
105+
],
106+
"source": [
107+
"import pandas as pd\n",
108+
"import json\n",
109+
"\n",
110+
"data = {\n",
111+
" \"name\": \"John\",\n",
112+
" \"age\": 30,\n",
113+
" \"address\": {\n",
114+
" \"street\": \"123 Main St\",\n",
115+
" \"city\": \"New York\",\n",
116+
" \"state\": \"NY\",\n",
117+
" \"contacts\": [\n",
118+
" {\n",
119+
" \"type\": \"email\",\n",
120+
" \"value\": \"john@example.com\"\n",
121+
" },\n",
122+
" {\n",
123+
" \"type\": \"phone\",\n",
124+
" \"value\": \"123-456-7890\"\n",
125+
" }\n",
126+
" ]\n",
127+
" }\n",
128+
"}\n",
129+
"\n",
130+
"df = pd.DataFrame()\n",
131+
"for key, value in data.items():\n",
132+
" if isinstance(value, dict):\n",
133+
" for subkey, subvalue in value.items():\n",
134+
" if isinstance(subvalue, list):\n",
135+
" df[f'{key}.{subkey}'] = [json.dumps(subvalue)]\n",
136+
" else:\n",
137+
" df[f'{key}.{subkey}'] = [subvalue]\n",
138+
" else:\n",
139+
" df[key] = [value]\n",
140+
"\n",
141+
"print(df)\n"
142+
],
143+
"metadata": {
144+
"collapsed": false,
145+
"ExecuteTime": {
146+
"end_time": "2023-07-07T06:48:36.982219Z",
147+
"start_time": "2023-07-07T06:48:36.965884Z"
148+
}
149+
}
150+
},
151+
{
152+
"cell_type": "code",
153+
"execution_count": 4,
154+
"outputs": [
155+
{
156+
"data": {
157+
"text/plain": " name age address.street address.city address.state \\\n0 John 30 123 Main St New York NY \n\n address.contacts \n0 [{\"type\": \"email\", \"value\": \"john@example.com\"... ",
158+
"text/html": "<div>\n<style scoped>\n .dataframe tbody tr th:only-of-type {\n vertical-align: middle;\n }\n\n .dataframe tbody tr th {\n vertical-align: top;\n }\n\n .dataframe thead th {\n text-align: right;\n }\n</style>\n<table border=\"1\" class=\"dataframe\">\n <thead>\n <tr style=\"text-align: right;\">\n <th></th>\n <th>name</th>\n <th>age</th>\n <th>address.street</th>\n <th>address.city</th>\n <th>address.state</th>\n <th>address.contacts</th>\n </tr>\n </thead>\n <tbody>\n <tr>\n <th>0</th>\n <td>John</td>\n <td>30</td>\n <td>123 Main St</td>\n <td>New York</td>\n <td>NY</td>\n <td>[{\"type\": \"email\", \"value\": \"john@example.com\"...</td>\n </tr>\n </tbody>\n</table>\n</div>"
159+
},
160+
"execution_count": 4,
161+
"metadata": {},
162+
"output_type": "execute_result"
163+
}
164+
],
165+
"source": [
166+
"df"
167+
],
168+
"metadata": {
169+
"collapsed": false,
170+
"ExecuteTime": {
171+
"end_time": "2023-07-07T06:48:40.293858Z",
172+
"start_time": "2023-07-07T06:48:40.278669Z"
173+
}
174+
}
175+
},
176+
{
177+
"cell_type": "code",
178+
"execution_count": 5,
179+
"outputs": [
180+
{
181+
"ename": "ValueError",
182+
"evalue": "DataFrame constructor not properly called!",
183+
"output_type": "error",
184+
"traceback": [
185+
"\u001B[0;31m---------------------------------------------------------------------------\u001B[0m",
186+
"\u001B[0;31mValueError\u001B[0m Traceback (most recent call last)",
187+
"\u001B[0;32m/var/folders/pg/zfq6crsd7kdd3qw8txc6c1h00000gn/T/ipykernel_47057/3621665856.py\u001B[0m in \u001B[0;36m?\u001B[0;34m()\u001B[0m\n\u001B[1;32m 23\u001B[0m \u001B[0;34m\u001B[0m\u001B[0m\n\u001B[1;32m 24\u001B[0m \u001B[0mdf_outer\u001B[0m \u001B[0;34m=\u001B[0m \u001B[0mpd\u001B[0m\u001B[0;34m.\u001B[0m\u001B[0mDataFrame\u001B[0m\u001B[0;34m(\u001B[0m\u001B[0mdata\u001B[0m\u001B[0;34m,\u001B[0m \u001B[0mindex\u001B[0m\u001B[0;34m=\u001B[0m\u001B[0;34m[\u001B[0m\u001B[0;36m0\u001B[0m\u001B[0;34m]\u001B[0m\u001B[0;34m)\u001B[0m\u001B[0;34m\u001B[0m\u001B[0;34m\u001B[0m\u001B[0m\n\u001B[1;32m 25\u001B[0m \u001B[0;34m\u001B[0m\u001B[0m\n\u001B[1;32m 26\u001B[0m \u001B[0;31m# Extract the subtable as a separate dataframe\u001B[0m\u001B[0;34m\u001B[0m\u001B[0;34m\u001B[0m\u001B[0m\n\u001B[0;32m---> 27\u001B[0;31m \u001B[0mdf_subtable\u001B[0m \u001B[0;34m=\u001B[0m \u001B[0mpd\u001B[0m\u001B[0;34m.\u001B[0m\u001B[0mDataFrame\u001B[0m\u001B[0;34m(\u001B[0m\u001B[0mdf_outer\u001B[0m\u001B[0;34m[\u001B[0m\u001B[0;34m'address'\u001B[0m\u001B[0;34m]\u001B[0m\u001B[0;34m.\u001B[0m\u001B[0mvalues\u001B[0m\u001B[0;34m[\u001B[0m\u001B[0;36m0\u001B[0m\u001B[0;34m]\u001B[0m\u001B[0;34m)\u001B[0m\u001B[0;34m\u001B[0m\u001B[0;34m\u001B[0m\u001B[0m\n\u001B[0m\u001B[1;32m 28\u001B[0m \u001B[0;34m\u001B[0m\u001B[0m\n\u001B[1;32m 29\u001B[0m \u001B[0;31m# Remove the subtable column from the main dataframe\u001B[0m\u001B[0;34m\u001B[0m\u001B[0;34m\u001B[0m\u001B[0m\n\u001B[1;32m 30\u001B[0m \u001B[0mdf_outer\u001B[0m\u001B[0;34m.\u001B[0m\u001B[0mdrop\u001B[0m\u001B[0;34m(\u001B[0m\u001B[0;34m'address'\u001B[0m\u001B[0;34m,\u001B[0m \u001B[0maxis\u001B[0m\u001B[0;34m=\u001B[0m\u001B[0;36m1\u001B[0m\u001B[0;34m,\u001B[0m \u001B[0minplace\u001B[0m\u001B[0;34m=\u001B[0m\u001B[0;32mTrue\u001B[0m\u001B[0;34m)\u001B[0m\u001B[0;34m\u001B[0m\u001B[0;34m\u001B[0m\u001B[0m\n",
188+
"\u001B[0;32m~/IdeaProjects/oracle/feature-store/advanced-ds/venv/lib/python3.10/site-packages/pandas/core/frame.py\u001B[0m in \u001B[0;36m?\u001B[0;34m(self, data, index, columns, dtype, copy)\u001B[0m\n\u001B[1;32m 777\u001B[0m )\n\u001B[1;32m 778\u001B[0m \u001B[0;31m# For data is scalar\u001B[0m\u001B[0;34m\u001B[0m\u001B[0;34m\u001B[0m\u001B[0m\n\u001B[1;32m 779\u001B[0m \u001B[0;32melse\u001B[0m\u001B[0;34m:\u001B[0m\u001B[0;34m\u001B[0m\u001B[0;34m\u001B[0m\u001B[0m\n\u001B[1;32m 780\u001B[0m \u001B[0;32mif\u001B[0m \u001B[0mindex\u001B[0m \u001B[0;32mis\u001B[0m \u001B[0;32mNone\u001B[0m \u001B[0;32mor\u001B[0m \u001B[0mcolumns\u001B[0m \u001B[0;32mis\u001B[0m \u001B[0;32mNone\u001B[0m\u001B[0;34m:\u001B[0m\u001B[0;34m\u001B[0m\u001B[0;34m\u001B[0m\u001B[0m\n\u001B[0;32m--> 781\u001B[0;31m \u001B[0;32mraise\u001B[0m \u001B[0mValueError\u001B[0m\u001B[0;34m(\u001B[0m\u001B[0;34m\"DataFrame constructor not properly called!\"\u001B[0m\u001B[0;34m)\u001B[0m\u001B[0;34m\u001B[0m\u001B[0;34m\u001B[0m\u001B[0m\n\u001B[0m\u001B[1;32m 782\u001B[0m \u001B[0;34m\u001B[0m\u001B[0m\n\u001B[1;32m 783\u001B[0m \u001B[0mindex\u001B[0m \u001B[0;34m=\u001B[0m \u001B[0mensure_index\u001B[0m\u001B[0;34m(\u001B[0m\u001B[0mindex\u001B[0m\u001B[0;34m)\u001B[0m\u001B[0;34m\u001B[0m\u001B[0;34m\u001B[0m\u001B[0m\n\u001B[1;32m 784\u001B[0m \u001B[0mcolumns\u001B[0m \u001B[0;34m=\u001B[0m \u001B[0mensure_index\u001B[0m\u001B[0;34m(\u001B[0m\u001B[0mcolumns\u001B[0m\u001B[0;34m)\u001B[0m\u001B[0;34m\u001B[0m\u001B[0;34m\u001B[0m\u001B[0m\n",
189+
"\u001B[0;31mValueError\u001B[0m: DataFrame constructor not properly called!"
190+
]
191+
}
192+
],
193+
"source": [
194+
"import pandas as pd\n",
195+
"import json\n",
196+
"\n",
197+
"data = {\n",
198+
" \"name\": \"John\",\n",
199+
" \"age\": 30,\n",
200+
" \"address\": {\n",
201+
" \"street\": \"123 Main St\",\n",
202+
" \"city\": \"New York\",\n",
203+
" \"state\": \"NY\",\n",
204+
" \"contacts\": [\n",
205+
" {\n",
206+
" \"type\": \"email\",\n",
207+
" \"value\": \"john@example.com\"\n",
208+
" },\n",
209+
" {\n",
210+
" \"type\": \"phone\",\n",
211+
" \"value\": \"123-456-7890\"\n",
212+
" }\n",
213+
" ]\n",
214+
" }\n",
215+
"}\n",
216+
"\n",
217+
"df_outer = pd.DataFrame(data, index=[0])\n",
218+
"\n",
219+
"# Extract the subtable as a separate dataframe\n",
220+
"df_subtable = pd.DataFrame(df_outer['address'].values[0])\n",
221+
"\n",
222+
"# Remove the subtable column from the main dataframe\n",
223+
"df_outer.drop('address', axis=1, inplace=True)\n",
224+
"\n",
225+
"# Combine the main dataframe with the subtable dataframe\n",
226+
"df_combined = pd.concat([df_outer, df_subtable], axis=1)\n",
227+
"\n",
228+
"print(df_combined)\n"
229+
],
230+
"metadata": {
231+
"collapsed": false,
232+
"ExecuteTime": {
233+
"end_time": "2023-07-07T06:49:19.205319Z",
234+
"start_time": "2023-07-07T06:49:18.929206Z"
235+
}
236+
}
237+
},
238+
{
239+
"cell_type": "code",
240+
"execution_count": 6,
241+
"outputs": [
242+
{
243+
"data": {
244+
"text/plain": " name age address.street address.city address.state \\\n0 John 30 123 Main St New York NY \n\n address.contacts street city \\\n0 [{'type': 'email', 'value': 'john@example.com'... 123 Main St New York \n\n state contacts \n0 NY [{'type': 'email', 'value': 'john@example.com'... ",
245+
"text/html": "<div>\n<style scoped>\n .dataframe tbody tr th:only-of-type {\n vertical-align: middle;\n }\n\n .dataframe tbody tr th {\n vertical-align: top;\n }\n\n .dataframe thead th {\n text-align: right;\n }\n</style>\n<table border=\"1\" class=\"dataframe\">\n <thead>\n <tr style=\"text-align: right;\">\n <th></th>\n <th>name</th>\n <th>age</th>\n <th>address.street</th>\n <th>address.city</th>\n <th>address.state</th>\n <th>address.contacts</th>\n <th>street</th>\n <th>city</th>\n <th>state</th>\n <th>contacts</th>\n </tr>\n </thead>\n <tbody>\n <tr>\n <th>0</th>\n <td>John</td>\n <td>30</td>\n <td>123 Main St</td>\n <td>New York</td>\n <td>NY</td>\n <td>[{'type': 'email', 'value': 'john@example.com'...</td>\n <td>123 Main St</td>\n <td>New York</td>\n <td>NY</td>\n <td>[{'type': 'email', 'value': 'john@example.com'...</td>\n </tr>\n </tbody>\n</table>\n</div>"
246+
},
247+
"execution_count": 6,
248+
"metadata": {},
249+
"output_type": "execute_result"
250+
}
251+
],
252+
"source": [
253+
"df_combined"
254+
],
255+
"metadata": {
256+
"collapsed": false,
257+
"ExecuteTime": {
258+
"end_time": "2023-07-07T06:49:23.854478Z",
259+
"start_time": "2023-07-07T06:49:23.842264Z"
260+
}
261+
}
262+
},
263+
{
264+
"cell_type": "code",
265+
"execution_count": null,
266+
"outputs": [],
267+
"source": [],
268+
"metadata": {
269+
"collapsed": false
270+
}
271+
}
272+
],
273+
"metadata": {
274+
"kernelspec": {
275+
"display_name": "Python 3",
276+
"language": "python",
277+
"name": "python3"
278+
},
279+
"language_info": {
280+
"codemirror_mode": {
281+
"name": "ipython",
282+
"version": 2
283+
},
284+
"file_extension": ".py",
285+
"mimetype": "text/x-python",
286+
"name": "python",
287+
"nbconvert_exporter": "python",
288+
"pygments_lexer": "ipython2",
289+
"version": "2.7.6"
290+
}
291+
},
292+
"nbformat": 4,
293+
"nbformat_minor": 0
294+
}

0 commit comments

Comments
 (0)