Skip to content

Commit 692b45d

Browse files
z7yez7ye
authored
huggingface userguide (#84)
2 parents ffc2336 + b050ac1 commit 692b45d

File tree

6 files changed

+286
-64
lines changed

6 files changed

+286
-64
lines changed

‎docs/source/ads.model.framework.rst

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -60,6 +60,14 @@ ads.model.framework.xgboost\_model module
6060
:undoc-members:
6161
:show-inheritance:
6262

63+
ads.model.framework.huggingface\_model module
64+
-----------------------------------------
65+
66+
.. automodule:: ads.model.framework.huggingface_model
67+
:members:
68+
:undoc-members:
69+
:show-inheritance:
70+
6371
Module contents
6472
---------------
6573

‎docs/source/user_guide/model_registration/framework_specific_instruction.rst

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,7 @@
99
frameworks/sparkpipelinemodel
1010
frameworks/lightgbmmodel
1111
frameworks/xgboostmodel
12+
frameworks/huggingfacemodel
1213
frameworks/automlmodel
1314
frameworks/genericmodel
14-
15+

‎docs/source/user_guide/model_registration/frameworks/huggingfacemodel.rst

Lines changed: 242 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,242 @@
1+
.. HuggingFacePipelineModel:
2+
3+
HuggingFacePipelineModel
4+
************************
5+
6+
.. versionadded:: 2.8.2
7+
8+
See `API Documentation <../../../ads.model_framework.html#ads.model.framework.huggingface_model.HuggingFacePipelineModel>`__
9+
10+
Overview
11+
========
12+
13+
The ``ads.model.framework.huggingface_model.HuggingFacePipelineModel`` class in ADS is designed to allow you to rapidly get a HuggingFace pipelines into production. The ``.prepare()`` method creates the model artifacts that are needed to deploy a functioning pipeline without you having to configure it or write code. However, you can customize the required ``score.py`` file.
14+
15+
.. include:: ../_template/overview.rst
16+
17+
The following steps take your trained ``HuggingFacePipelineModel`` model and deploy it into production with a few lines of code.
18+
19+
**Create a HuggingFace Pipeline**
20+
21+
Load a `ImageSegmentationPipeline <https://huggingface.co/docs/transformers/main_classes/pipelines#transformers.ImageSegmentationPipeline>`_ pretrained model.
22+
23+
.. code-block:: python3
24+
25+
>>> from transformers import pipeline
26+
27+
>>> segmenter = pipeline(task="image-segmentation", model="facebook/detr-resnet-50-panoptic", revision="fc15262")
28+
>>> preds = segmenter(
29+
... "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/pipeline-cat-chonk.jpeg"
30+
... )
31+
>>> preds
32+
[{'score': 0.987885,
33+
'label': 'LABEL_184',
34+
'mask': <PIL.Image.Image image mode=L size=960x686>},
35+
{'score': 0.997345,
36+
'label': 'snow',
37+
'mask': <PIL.Image.Image image mode=L size=960x686>},
38+
{'score': 0.997247,
39+
'label': 'cat',
40+
'mask': <PIL.Image.Image image mode=L size=960x686>}]
41+
42+
Prepare Model Artifact
43+
======================
44+
45+
.. code-block:: python3
46+
47+
>>> from ads.common.model_metadata import UseCaseType
48+
>>> from ads.model import HuggingFacePipelineModel
49+
50+
>>> import tempfile
51+
52+
>>> # Prepare the model
53+
>>> artifact_dir = "huggingface_pipeline_model_artifact"
54+
>>> huggingface_pipeline_model = HuggingFacePipelineModel(model, artifact_dir=artifact_dir)
55+
>>> huggingface_pipeline_model.prepare(
56+
... inference_conda_env="<your-conda-pack-path>",
57+
... inference_python_version="<your-python-version>",
58+
... training_conda_env="<your-conda-pack-path>",
59+
... use_case_type=UseCaseType.OTHER,
60+
... force_overwrite=True,
61+
...)
62+
# You don't need to modify the score.py generated. The model can be loaded by the transformers.pipeline.
63+
# More info here - https://huggingface.co/docs/transformers/main_classes/pipelines#transformers.pipeline
64+
65+
66+
Instantiate a ``HuggingFacePipelineModel()`` object with HuggingFace pipelines. All the pipelines related files are saved under the ``artifact_dir``.
67+
68+
For more detailed information on what parameters that ``HuggingFacePipelineModel`` takes, refer to the `API Documentation <../../../ads.model_framework.html#ads.model.framework.huggingface_model.HuggingFacePipelineModel>`__
69+
70+
71+
72+
Summary Status
73+
==============
74+
75+
.. include:: ../_template/summary_status.rst
76+
77+
.. figure:: ../figures/summary_status.png
78+
:align: center
79+
80+
Register Model
81+
==============
82+
83+
.. code-block:: python3
84+
85+
>>> # Register the model
86+
>>> model_id = huggingface_pipeline_model.save()
87+
88+
Model is successfully loaded.
89+
['.model-ignore', 'score.py', 'config.json', 'runtime.yaml', 'preprocessor_config.json', 'pytorch_model.bin']
90+
91+
'ocid1.datasciencemodel.oc1.xxx.xxxxx'
92+
93+
Deploy and Generate Endpoint
94+
============================
95+
96+
.. code-block:: python3
97+
98+
>>> # Deploy and create an endpoint for the huggingface_pipeline_model
99+
>>> huggingface_pipeline_model.deploy(
100+
... display_name="HuggingFace Pipeline Model For Image Segmentation",
101+
... deployment_log_group_id="ocid1.loggroup.oc1.xxx.xxxxx",
102+
... deployment_access_log_id="ocid1.log.oc1.xxx.xxxxx",
103+
... deployment_predict_log_id="ocid1.log.oc1.xxx.xxxxx",
104+
... )
105+
>>> print(f"Endpoint: {huggingface_pipeline_model.model_deployment.url}")
106+
107+
Run Prediction against Endpoint
108+
===============================
109+
110+
.. code-block:: python3
111+
112+
>>> # Download an image
113+
>>> import PIL.Image
114+
>>> import requests
115+
>>> import cloudpickle
116+
>>> image_url = "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/pipeline-cat-chonk.jpeg"
117+
>>> image = PIL.Image.open(requests.get(image_url, stream=True).raw)
118+
119+
>>> # Generate prediction by invoking the deployed endpoint
120+
>>> preds = huggingface_pipeline_model.predict(image)["prediction"]
121+
>>> print([{"score": round(pred["score"], 4), "label": pred["label"]} for pred in preds['prediction']])
122+
[{'score': 0.9879, 'label': 'LABEL_184'},
123+
{'score': 0.9973, 'label': 'snow'},
124+
{'score': 0.9972, 'label': 'cat'}]
125+
126+
127+
Predict with Image
128+
------------------
129+
130+
Predict Image by passing a PIL.Image.Image object using the ``data`` argument in ``predict()`` to predict a single image.
131+
The image will be converted to bytes using cloudpickle so it can be passed to the endpoint.
132+
It will be loaded back to PIL.Image.Image in ``score.py`` before pass into the pipeline.
133+
134+
.. note::
135+
- The payload size limit is 10 MB. Read more about invoking a model deployment `here <https://docs.oracle.com/iaas/data-science/using/model-dep-invoke.htm#model_dep_invoke>`_.
136+
- Model deployment currently does not support internet(coming soon), hence you cannot pass in a url.
137+
138+
Predict with Multiple Arguments
139+
-------------------------------
140+
141+
If your model takes more than one argument, you can pass in through dictionary with the keys as the argument name and values as the value of the arguement.
142+
143+
.. code-block:: python3
144+
145+
>>> your_huggingface_pipeline_model.verify({"parameter_name_1": "parameter_value_1", ..., "parameter_name_n": "parameter_value_n"})
146+
>>> your_huggingface_pipeline_model.predict({"parameter_name_1": "parameter_value_1", ..., "parameter_name_n": "parameter_value_n"})
147+
148+
149+
Run Prediction with oci sdk
150+
==============================
151+
152+
Model deployment endpoints can be invoked with the oci sdk. This example invokes a model deployment with the oci sdk with a ``bytes`` payload:
153+
154+
``bytes`` payload example
155+
------------------------------
156+
157+
.. code-block:: python3
158+
159+
>>> # The OCI SDK must be installed for this example to function properly.
160+
>>> # Installation instructions can be found here: https://docs.oracle.com/en-us/iaas/Content/API/SDKDocs/pythonsdk.htm
161+
162+
>>> import requests
163+
>>> import oci
164+
>>> from ads.common.auth import default_signer
165+
>>> import cloudpickle
166+
>>> import PIL.Image
167+
>>> import cloudpickle
168+
>>> headers = {"Content-Type": "application/octet-stream"}
169+
>>> endpoint = huggingface_pipeline_model.model_deployment.url + "/predict"
170+
171+
>>> ## download the image
172+
>>> image_url = "https://huggingface.co/datasets/Narsil/image_dummy/raw/main/parrots.png"
173+
>>> image = PIL.Image.open(requests.get(image_link, stream=True).raw)
174+
>>> image_bytes = cloudpickle.dumps(image)
175+
176+
>>> preds = requests.post(endpoint, data=image_bytes, auth=default_signer()['signer'], headers=headers).json()
177+
>>> print([{"score": round(pred["score"], 4), "label": pred["label"]} for pred in preds['prediction']])
178+
[{'score': 0.9879, 'label': 'LABEL_184'},
179+
{'score': 0.9973, 'label': 'snow'},
180+
{'score': 0.9972, 'label': 'cat'}]
181+
182+
183+
Example
184+
=======
185+
186+
.. code-block:: python3
187+
188+
from transformers import pipeline
189+
from ads.model import HuggingFacePipelineModel
190+
191+
import tempfile
192+
import PIL.Image
193+
from ads.common.auth import default_signer
194+
import requests
195+
import cloudpickle
196+
197+
## download the image
198+
image_url = "https://huggingface.co/datasets/Narsil/image_dummy/raw/main/parrots.png"
199+
image = PIL.Image.open(requests.get(image_link, stream=True).raw)
200+
201+
## download the pretrained model
202+
classifier = pipeline(model="openai/clip-vit-large-patch14")
203+
classifier(
204+
images=image,
205+
candidate_labels=["animals", "humans", "landscape"],
206+
)
207+
208+
## Initiate a HuggingFacePipelineModel instance
209+
zero_shot_image_classification_model = HuggingFacePipelineModel(classifier, artifact_dir=empfile.mkdtemp())
210+
211+
# Autogenerate score.py, serialized model, runtime.yaml
212+
conda_pack_path = "oci://bucket@namespace/path/to/conda/pack"
213+
python_version = "3.x" # Remember to update 3.x with your actual python version, e.g. 3.8
214+
zero_shot_image_classification_model.prepare(inference_conda_env=conda_pack_path, inference_python_version = python_version, force_overwrite=True)
215+
216+
## Convert payload to bytes
217+
data = {"images": image, "candidate_labels": ["animals", "humans", "landscape"]}
218+
body = cloudpickle.dumps(data) # convert image to bytes
219+
220+
# Verify generated artifacts
221+
zero_shot_image_classification_model.verify(data=data)
222+
zero_shot_image_classification_model.verify(data=body)
223+
224+
# Register HuggingFace Pipeline model
225+
zero_shot_image_classification_model.save()
226+
227+
## Deploy
228+
log_group_id = "<log_group_id>"
229+
log_id = "<log_id>"
230+
zero_shot_image_classification_model.deploy(deployment_bandwidth_mbps=100,
231+
wait_for_completion=False,
232+
deployment_log_group_id = log_group_id,
233+
deployment_access_log_id = log_id,
234+
deployment_predict_log_id = log_id)
235+
zero_shot_image_classification_model.predict(image)
236+
zero_shot_image_classification_model.predict(body)
237+
238+
### Invoke the model by sending bytes
239+
auth = default_signer()['signer']
240+
endpoint = zero_shot_image_classification_model.model_deployment.url + "/predict"
241+
headers = {"Content-Type": "application/octet-stream"}
242+
requests.post(endpoint, data=body, auth=auth, headers=headers).json()

‎docs/source/user_guide/model_registration/frameworks/pytorchmodel.rst

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -215,7 +215,7 @@ Deploy and Generate Endpoint
215215

216216
.. code-block:: python3
217217
218-
>>> # Deploy and create an endpoint for the TensorFlow model
218+
>>> # Deploy and create an endpoint for the PyTorch model
219219
>>> pytorch_model.deploy(
220220
display_name="PyTorch Model For Classification",
221221
deployment_log_group_id="ocid1.loggroup.oc1.xxx.xxxxx",

0 commit comments

Comments
 (0)