Skip to content

✨ implement support for V2 API #328

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 16 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/_test-code-samples.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ jobs:
max-parallel: 2
matrix:
python-version:
- "3.7"
- "3.8"
- "3.12"
runs-on: "ubuntu-22.04"
steps:
Expand Down Expand Up @@ -40,7 +40,7 @@ jobs:

- name: Tests code samples
run: |
./tests/test_code_samples.sh ${{ secrets.MINDEE_ACCOUNT_SE_TESTS }} ${{ secrets.MINDEE_ENDPOINT_SE_TESTS }} ${{ secrets.MINDEE_API_KEY_SE_TESTS }}
./tests/test_code_samples.sh ${{ secrets.MINDEE_ACCOUNT_SE_TESTS }} ${{ secrets.MINDEE_ENDPOINT_SE_TESTS }} ${{ secrets.MINDEE_API_KEY_SE_TESTS }} ${{ secrets.MINDEE_V2_SE_TESTS_API_KEY }} ${{ secrets.MINDEE_V2_SE_TESTS_FINDOC_MODEL_ID }}

- name: Notify Slack Action on Failure
uses: ravsamhq/notify-slack-action@2.3.0
Expand Down
4 changes: 3 additions & 1 deletion .github/workflows/_test-integrations.yml
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ jobs:
- "ubuntu-22.04"
- "windows-2022"
python-version:
- "3.7"
- "3.8"
- "3.12"
runs-on: ${{ matrix.os }}
steps:
Expand Down Expand Up @@ -47,6 +47,8 @@ jobs:
env:
MINDEE_API_KEY: ${{ secrets.MINDEE_API_KEY_SE_TESTS }}
WORKFLOW_ID: ${{ secrets.WORKFLOW_ID_SE_TESTS }}
MINDEE_V2_API_KEY: ${{ secrets.MINDEE_V2_SE_TESTS_API_KEY }}
MINDEE_V2_FINDOC_MODEL_ID: ${{ secrets.MINDEE_V2_SE_TESTS_FINDOC_MODEL_ID }}
run: |
pytest -m integration

Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/_test-regressions.yml
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ jobs:
- "ubuntu-22.04"
- "windows-2022"
python-version:
- "3.7"
- "3.8"
- "3.12"
runs-on: ${{ matrix.os }}
steps:
Expand Down
1 change: 0 additions & 1 deletion .github/workflows/_test-units.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,6 @@ jobs:
- "ubuntu-22.04"
- "windows-2022"
python-version:
- "3.7"
- "3.8"
- "3.9"
- "3.10"
Expand Down
10 changes: 5 additions & 5 deletions .github/workflows/pull-request.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,20 +5,20 @@

jobs:
static-analysis:
uses: mindee/mindee-api-python/.github/workflows/_static-analysis.yml@main
uses: ./.github/workflows/_static-analysis.yml
test-units:

Check warning

Code scanning / CodeQL

Workflow does not contain permissions Medium

Actions job or workflow does not limit the permissions of the GITHUB_TOKEN. Consider setting an explicit permissions block, using the following as a minimal starting point: {}
uses: mindee/mindee-api-python/.github/workflows/_test-units.yml@main
uses: ./.github/workflows/_test-units.yml
needs: static-analysis
secrets: inherit
test-regressions:

Check warning

Code scanning / CodeQL

Workflow does not contain permissions Medium

Actions job or workflow does not limit the permissions of the GITHUB_TOKEN. Consider setting an explicit permissions block, using the following as a minimal starting point: {}
uses: mindee/mindee-api-python/.github/workflows/_test-regressions.yml@main
uses: ./.github/workflows/_test-regressions.yml
needs: test-units
secrets: inherit
test-integrations:

Check warning

Code scanning / CodeQL

Workflow does not contain permissions Medium

Actions job or workflow does not limit the permissions of the GITHUB_TOKEN. Consider setting an explicit permissions block, using the following as a minimal starting point: {}
uses: mindee/mindee-api-python/.github/workflows/_test-integrations.yml@main
uses: ./.github/workflows/_test-integrations.yml
needs: test-units
secrets: inherit
test-code-samples:

Check warning

Code scanning / CodeQL

Workflow does not contain permissions Medium

Actions job or workflow does not limit the permissions of the GITHUB_TOKEN. Consider setting an explicit permissions block, using the following as a minimal starting point: {}
uses: mindee/mindee-api-python/.github/workflows/_test-code-samples.yml@main
uses: ./.github/workflows/_test-code-samples.yml
needs: test-units
secrets: inherit

Check warning

Code scanning / CodeQL

Workflow does not contain permissions Medium

Actions job or workflow does not limit the permissions of the GITHUB_TOKEN. Consider setting an explicit permissions block, using the following as a minimal starting point: {}
25 changes: 25 additions & 0 deletions docs/extras/code_samples/default_v2.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
#
# Install the Python client library by running:
# pip install mindee
#

from mindee import ClientV2, InferencePredictOptions
from mindee.parsing.v2 import InferenceResponse, PollingResponse
from tests.product import PRODUCT_DATA_DIR

input_path = "/path/to/the/file.ext"
api_key = "MY_API_KEY"
model_id = "MY_MODEL_ID"

# Init a new client
mindee_client = ClientV2(api_key)

# Load a file from disk
input_doc = mindee_client.source_from_path(input_path)
options = InferencePredictOptions(model_id=model_id)

# Parse the file.
response: InferenceResponse = mindee_client.enqueue_and_parse(input_doc, options)

# Print a brief summary of the parsed data
print(response.inference)
2 changes: 2 additions & 0 deletions mindee/__init__.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
from mindee import product
from mindee.client import Client
from mindee.client_v2 import ClientV2
from mindee.input.inference_predict_options import InferencePredictOptions
from mindee.input.local_response import LocalResponse
from mindee.input.page_options import PageOptions
from mindee.parsing.common.api_response import ApiResponse
Expand Down
91 changes: 3 additions & 88 deletions mindee/client.py
Original file line number Diff line number Diff line change
@@ -1,18 +1,14 @@
from pathlib import Path
from time import sleep
from typing import BinaryIO, Dict, Optional, Type, Union
from typing import Dict, Optional, Type, Union

from mindee.client_mixin import ClientMixin
from mindee.error.mindee_error import MindeeClientError, MindeeError
from mindee.error.mindee_http_error import handle_error
from mindee.input import WorkflowOptions
from mindee.input.local_response import LocalResponse
from mindee.input.page_options import PageOptions
from mindee.input.predict_options import AsyncPredictOptions, PredictOptions
from mindee.input.sources.base_64_input import Base64Input
from mindee.input.sources.bytes_input import BytesInput
from mindee.input.sources.file_input import FileInput
from mindee.input.sources.local_input_source import LocalInputSource
from mindee.input.sources.path_input import PathInput
from mindee.input.sources.url_input_source import UrlInputSource
from mindee.logger import logger
from mindee.mindee_http.endpoint import CustomEndpoint, Endpoint
Expand Down Expand Up @@ -55,7 +51,7 @@ def _clean_account_name(account_name: str) -> str:
return account_name


class Client:
class Client(ClientMixin):
"""
Mindee API Client.

Expand Down Expand Up @@ -275,23 +271,6 @@ def execute_workflow(
logger.debug("Sending document to workflow: %s", workflow_id)
return self._send_to_workflow(GeneratedV1, input_source, workflow_id, options)

def _validate_async_params(
self, initial_delay_sec: float, delay_sec: float, max_retries: int
) -> None:
min_delay = 1
min_initial_delay = 1
min_retries = 1
if delay_sec < min_delay:
raise MindeeClientError(
f"Cannot set auto-parsing delay to less than {min_delay} second(s)."
)
if initial_delay_sec < min_initial_delay:
raise MindeeClientError(
f"Cannot set initial parsing delay to less than {min_initial_delay} second(s)."
)
if max_retries < min_retries:
raise MindeeClientError(f"Cannot set retries to less than {min_retries}.")

def enqueue_and_parse( # pylint: disable=too-many-locals
self,
product_class: Type[Inference],
Expand Down Expand Up @@ -584,70 +563,6 @@ def create_endpoint(
version = "1"
return self._build_endpoint(endpoint_name, account_name, version)

@staticmethod
def source_from_path(
input_path: Union[Path, str], fix_pdf: bool = False
) -> PathInput:
"""
Load a document from an absolute path, as a string.

:param input_path: Path of file to open
:param fix_pdf: Whether to attempt fixing PDF files before sending.
Setting this to `True` can modify the data sent to Mindee.
"""
input_doc = PathInput(input_path)
if fix_pdf:
input_doc.fix_pdf()
return input_doc

@staticmethod
def source_from_file(input_file: BinaryIO, fix_pdf: bool = False) -> FileInput:
"""
Load a document from a normal Python file object/handle.

:param input_file: Input file handle
:param fix_pdf: Whether to attempt fixing PDF files before sending.
Setting this to `True` can modify the data sent to Mindee.
"""
input_doc = FileInput(input_file)
if fix_pdf:
input_doc.fix_pdf()
return input_doc

@staticmethod
def source_from_b64string(
input_string: str, filename: str, fix_pdf: bool = False
) -> Base64Input:
"""
Load a document from a base64 encoded string.

:param input_string: Input to parse as base64 string
:param filename: The name of the file (without the path)
:param fix_pdf: Whether to attempt fixing PDF files before sending.
Setting this to `True` can modify the data sent to Mindee.
"""
input_doc = Base64Input(input_string, filename)
if fix_pdf:
input_doc.fix_pdf()
return input_doc

@staticmethod
def source_from_bytes(
input_bytes: bytes, filename: str, fix_pdf: bool = False
) -> BytesInput:
"""
Load a document from raw bytes.

:param input_bytes: Raw byte input
:param filename: The name of the file (without the path)
:param fix_pdf: Whether to attempt fixing PDF files before sending.
Setting this to `True` can modify the data sent to Mindee.
"""
input_doc = BytesInput(input_bytes, filename)
if fix_pdf:
input_doc.fix_pdf()
return input_doc

@staticmethod
def source_from_url(
url: str,
Expand Down
91 changes: 91 additions & 0 deletions mindee/client_mixin.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
from pathlib import Path
from typing import BinaryIO, Union

from mindee.error import MindeeClientError
from mindee.input import Base64Input, BytesInput, FileInput, PathInput


class ClientMixin:
"""Mixin for client Client V1 & V2 common static methods."""

@staticmethod
def source_from_path(
input_path: Union[Path, str], fix_pdf: bool = False
) -> PathInput:
"""
Load a document from an absolute path, as a string.

:param input_path: Path of file to open
:param fix_pdf: Whether to attempt fixing PDF files before sending.
Setting this to `True` can modify the data sent to Mindee.
"""
input_doc = PathInput(input_path)
if fix_pdf:
input_doc.fix_pdf()
return input_doc

@staticmethod
def source_from_file(input_file: BinaryIO, fix_pdf: bool = False) -> FileInput:
"""
Load a document from a normal Python file object/handle.

:param input_file: Input file handle
:param fix_pdf: Whether to attempt fixing PDF files before sending.
Setting this to `True` can modify the data sent to Mindee.
"""
input_doc = FileInput(input_file)
if fix_pdf:
input_doc.fix_pdf()
return input_doc

@staticmethod
def source_from_b64string(
input_string: str, filename: str, fix_pdf: bool = False
) -> Base64Input:
"""
Load a document from a base64 encoded string.

:param input_string: Input to parse as base64 string
:param filename: The name of the file (without the path)
:param fix_pdf: Whether to attempt fixing PDF files before sending.
Setting this to `True` can modify the data sent to Mindee.
"""
input_doc = Base64Input(input_string, filename)
if fix_pdf:
input_doc.fix_pdf()
return input_doc

@staticmethod
def source_from_bytes(
input_bytes: bytes, filename: str, fix_pdf: bool = False
) -> BytesInput:
"""
Load a document from raw bytes.

:param input_bytes: Raw byte input
:param filename: The name of the file (without the path)
:param fix_pdf: Whether to attempt fixing PDF files before sending.
Setting this to `True` can modify the data sent to Mindee.
"""
input_doc = BytesInput(input_bytes, filename)
if fix_pdf:
input_doc.fix_pdf()
return input_doc

@staticmethod
def _validate_async_params(
initial_delay_sec: float, delay_sec: float, max_retries: int
) -> None:
min_delay = 1
min_initial_delay = 1
min_retries = 1
if delay_sec < min_delay:
raise MindeeClientError(
f"Cannot set auto-parsing delay to less than {min_delay} second(s)."
)
if initial_delay_sec < min_initial_delay:
raise MindeeClientError(
f"Cannot set initial parsing delay to less than {min_initial_delay} second(s)."
)
if max_retries < min_retries:
raise MindeeClientError(f"Cannot set retries to less than {min_retries}.")
Loading