Skip to content

OneDrive source connector: workflow run trigger #687

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

OneDrive source connector: workflow run trigger #687

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -276,6 +276,7 @@
"examplecode/tools/azure-storage-events",
"examplecode/tools/gcs-events",
"examplecode/tools/google-drive-events",
"examplecode/tools/onedrive-events",
"examplecode/tools/sharepoint-events",
"examplecode/tools/jq",
"examplecode/tools/firecrawl",
Expand Down
193 changes: 193 additions & 0 deletions examplecode/tools/onedrive-events.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,193 @@
---
title: OneDrive event triggers
---

You can use OneDrive events, such as adding new files to—or updating existing files within—OneDrive folders, to automatically run Unstructured ETL+ workflows
that rely on those folders as sources. This enables a no-touch approach to having Unstructured automatically process new and updated files in OneDrive folders as they are added or updated.

This example shows how to automate this process by adding two custom [Power Automate](https://www.microsoft.com/power-platform/products/power-automate) flows to your Microsoft 365 or Office 365 account. One flow runs
whenever a new file is detected in the specified OneDrive folder. The other flow runs whenever an existing is file is updated within that folder. These flows then call the [Unstructured Workflow Endpoint](/api-reference/workflow/overview) to automatically run the
specified corresponding Unstructured ETL+ workflow within your Unstructured account.

<Note>
This example uses custom Power Automate flows that you create and maintain.
Any issues with file detection, timing, or flow invocation could be related to your custom flows,
rather than with Unstructured. If you are getting unexpected or no results, be sure to check your custom
flows' run histories first for any informational and error messages.
</Note>

## Requirements

import GetStartedSimpleApiOnly from '/snippets/general-shared-text/get-started-simple-api-only.mdx'

To use this example, you will need the following:

- A Microsoft 365 or Office 365 Business or enterprise plan that includes SharePoint Online, OneDrive, and Power Automate.
(Even if you only plan to use OneDrive, you still need a plan that includes SharePoint Online, because OneDrive is built on SharePoint technology.)
[Shop for business plans](https://www.microsoft.com/microsoft-365/business/compare-all-microsoft-365-business-products).
[Shop for enterprise plans](https://www.microsoft.com/microsoft-365/enterprise/microsoft365-plans-and-pricing).

- A Power Automate Premium license. [Learn how](https://learn.microsoft.com/power-platform/admin/power-automate-licensing/buy-licenses).

- An Unstructured account, and an Unstructured API key for your account, as follows:

<GetStartedSimpleApiOnly />

- The Unstructured Workflow Endpoint URL for your account, as follows:

1. In the Unstructured UI, click **API Keys** on the sidebar.<br/>
2. Note the value of the **Unstructured Workflow Endpoint** field.

- A OneDrive source connector in your Unstructured account. [Learn how](/ui/sources/onedrive).
- Some available [destination connector](/ui/destinations/overview) in your Unstructured account.
- A workflow that uses the preceding source and destination connectors. [Learn how](/ui/workflows).

## Step 1: Create an app registration

In this step, you create an app registration in Microsoft Entra ID to access the Azure key vault that you will create in the next step.

1. Sign in to your [Azure portal](https://portal.azure.com).
2. In the **Search resources, services, and docs** box, enter **App registrations**, and then in the list of **Services**, click **App registrations**.
3. Click **+ New registration**.
4. Enter some **Name** for the app registration.
5. For **Supported account types**, select **Accounts in this organizational directory only**.
6. For **Redirect URI**, select **Web**, and then enter `https://localhost`.
7. Click **Register**.
8. In the sidebar, click **Overview**.
9. Expand **Essentials**, and note the values of the **Application (client) ID** and **Directory (tenant) ID** fields.
10. Next to **Client credentials**, click the **Add a certificate or secret** link.
11. On the **Client secrets** tab, click **+ New client secret**.
12. (Optional) Enter some **Description** for the client secret.
13. (Optional) Enter some non-default **Expires** value.
14. Click **Add**.
15. Note the value of the **Value** field. This is the app registration's client secret value. (Do not use the **Secret ID** value.)

## Step 2: Create an Azure key vault

In this step, you store your Unstructured API key in an Azure key vault for your Power Automate flow to access later.

1. With your Azure portal still open from the previous step, in the **Search resources, services, and docs** box, enter **Key vaults**, and then in the list of **Services**, click **Key vaults**.
2. Click **Create**.
3. On the **Basics** tab, set the following key vault settings:

| Setting | Suggested value | Description |
|---|---|---|
| **Subscription** | Your subscription | The Azure subscription within which to create your new key vault. |
| **Resource Group** | **Create new** | After you click **Create new**, enter some name for the new resource group within which to create your new key vault. |
| **Key vault name** | Some globally unique name | Some name that identifies your new key vault. Valid characters are `A`-`Z`, `a`-`z`, `0`-`9`, and `-`. |
| **Region** | Your preferred region | Select a region that's near you or near other services that your key vault can access. |
| **Pricing tier** | **Standard** | Choose the pricing tier for your key vault. |

4. Click **Review + create**.
5. Click **Create**.
6. After the deployment is complete, click **Go to resource**.
7. In the sidebar, click **Access control (IAM)**.
8. Click **+ Add > Add role assignment**.
9. On the **Role** tab, in the **Job function roles** list, select **Key Vault Administrator**.
10. Click **Next**.
11. On the **Members** tab, with **Assign access to** set to **User, group, or service principal**, click **+ Select members**.
12. Search for and select the name of the app registration that you created earlier in Step 1.
13. Search also for and select your user account.
14. Click **Select**.
15. Click **Review + assign** twice.
16. In the sidebar, expand **Objects**, and then click **Secrets**.
17. Click **+ Generate/Import**.
18. Enter some **Name** for the secret.
19. For **Secret value**, enter your Unstructured API key value.
20. Click **Create**.

## Step 3: Create the first Power Automate flow

1. Sign in to [Power Automate](https://make.powerautomate.com).
2. On the sidebar, click **Create**.
3. In the **Start from blank** area, click **Automated cloud flow**.
4. Enter some **Flow name** for your new flow.
5. For **Search all triggers**, enter `When a file is created`, and then click
**When a file is created (OneDrive for Business)**.
6. Click **Create**.
7. In the flow designer, click the **When a file is created** node.
8. When prompted, sign in to OneDrive.
9. On the **Parameters** tab, set the following node properties:

- For **Folder**, click the folder icon, and the select the folder that matches the value of the **Path** value for the OneDrive source connector for your workflow.
- For **Include subfolders**, select **Yes** only if you also checked the **Recursive** checkbox for the OneDrive source connector for your workflow.

10. Below the **When a file is created** node, click the plus (**Insert a new action...**) button.
11. In the **Search for an action or connector** box, enter `Get secret`, and then under **Azure Key Vault**, select **Get secret**.
12. Set the following node properties:

- Enter some **Connection name** for the connection.
- For **Authentication type**, select **Service principal authentication**.
- For **Client ID**, enter the **Application (client) ID** value of the app registration that you created earlier in Step 1.
- For **Client secret**, enter the client secret value of the app registration that you created earlier in Step 1.
- For **Tenant ID**, enter the **Directory (tenant) ID** value of the app registration that you created earlier in Step 1.
- For **Key vault name**, enter the name of the key vault that you created earlier in Step 2.

13. Click **Create new**.
14. In the **Name of the secret** list, select the name of the secret that you created earlier in Step 2.
15. Below the **Get secret** node, click the plus (**Insert a new action...**) button.
16. In the **Search for an action or connector** box, enter `HTTP`, and then under **HTTP**, select **HTTP**.
17. On the **Parameters** tab, set the following node properties:

- For **URI**, enter `<unstructured-api-url>/workflows/<workflow-id>/run`, and replace the following placeholders:

- Replace `<unstructured-api-url>` with your Unstructured Worfklow Endpoint value.
- Replace `<workflow-id>` with the ID of your Unstructured workflow.

The **URI** should now look similar to the following:

```text
https://platform.unstructuredapp.io/api/v1/workflows/11111111-1111-1111-1111-111111111111/run
```

- For **Method**, select **POST**.
- For **Headers**, enter the following:

- `accept`, set to `application/json`
- `unstructured-api-key`, set to your key vault value. To do this, type `/` and select **Insert dynamic content**.
Under **Get secret**, select **body/value**.

18. In the top navigation bar, click the disk (**Save**) button.
19. Click **Back** to go to the flow's home page.
20. In the top navigation bar, if the power button shows **Turn on**, click it to change it to **Turn off** (which means that the flow is on and active).

## Step 4: Create the second Power Automate flow

1. With the flow's home page still open, on the sidebar, click **My flows**.
2. In the list of flows, rest your mouse pointer on the flow that you created earlier in Step 3, and then click the ellipsis (three dots, **More commands**) icon.
3. Click **Save as**.
4. Give the new flow a **Flow name** that is different from the original flow's name.
5. Click **Save**.
6. In the list of flows, rest your mouse pointer on the flow that you just created, and then click the pencil (**Edit**) icon.
7. In the flow designer, right click the **When a file is created** node, and then click **Delete**.
8. Click the **Add a trigger** node.
9. In the **Search for an action or connector** box, enter `When a file is modified`, and then click
**When a file is modified (OneDrive for Business)**.
10. On the **Parameters** tab, set the following node properties:

- For **Folder**, click the folder icon, and the select the folder that matches the value of the **Path** value for the OneDrive source connector for your workflow.
- For **Include subfolders**, select **Yes** only if you also checked the **Recursive** checkbox for the OneDrive source connector for your workflow.

11. In the top navigation bar, click the disk (**Save**) button.
12. Click **Back** to go to the flow's home page.
13. In the top navigation bar, if the power button shows **Turn on**, click it to change it to **Turn off** (which means that the flow is on and active).

## Step 5: Trigger the flows and view results

1. On a separate tab in your web browser, go to the OneDrive folder that corresponds to both your Unstructured workflow's
OneDrive source connector and the Power Automate flows' **When a file is created** and **When a file is modified** nodes.
2. Add a file to the OneDrive folder.
3. On a separate tab in your web browser, in the Unstructured user interface for your account, click **Jobs** on the sidebar.
4. In the list of jobs, click the newly running job for your workflow. It might take a minute or so for the new job to appear in the list.
5. After the job status shows **Finished**, go to your destination location to see the results.
6. Repeat this step, but this time, update an existing file in the OneDrive folder.

## Step 6 (Optional): Turn off the flows

To stop the Power Automate flows from running automatically, go back to the web browser tab from earlier in Steps 3 and 4. Then, do the following:

1. On the sidebar, click **My flows**.
2. In the list of flows, click the name of the flow that you created earlier in Step 3.
3. With the flow's home page still open, in the top navigation bar, click the power (**Turn off**) button
to change it to **Turn on** (which means that the flow is off and inactive).
4. Repeat this step, but this time, click the name of the flow that you created earlier in Step 4.
20 changes: 11 additions & 9 deletions examplecode/tools/sharepoint-events.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ In this step, you create an app registration in Microsoft Entra ID to access the
7. Click **Register**.
8. In the sidebar, click **Overview**.
9. Expand **Essentials**, and note the values of the **Application (client) ID** and **Directory (tenant) ID** fields.
10. Click **Client credentials**.
10. Next to **Client credentials**, click the **Add a certificate or secret** link.
11. On the **Client secrets** tab, click **+ New client secret**.
12. (Optional) Enter some **Description** for the client secret.
13. (Optional) Enter some non-default **Expires** value.
Expand Down Expand Up @@ -90,7 +90,7 @@ In this step, you store your Unstructured API key in an Azure key vault for your
14. Click **Select**.
15. Click **Review + assign** twice.
16. In the sidebar, expand **Objects**, and then click **Secrets**.
17. Click **+ Generate/import**.
17. Click **+ Generate/Import**.
18. Enter some **Name** for the secret.
19. For **Secret value**, enter your Unstructured API key value.
20. Click **Create**.
Expand Down Expand Up @@ -123,9 +123,10 @@ In this step, you store your Unstructured API key in an Azure key vault for your
- For **Key vault name**, enter the name of the key vault that you created earlier in Step 2.

12. Click **Create new**.
13. Below the **Get secret** node, click the plus (**Insert a new action...**) button.
14. In the **Search for an action or connector** box, enter `HTTP`, and then under **HTTP**, select **HTTP**.
15. On the **Parameters** tab, set the following node properties:
13. In the **Name of the secret** list, select the name of the secret that you created earlier in Step 2.
14. Below the **Get secret** node, click the plus (**Insert a new action...**) button.
15. In the **Search for an action or connector** box, enter `HTTP`, and then under **HTTP**, select **HTTP**.
16. On the **Parameters** tab, set the following node properties:

- For **URI**, enter `<unstructured-api-url>/workflows/<workflow-id>/run`, and replace the following placeholders:

Expand All @@ -144,9 +145,9 @@ In this step, you store your Unstructured API key in an Azure key vault for your
- `accept`, set to `application/json`
- `unstructured-api-key`, set to your Unstructured API key value.

16. In the top navigation bar, click the disk (**Save**) button.
17. Click **Back** to go to the flow's home page.
18. In the top navigation bar, click the power (**Turn on**) button.
17. In the top navigation bar, click the disk (**Save**) button.
18. Click **Back** to go to the flow's home page.
19. In the top navigation bar, if the power button shows **Turn on**, click it to change it to **Turn off** (which means that the flow is on and active).

## Step 4: Trigger the flow and view results

Expand All @@ -160,4 +161,5 @@ In this step, you store your Unstructured API key in an Azure key vault for your
## Step 5 (Optional): Turn off the flow

To stop the Power Automate flow from running automatically, go back to the web browser tab from earlier in Step 3. Then,
with the flow's home page still open, in the top navigation bar, click the power (**Turn off**) button.
with the flow's home page still open, in the top navigation bar, click the power (**Turn off**) button
to change it to **Turn on** (which means that the flow is off and inactive).