Skip to content

Schedule batch job

Jump to bottom
Kwangje Cho edited this page Aug 13, 2019 · 14 revisions

In this tutorial you will learn how to:

  • Create a new Flow for batching
  • Configure input source of data (Supported today: blob)
  • Inspect schema of incoming data
  • Schedule jobs
  • Save

Steps to follow

  • Open Data Accelerator portal and go to the Flows tab

  • Click on "New" on the Flows tab, this will create a new Flow:

    New Flow

  • Switch to the Input tab and select Batching for Mode and Azure Blob for Type. Data Accelerator currently supports only Blob for Batching

  • Provide the following information of the blob you want to use as an input

    • The Connection string
    • The Blob Path. This should be a wasbs URL format. If your path contains date and time, you can use DateTime pattern with curly brackets '{' and '}'
      e.g. wasbs://<container_name>@<storage_account_name>.blob.core.windows.net/<folder_name>/{yyyy-MM-dd}.

Scheduling

  • Switch to the schedule tab and click Add button. There are two types of scheduling.
    • Recurring This creates a job based on the given recurrence. e.g. If Recurrence is 1 day, it will create one job per day.

Scheduling

  • One Time This creates all jobs for the given the date range at once (between the start time and the end time). And once all jobs are scheduled, the schedule will be disabled and read-only.

Scheduling

 * Recurrence:
It specifies an interval between jobs. The available values are day, hour and minute. <br/>

 * Delay:
It specifies an offset for a job. The offset is negative so an actual start time will be calculated by subtracting offset from the original start time. The available values are day, hour and minute.

 * Window:
It specifies how much a job will look back. The available values are day, hour and minute.

Data Accelerator

Install

Docs

Clone this wiki locally