Adding Transport Only Database Query Data Feeds

This feature is only available for Archer on-premises installations.

Complete the following tasks to add a Transport Only Database Query Data Feed.

On this page

Task 1: Add a transport data feed

  1. From the menu, click Admin menu > Integration > Data Feeds.

  2. Click Add to create a new data feed.
  3. In the General Information section, do the following:
    1.  Enter the name and description of the data feed.

      Note: The alias populates automatically when you set the name of the data feed for the first time and save the data feed. You can modify the alias after you save the data feed. The remaining fields in the General Information section are read-only and populate when the data feed is created, updated, and run.

    2. Select Select to make the data feed active.
  4. In the Feed Information section, do the following:
    1. In the Feed Type field, select Transport Only. See Adding Standard Data Feeds to add a Standard data feed.
    2. In the Target Path field, enter the directory path that the exported source file data is saved to.
    3. In the Service Account Name field, enter the user account associated with the data feed. If the user does not exist, you can create a new user. Enter the username. See Data Feeds Service Account for more information on the Service Account Name.

  5. Do one of the following:

    • To continue configuring the data feed, go to the next task.
    • To finish setting up the feed later, click Save or Save and Close.

Task 2: Define the transport method

Important: For the data feed to run successfully, the server responsible for running the data feed must have the required network access to the database.

  1. Go to the Source Connection tab of the data feed.

  2. From the Source Locale list, select a locale.
  3. From the Transport Method list, select a transport type.

  4. In the Database Configuration section, complete the configuration options.

    Complete the User Name and Password fields to specify credentials for an account that queries the database, only if the Connection String does not include credentials.

    The following table describes the options.

    Option

    Description

    Provider

    Specifies the data provider based on the type of connection string used.

    If Oracle dotConnect is the data provider, you must install the drivers for Oracle because these drivers are not included in the Archer installation. To determine the connection string for this data provider, go to the following URL:
    http://www.devart.com/dotconnect/oracle/docs/Devart.Data.Oracle~
    Devart.Data.Oracle.OracleConnection~ConnectionString.html    .

    Connection Timeout

    Specifies the timeout parameter in seconds to force the feed to fail because of long-running queries.

    Connection String

    Allows the data feed to locate and access the database and retrieve the specified source data.

    Query

    Specifies the query that runs against the database. This field cannot be longer than 4,000 characters.

    You can also run a stored procedure, such as the example in the following figure, by entering it in the Query field.

    Or operator logic

    Contact your database administrator prior to running any queries against the corporate data of your company. If you configure the query string incorrectly, you can alter the data stored in the database.
    Always verify your results by manually running the query directly against the database first.

  5. (Optional) The data feed creates a local copy of the source data for further processing. In the Post Processing - Local copy of source file section, select from the following options to specify how the data feed handles the local copy of the source data after processing the source data.

    The following table describes the options for post processing the local copy of the source data.

    Option

    Description

    Delete

    Deletes the processed source file when the data feed successfully completes. The data feed also deletes any local copy of the source information.

    Rename

    Saves the source file under a new name when the data feed successfully completes. In File Path and Name, specify the new name for the file and the location to save the file.

    To save the data, ensure that the account running the Job Engine service can access the path of the destination file.

    If you select this option, use filename tokens for specifying the location or name of the file.

    Filename tokens

    Filename tokens are available for post processing when you want to save the source information and specify a location or name for the file. When you select the Rename option, you can use tokens to generate unique names automatically for the files.

    Here are the usable tokens for renaming data files.

    • Now. Insert a user-defined date format within the new filename. Possible formats include Now(MM/dd/yyyy) or Now(MM-dd-yyyy). See the Microsoft .Net Framework Developer Center for available custom date/time formats.

    • DataFileDirectoryName. Update the filename with the directory name, including the drive, of your file.

    • DataFileName. Insert the original filename, excluding the directory name and extension.

    • DataFileExtension. Insert the file extension, such as .csv, in the new filename.

    • DataFileFullName. Insert the fully qualified filename. This data includes the drive, directory, filename, and extension of the original file.

    For example, if the data file came from the following location, C:\DataFeed\Source\ESL\processed\ThreatData.csv, any files renamed using tokens provide the following output.

    Example 1

    • Input Tokens: {DataFileDirectoryName}\success\{DataFileName}_{Now(MM.dd.yyyy)}.{DataFileExtension}
    • Output: C:\DataFeed\Source\ESL\processed\success\ThreatData_01.31.2008.csv

    Example 2

    • Input Tokens: \\DFSRepository\{Now(yyyy)}\{Now(MM)}\{DataFileName}_success.{DataFileExtension}
    • Output: \\DFSRepository\2008\01\ThreatData_success.csv

  6. If you selected the Rename post-processing option, enter the location and name of the new file you want to save in the File Path and Name field.

  7. Do one of the following:

    • To continue configuring the data feed, go to the next task.
    • To finish setting up the feed later, click Save or Save and Close.

Task 3: Define the data feed schedule

You can set up data feeds to run automatically at regular intervals. This reduces the time and effort required to import data from an external file. You can initiate data feeds at various times and configure them to run in regular increments for an indefinite period of time.

You can also run the data feed immediately.

To prevent excess server load, schedule data feeds on a staggered basis.

A reference feed allows you to specify another feed. This indicates to the Data Feed Service that this feed will start running as soon as the referenced feed completes successfully. A successful data feed run processes all input data, completes all expected record updates, and does not report any failures in the Run Details Report.

  1. Go to the Run Configuration tab > Schedule section.

  2. Do one of the following to schedule your data feed.

    Run on Schedule

    You can configure your data feed to run on a defined schedule.

    The following table describes the fields in the Run on Schedule option.

    Field

    Description

    Start Date

    Specifies the date on which the data feed schedule begins.

    Start Time

    Specifies the time the data feed starts running.

    Time Zone

    Specifies the time zone in which the data feed schedule begins.

    Recurring

    Specifies the interval in which the data feed runs, for example, Minutely, Hourly, Daily, Weekly, or Monthly.

    • Minutely. Runs the data feed by the interval set.

      • For example, if you specify 45 in the Every list, the data feed runs every 45 minutes.

    • Hourly. Runs the data feed by the interval set, for example, every hour (1), every other hour (2) and so forth.
    • Daily. Runs the data feed by the interval set, for example, every day (1), every other day (2) and, so forth.
    • Weekly. Runs the data feed based on a specified day of the week, for example, every other Monday (2), every third Monday (3), and so forth.
    • Monthly. Runs the data feed based on a specified week of the month, for example, on the first Monday of every month, on the second Tuesday of every third month, and so forth.

    Every

    Specifies the interval of the frequency in which the data feed runs.

    On

    Specifies the frequency of the days of the week on which the data feed runs.

    Weekday

    Specifies the days of the week on which the data feed runs.

    Run After

    Runs a specified data feed before the current one. The Data Feed Service starts the current data feed after the referenced data feed completes successfully.

    For example, you can select to have a Threats data feed run immediately after your Assets data feed finishes. From the Reference Feed dropdown, select the data feed that runs before the current data feed.

    Run Now

    Click the Run Now button in the toolbar on the Manage Data Feed page to run the data feed manually.

  3. To save the data feed, click Save or Save and Close.

  4. Do one of the following:

    • To continue configuring the data feed, go to the next task.
    • To finish setting up the feed later, click Save or Save and Close.