Archer Data Feed Monitor

Release date	Release	Notes
July 2025	2024.11	Updated to add a configuration parameter that allows a user to request SOAP authentication instead of REST
April 2022	6.8	Updated documentation with notes regarding implementation in Archer SaaS environments
March 2021	6.8	Updated documentation with the added support for the use of Stored Procedures in the SQL Monitor.
September 2020	6.8	The latest version of Archer Data Feed Monitor has the following updates: Exceptions that occur while checking the criteria or executing a feed for a given monitor result in the rest of the monitors being skipped for that execution. This bug has been fixed. App Settings in Config file are encrypted when the utility is run for the first time. Encrypt and decrypt batch files are no longer required.
October 2019	6.6	Added details for the Archer Record Statistical Monitor
May 2019	6.4 P2	The new version of Archer Data Feed Monitor uses the NLog framework to enable logging. There is no change in the business logic of how the Data Feed Monitor works, only a change in the logging functionality. The previous version of this tool did not use a logging framework and was writing data to one log file. Over time, this file will become quite large and difficult to work with. NLog provides a configurable option for logging such as setting the maximum log file size and options to configure a Rolling File Appender to create new log files at certain time intervals.
August 2018	6.4 SP1	Initial release

1. Extract the zip file DataFeedMonitor.zipto a location/folder.

2. Edit the appSettings section of the XML configuration file “DataFeedMonitor.exe.config”. If a named config file is being used, edit that file instead.

   1. Navigate to the web application folder DataFeedMonitor, and locate the DataFeedMonitor.exe.config file.

   2. Open DataFeedMonitor.exe.config file and provide the following information:

      <appSettings>

      <add key="useSoapAuth" value="[true/false]" /><addkey="archerBaseUrl"value="https://[YourArcherHostURL]"/>

      <addkey="cacheFilePath"value="[cacheFilePath]"/>

      <addkey="cacheFileEncryptionPassphrase"value="[EncryptionPassphrase]"/>

      <addkey="useSslValidation"value="[true/false]"/>

      <addkey="securityProtocolVersion" value="[Ssl3/Tls/Tls11/Tls12]" />

      <addkey="archerInstance"value="[InstanceName]"/>

      <addkey="archerUser"value="[UserName]"/>

      <addkey="archerPwd"value="[Password]"/>

      <addkey="httpTimeoutMilliseconds" value="[httpTimeoutMilliseconds]" />

      <addkey="useDefaultCredentials" value="[true/false]" />

      </appSettings>

The following table describes each setting.

Setting	Description and example
[useSoapAuth]	true/false. If archer instance has sso enabled with manual bypass disabled, then use "true". Else use "false"
[YourArcherHostURL]	Archer host URL Example: company.com or 10.20.30.40:99
[cacheFilePath]	Cache file path Example:  c:\temp\
[EncryptionPassphrase]	Encryption pass phrase Example:  d0n7TryTh1s@h0m3!
[true/false]	True or false to validate SSL Example: false
[Ssl3/Tls/Tls11/Tls12]	The version of security protocol to be used incommunicating with the Archer web server. Example:Tls12
[InstanceName]	Instance name of your Archer environment Example: ArcherProd Note: Instance name is case sensitive.
[UserName]	Username of the account Example: adminuser
[Password]	Password of the user account Example:  Password@123
[httpTimeoutMilliseconds]	httpTimeoutMilliseconds Example: 30000
[useDefaultCredentials[	true/false Example: false

3. Edit the monitorSection section of the XML configuration file.    

Edit the monitorSection section of the XML configuration file.    

<monitorSection>

<monitors>

<monitor monitorType="ArcherRecordMonitor"

  name="[Name of the datafeed. Value should be Unique]"         

  associatedFeedGuid="[Data Feed Guid]"

isFeedConvoy="[true/false]"

useSoapAuthentication="[true/false]"

archerBaseUrl="http://[YourArcherHostURL]"

cacheFilePath="[cacheFilePath]"

cacheFileEncryptionPassphrase="[EncryptionPassphrase]"

useSslValidation="[true/false]"

archerInstance="[InstanceName]"

archerUser="[UserName]"

archerPassword="[Password]"   

reportGuid="[Archer Report Guid]

useDefaultCredentials="[true/false]"/>

 

<monitor monitorType="ArcherRecordStatisticalMonitor"

name="[Name of the datafeed. Value should be Unique]"         

associatedFeedGuid="[Data Feed Guid]"

             isFeedConvoy="[true/false]"

archerBaseUrl="http://[YourArcherHostURL]"

cacheFilePath="[cacheFilePath]"

cacheFileEncryptionPassphrase="[EncryptionPassphrase]"

useSslValidation="[true/false]"

archerInstance="[InstanceName]"

archerUser="[UserName]"

archerPassword="[Password]"   

reportGuid="[Archer Statistical Report Guid]"

useDefaultCredentials="[true/false]"/>

 

<monitor monitorType="FileSystemMonitor"

name="[Name of the datafeed. Value should be Unique]"   

associatedFeedGuid="[Data Feed Guid]"

isFeedConvoy="[true/false]"

filePatternToMatch="[File Type to watch]"

directoryToWatch="[Directory Path to be monitored]" />

 

<monitor monitorType="SqlDatabaseMonitor"

name="[Name of the datafeed. Value should be Unique]"

associatedFeedGuid="[Data Feed Guid]"

isFeedConvoy="[true/false]"

connectionString="[a connection string to the SQL DB Source]">

<command

isProc="[true/false]"

isScalar="[true/false]"

commandText="[The SQL Query or stored procedure name to be executed]">

<params>

<param name="[the name of the SQL Parameter]"

type="[The SQL DB Type of the SQL Parameter]"value="[the value of the SQL Parameter]"/>

</params>

</command>

</monitor>

 

<monitor monitorType="FtpMonitor"

name="[Name of the datafeed. Value should be Unique]"

associatedFeedGuid="[Data Feed Guid]"

isFeedConvoy="[true/false]"

filePatternToMatch="[Regex identifying the files to be watched]"

ftpServerUrl="[FTP Url]"

isAnonymous="[true/false]"

user="[FTP UserName]"

password="[FTP Password]" />

</monitors>

</monitorSection>

4. Use seperate config file for <monitorSection> (Optional) Users can also use a separate config file for the <monitorSection>. To use a separate file for monitor section:

• Edit and replace the <monitorSection> in the xml configuration file with 

  <monitorSection configSource="[FolderName\monitor.config]" />.

  “FolderName” is the folder where the monitor config files are placed and “monitor.config” is the file containing the feed details.

  Ex: <monitorSection configSource="ConfigFiles\Test1.config">

• The monitor file should have the following format:

  <?xmlversion="1.0"?>

  <monitorSection>

    <monitors>

      [Described above]

    </monitors>

  </monitorSection>

5. Save and Close the DataFeedMonitor.exe.config file.

   Note: Changing the Archer URL in the Config file requires the deletion of the previously used cache file.

6. Open NLog.config. Locate file target and provide the following information.

   filename="[logFilePath]"

   Example: fileName="c:\temp\DataFeedMonitor_Nlog.log"

   Name of the file to write to.Provide name of the file to write to with the full path.

   archiveFileName="[archivelogFilePath]"

   Example: archiveFileName="c:\temp\logArchive\DataFeedMonitor_Nlog.{#}.log"

   Name of the file to be used for an archive. It contains a special placeholder {#} that will be replaced with a sequence of numbers depending on the archiving strategy. The number of hash characters used determines the number of numerical digits to be used for numbering files.

Open Windows Task Scheduler and create a task to run DataFeedMonitor.exe according to the desired schedule. If you are only scheduling a single task to run the application, you can use the default configuration file. If you need to schedule multiple tasks to run data feeds on different schedules, store a separate configuration file for each task and pass the name and path of that configuration file to the application on the command line in the task. Whether using the default or a separate named configuration file, create unique Archer user accounts for each task instance so that no other process can authenticate with the same credentials and invalidate the session being used by your task.

To create a Windows Scheduler Task for the application:

1. Open Windows Task Scheduler

2. On the right side of the window under Action, click on Create Task.

3. In the dialog, enter the following:

   Section	Description
   General	Provide the DataFeed Name
   Triggers	Create a new trigger by providing the Frequency to run the tool
   Actions	Select the DataFeedMonitor.exe from extracted application folder Archer Data Feed Monitor. Also select a config file in “Add arguments” if you need to set up multiple schedules and are using named configuration files.

4. Once all the properties for windows task schedule is complete, click on OK to start the schedule.

Note: If Data Feed Monitor is configured to run in Windows Task Scheduler, all associated data feeds referenced in the config file should have start dates several years in the future in Archer to prevent unexpected executions.

The solution makes use of a well-known logging framework called NLog. NLog allows for the creation of one or more log <target> entries, and one or more associated <logger>entries. The targets define potential “listeners” for log information, and the loggers define the rules associated with each of them, including log level. Although a sample config file is provided with the solution, a full discussion of the configuration file and its use is outside the scope of this document. For more details, please see the NLog documentation.

<appSettings>

<add key="useSoapAuth" value="[true/false]" />

<addkey="archerBaseUrl"value="https://[YourArcherHostURL]"/>

<addkey="cacheFilePath"value="[cacheFilePath]"/>

<addkey="cacheFileEncryptionPassphrase"value="[EncryptionPassphrase]"/>

<addkey="useSslValidation"value="[true/false]"/>

<addkey="securityProtocolVersion" value="[Ssl3/Tls/Tls11/Tls12]" />

   <addkey="archerInstance"value="[InstanceName]"/>

<addkey="archerUser"value="[UserName]"/>

<addkey="archerPassword"value="[Password]"/>

<addkey="httpTimeoutMilliseconds" value="[httpTimeoutMilliseconds]" />

<add key="useDefaultCredentials" value="[true/false]" />

</appSettings>

</appSettings>

This file contains the following configuration details:

<appSettings>

<add key="useSoapAuth" value="[true/false]" />

<addkey="archerBaseUrl"value="https://[YourArcherHostURL]"/>

<addkey="cacheFilePath"value="[cacheFilePath]"/>

<addkey="cacheFileEncryptionPassphrase"value="[EncryptionPassphrase]"/>

<addkey="useSslValidation"value="[true/false]"/>

<addkey="securityProtocolVersion" value="[Ssl3/Tls/Tls11/Tls12]" />

   <addkey="archerInstance"value="[InstanceName]"/>

<addkey="archerUser"value="[UserName]"/>

<addkey="archerPassword"value="[Password]"/>

<addkey="httpTimeoutMilliseconds" value="[httpTimeoutMilliseconds]" />

<add key="useDefaultCredentials" value="[true/false]" />

</appSettings>

</appSettings>

<appSettings>

<add key="useSoapAuth" value="false" />

<addkey="archerBaseUrl"value="https://localhost/archer"/>

<addkey="cacheFilePath"value="c:\temp\"/>

<addkey="cacheFileEncryptionPassphrase"value="d0n7TryTh1s@h0m3!"/>

<addkey="securityProtocolVersion" value="Tls" />

<addkey="useSslValidation"value="false"/>

<addkey="archerInstance"value="Archer"/>

<addkey="archerUser"value="adminuser"/>

<addkey="archerPassword"value="Password@123"/>

<addkey="httpTimeoutMilliseconds" value="30000" />

<add key="useDefaultCredentials" value="false" />

</appSettings>

The following table describes the required monitor attributes.

Attribute	Description
monitorType	Must be one of 5 supported values corresponding to the type of transporter being used by the feed to be monitored.
archerRecordStatisticalMonitor	Archer Web Services Transporter (statistical report)
fileSystemMonitor	File Transporter
sqlDatabaseMonitor	Database Query Transporter
ftpMonitor	FTP Transporter
name	Name of the monitor is used only to allow the administrator to determine what has been configured. It is recommended that monitors be named for the feeds they will run.
associatedFeedGuid	GUID associated with the feed that should be run when the monitor finds pending work.
isFeedConvoy	Boolean (true/false) value indicating whether or not this feed has others that reference it. When this value is set to “true”, the next feed will be automatically executed upon completion of the reference feed.

Note: File Transporter and Database Transporter datafeeds are not supported in SaaS.

The following table lists the monitor attributes that may be required by the type of monitor being configured. If these attributes are not specified in the configuration for the monitor, an exception will be logged, and the application will exit.

Monitor Attribute	Description
reportGuid	GUID associated with the Archer report that supplies the source data for the configured feed. This report will be executed using the SOAP API. If the report is a statistics report, the Archer Record Statistical Monitor must be used.
Filesystem Monitor	directoryToWatch– the path to the directory that will be monitored. filePatternToMatch– a file specifier that identifies which files to watch for in the configured watch directory. For example, “*.csv”.
SQL Database Monitor	connectionString– a valid connection string for a Microsoft SQL Server database. This is using the .NET SqlClient class under the covers and does not currently support Oracle connections. command– an XML structure specifying the details of the command to be executed. command/isProc – a boolean [true/false] value indicating whether the configured command text is a stored procedure to be executed. command/isScalar – a boolean [true/false] value indicating whether the configured stored procedure returns a scalar value. Set this value to false for stored procedures that return row results. command/commandText – the actual text of the command to be executed. For SQL queries, this will be the text of the query. For stored procedures, this will be the name of the procedure. command/Param – an individual parameter to be passed to the configured stored procedure. command/Param/name – the name of the parameter command/param/type – the SQL DB Type associated with the parameter. Valid values are: BigInt, Binary, Bit, Char, DateTime, Decimal, Float, Image, Int, Money, NChar, NText, NVarChar, Real, UniqueIdentifier, SmallDateTime, SmallInt, SmallMoney, Text, Timestamp, TinyInt, VarBinary, VarChar, Variant, Xml, Udt, Structured, Date, Time, DateTime2, DateTimeOffset command/param/value – the value of the parameter
FTP Monitor	ftpServerUrl– the URL to the FTP server to be monitored. The current version of the FTP monitor does not support secure FTP or monitoring in folders other than the root. This is provided as a reference implementation. isAnonymous– a Boolean (true/false) value indicating whether the connection should be made using anonymous credentials. When false, username and password should be supplied. User– a valid FTP user id Password– the password associated with the configured FTP user id filePatternToMatch– a regular expression identifying the files to watched for in the configured FTP server. Since FTP does not support file filtering in the same way that the filesystem does, a directory listing is requested and each row of the result is checked for a match against the regular expression. The first row that matches will cause the feed to be executed. For example, “^.*\.(csv)$” would be equivalent to “*.csv “.

• Use HTTPS/TLS for secured connection between the DFM Application and Archer.

• Run the DFM application with the least privileged account (Logon Service permission).

• Run the DFM Utility immediately after configuring to encrypt the monitor and app settings in the config file.

The Data Feed Monitor must be executed immediately after configuring. Failure to do so will result in a risk exposure, leaving parameters entered in monitor Section and app Settings (includes Archer credentials, ftp credentials and sql db parameters if configured) in the config file in plain text.