PI Interface for Universal File and Stream Loading...

PI Interface for Universal File and Stream Loading (UFL)

Version 3.3.12.x

OSIsoft, LLC

777 Davis St., Suite 250

San Leandro, CA 94577 USA

Tel: (01) 510-297-5800

Fax: (01) 510-357-8136

Web: http://www.osisoft.com

OSIsoft Australia • Perth, Australia

OSIsoft Europe GmbH • Frankfurt, Germany

OSIsoft Asia Pte Ltd. • Singapore

OSIsoft Canada ULC • Montreal & Calgary, Canada

OSIsoft, LLC Representative Office • Shanghai, People’s Republic of China

OSIsoft Japan KK • Tokyo, Japan

OSIsoft Mexico S. De R.L. De C.V. • Mexico City, Mexico

OSIsoft do Brasil Sistemas Ltda. • Sao Paulo, Brazil

PI Interface for Universal File and Stream Loading (UFL)

Copyright: © 2013 OSIsoft, LLC. All rights reserved.

No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, mechanical, photocopying, recording, or otherwise, without the prior written permission of OSIsoft, LLC.

OSIsoft, the OSIsoft logo and logotype, PI Analytics, PI ProcessBook, PI DataLink, ProcessPoint, PI Asset Framework(PI-AF), IT Monitor, MCN Health Monitor, PI System, PI ActiveView, PI ACE, PI AlarmView, PI BatchView, PI Data Services, PI Manual Logger, PI ProfileView, PI WebParts, ProTRAQ, RLINK, RtAnalytics, RtBaseline, RtPortal, RtPM, RtReports and RtWebParts are all trademarks of OSIsoft, LLC. All other trademarks or trade names used herein are the property of their respective owners.

U.S. GOVERNMENT RIGHTS

Use, duplication or disclosure by the U.S. Government is subject to restrictions set forth in the OSIsoft, LLC license agreement and as provided in DFARS 227.7202, DFARS 252.227-7013, FAR 12.212, FAR 52.227, as applicable. OSIsoft, LLC.

Published: 03/2013

http://www.osisoft.com/

PI Interface for Universal File and Stream Loading (UFL) iii

Table of Contents

Chapter 1. Introduction ............................................................................................................ 1

Related Manuals ...................................................................................................... 2

Chapter 2. How UFL Works ..................................................................................................... 3

Interface Logic ......................................................................................................... 3 How Data is Processed ........................................................................................... 3 How Data Sources are Handled .............................................................................. 4 Logging .................................................................................................................... 5

Chapter 3. Installing the UFL Interface ................................................................................... 6

Prerequisites ............................................................................................................ 6 Upgrading From Pre-3.x Versions ........................................................................... 6 Detailed Installation Procedure ................................................................................ 8

Chapter 4. Configuring Diagnostics ..................................................................................... 11

Health Monitoring Points ........................................................................................ 11 I/O Rate Point ........................................................................................................ 13 Interface Status Point ............................................................................................ 13 Scan Class Performance Points ............................................................................ 14 Performance Counter Points ................................................................................. 15

Chapter 5. Loading Data into PI ............................................................................................ 17

Defining the Configuration File .............................................................................. 17 Performing Calculations Using Fields .................................................................... 33 Storing Data in PI Tags .......................................................................................... 36 Parsing Standard File Types ................................................................................. 37

Chapter 6. Configuring UFL Using UFLDesigner ................................................................ 42

Appendix A. Platform Details and Features ............................................................................ 44

Supported Features ............................................................................................... 44 Platform .................................................................................................................. 46

Appendix B. Installation and Configuration Checklist ........................................................... 47

Appendix C. Command Line Parameters ................................................................................ 49

Appendix D. UFL Examples ...................................................................................................... 51

Calculation Examples ............................................................................................ 51 Action Examples .................................................................................................... 57

Table of Contents

iv

Appendix E. Migrating from the Batch File Loader Interface ................................................ 61

Appendix F. Glossary ................................................................................................................ 64

Appendix G. Technical Support and Resources..................................................................... 65

PI Interface for Universal File and Stream Loading (UFL) 1

Chapter 1. Introduction

OSIsoft’s Universal File and Stream Loading Interface (PI UFL interface) reads data from

ASCII data sources and writes data to PI. The PI UFL interface can read text from ASCII

files, a serial port, or a POP3 email server, parse and transform the incoming data, and update

PI points. To specify how the incoming data is handled, you create a configuration (INI) file,

which defines how data is filtered, parsed, manipulated and written to PI. The PI UFL

interface is intended to replace the PI Batch File interface. For details about migration, see

Appendix E.

NOTE: The PI UFL interface can process XML files, but for applications that are best addressed using the OPC XML-DA specification, use the PI Interface for OPC DA XML.

Introduction

2

The following figure illustrates the basic configuration of the hardware and software

components.

Separate PI Server and interface nodes All PI Software installed on one node

Related Manuals

PI Server manuals

PI API Installation Manual

PI Buffer Subsystem User Guide


Chapter 2. How UFL Works

Interface Logic

The UFL interface is launched using a Windows batch file that invokes the UFL executable

with a set of parameters. When launched, the interface performs the following steps:

1. Validate the startup parameters.

2. Check the syntax of the expressions in the configuration (.INI) file. If errors are

detected, log the errors and exit.

3. Compile the configuration file.

4. If a point source or tag mask is specified, load the corresponding points.

Main processing loop:

1. Check whether new data has arrived.

2. If new data is available, read it and, for each message (line), check whether the message

satisfies any of the filters in the INI file.

3. If a message passes the filters, determine its message type (details below). Messages that

are not caught by a filter are not processed or logged.

4. Break the message into its component fields, perform any required calculations and

transformations, then write the resulting data to PI.

How Data is Processed

The UFL interface parses incoming text one line at a time. Lines of text are referred to as

“messages,” and you can use text-matching expressions to specify how a message is parsed

into its component fields.

For example, the following report file, created by a manufacturing process, is composed of

two heading lines followed by data:

HEADING 1: Pressure Report

HEADING 2: Unit AB12 02-Dec-2013

Date: 01-Dec-2013 12:00:00 Value: 341 Tag: Pressure



How UFL Works

4

To process the preceding file, your INI file must handle three types of messages:

First line of header (no data, no need to process)

Second line of header (contains report date)

Data (contains timestamp, value and tag)

To process the data lines, you define fields for the data, a message name, and statements that

parse the line into fields. For example, to parse the preceding data lines, you filter for lines

beginning with the string “Date:” and parse the lines into timestamp, value and tag fields

according to their position in the line, as follows:

'Declare fields, with name and data type

[FIELD]

FIELD(1).Name = "time"

FIELD(1).Type = "DateTime"

FIELD(2).Name = "value"

FIELD(2).Type = "Number"

FIELD(3).Name = "tag"

FIELD(3).Type = "String"

'Define name for message

msg(1).name = "PressureData"

'Match on contents of message and parse into fields

[PressureData]

PressureData = C1=="Date:*"

time=C7-C26 'timestamp field starts at column 7

value=C35-C37 'value field starts at column 35

tag=C43-C51 'tag name starts at column 43

(For the complete example file, see “Defining the Configuration File.”)

How Data Sources are Handled

The logic for each type of incoming data that the UFL interface handles (ASCII files, serial

port and POP3 server) is contained in a plug-in, installed as a DLL. You specify the required

DLL in the INI file. (The interface contains its own logic for BatchFL interface files.) The

following sections describe how each method of data delivery is handled.

ASCII Files

In the INI file, you specify one or more input files. For example, to process all text files with

names that start with “ProcData,” set the IFM parameter as follows:

IFM = "C:\Data\ProcData*.txt"

Files can be processed in order by filename, creation date or modification date. After

successful processing, the input files are renamed with a suffix indicating the time of

processing. You can configure the interface to purge processed files at a specified interval.


Serial Port

The PI UFL interface initializes the COM port using parameters specified in the INI file.

Characters are read from the port and assembled into messages that are periodically submitted

to the interface for parsing and processing. To configure the frequency with which messages

are submitted, use PI ICU to set Scan Class on the General tab.

If the serial port plug-in fails to initialize, the UFL interface logs Microsoft Windows system

error codes, which you can look up on Microsoft Support web sites (search for the results of

the Windows function call GetLastError()). Common errors and likely causes:

Error Message Likely Cause

2 – The system cannot find the file

specified The specified serial port does not exist.

5 – Access denied The specified serial port is being used by some other driver.

87 – The parameter is incorrect One of the port parameters is not properly specified.

POP3 Server

The PI UFL interface connects to a specified POP3 server as the specified user and

periodically downloads emails for processing. The processed emails are then deleted from the

POP3 server. Emails can be forwarded to a specified SMTP server. Communication is

performed over a TCP/IP connection using TCP port 110. Communication over the SSL

(Secure Socket Layer) on alternate port 995 (known as POP3S) is not supported.

Batch Files

The UFL interface can process comma-delimited text files that use the format accepted by the

PI Batch File interface.

Logging

Errors and informational messages are sent to standard output and logged to the local

message log or the interface log (if the OUTPUT keyword is set in the INI file). When

logging identical messages, the interface logs a maximum of ten, even if the same error

occurs more than ten times.

If an error includes an error number, you can display more details by issuing the following

command:

\%PIPC%\adm\pidiag /e error_number

Note: If you enable “Laboratory Snapshots” and snapshot bypass (/lb and /lbs flags)

to take advantage of caching for efficiency, be advised of a side effect on logging: the interface is sometimes unable to log the data line that caused an error when it logs the error, because the line might no longer be in the cache when the error is logged.


Chapter 3. Installing the UFL Interface

This chapter provides detailed instructions for installing and configuring the UFL interface. If

you’re familiar with PI interface installation, refer to the installation checklist appendix in

this manual for basic installation guidelines.

Prerequisites

Before installing the interface:

Verify that the PI Server is up and running.

Verify that the interface node time zone is set correctly.

Each instance of the UFL interface requires its own startup batch file. The batch file contains

commands and flags that configure settings for the interface. The batch files reside in the

interface installation directory, which also includes a template batch file

(PI_UFL.bat_new) that you can use to create new instances of the interface.

To ensure a correctly-formatted batch file, use the PI Interface Configuration Utility (ICU) to

configure and troubleshoot the interface (as opposed to manually editing the startup batch

file). For a complete list of valid command line flags, open a command window, navigate to

the interface installation directory, and issue the following command:

PI_UFL -help

Upgrading From Pre-3.x Versions

Version 3.x of the PI UFL interface includes the functionality of the PI BatchFL interface and

the PI Message Logger interface. The new PI UFL interface consists of a basic framework

plus data source-specific plugins, which are implemented as DLLs. The syntax for the

message detection, field descriptions and configuration (INI) file remain the same. In Version

3.x, a few configuration parameters have been changed and moved from the batch startup file

to the INI file. The following sections provide details about the changes made in version 3.x.

Interface Startup Batch File Changes

New Parameters

/am

/disablecounters

/ec

/imt


/lbs

/perf=#

/rbo

/runonce

/wd

/ws

Obsolete Parameters

The /id and /test parameters are removed in version 3.x.

Changed Parameters

The following table lists the startup parameters that moved from the startup batch file to the

INI file in version 3.x:

Old Parameter Name New Parameter Name Location in INI File

/db deb=n [SETTING]

/err err [PLUG-IN]

/if ifm [PLUG-IN]

/ifs ifs [PLUG-IN]

/output output [SETTING]

/pu purgetime [PLUG-IN]

/ren ren [PLUG-IN]

File Changes

In PI_UFL 3.x version, the configuration file not only defines the definitions for parsing the

messages, it also contains some of the interface’s start-up parameters. The [INTERFACE],

[PLUG-IN] and [SETTING] sections must be defined at the beginning of the configuration

file, followed by the [FIELD] or [MSG] sections.

Version 3.x has much stricter data type control. Time data types are a little different:

DateTime (formerly Time): Full timestamp

Time: Interval

Other important changes:

New function: Now()

New Annotation parameter for the StoreInPI() function, which now returns a

value indicating success or failure of the operation.

StoreInPIDST() is no longer supported.

New functionality for automatic tag and digital set/state creation. See the descriptions of

MSG(n).EPC and MSG(n).DigitalSet for details.

The IF/THEN logical operators have been added.

Error messages log can now be configured using the MSGInError keyword.

Installing the UFL Interface

8

Logic for renaming processed has changed. Files that cannot be read are assigned an

“Err” suffix.

Changes in Use of Point Attributes

Version 3.x uses the following point attributes differently from earlier versions:

Convers: Applied to numeric tags as a coefficient.

Location5: Set to 0 to enable exception reporting, 1 to replace the value at the same

timestamp, or 2 to add the value at the same timestamp.

Detailed Installation Procedure

To install the UFL interface, download and run its setup kit. By default, the UFL interface is

installed in the following location:

%PIPC%\Interfaces\PI_UFL\

The UFL interface installation directory contains all the files and folders required to

configure and run the UFL interface and includes example configurations.

Recommendation: Reserve the C: drive for the operating system, and install the interface on

another drive.

Configuring the UFL interface

To configure the UFL interface, you must:

Create and configure an interface instance

Create any required trusts

Configure the Windows service

To start the interface, you must create a valid INI file. For details, see Defining the

Configuration File.

The following sections describe these tasks in detail.

Creating and Configuring the Interface Instance

For each instance you create, settings are stored in a separate Windows command file (a .bat

file) in the UFL interface installation folder.

To create an instance of the UFL interface, perform the following steps:

1. Launch PI ICU.

2. Choose Interface > New from BAT file…

3. Browse to the folder where the UFL interface is installed, select PI_UFL.bat_new and

click Open. The Select PI Host Server dialog is displayed.

4. Specify your PI Server and click OK. ICU displays the settings for the new instance of

the UFL interface.

5. Edit the settings on the UFL tab as follows.


Field (Flag) Description

Configuration

File(/CF)

Browse to the INI file that specifies how to parse incoming data.

Send data to PI

Archive(/LB)

Bypass snapshot and write data directly to archive. Because data is written in bulk, throughput is faster but error reporting is less precise.

Archive Mode(/AM) ARCNOREPLACE: Do not add if event with same timestamp

exists.

ARCAPPEND: Add regardless of existing events with same

timestamp, with compression.

ARCREPLACE: Replace event with same timestamp.

ARCREPLACEX: Replace only if event with same timestamp

exists.

ARCDELETE: Remove existing event, do not replace.

ARCAPPENDX: Add event regardless of existing events, with

no compression.

Laboratory

Snapshots(/LBS)

Send events to snapshot in bulk, for efficiency.

Read Before

Overwrite(/RBO)

Overwrite existing event with same timestamp only if the new

value is different. Tag must have Location5 set to 1, and Send

data to PI Archive and Laboratory Snapshots must

be disabled

Use UTC

Timestamps(/UTC)

Incoming timestamp are UTC.

Ignore Missing

Tags(/IMT)

Do not issue errors if PI tag does not exist.

Tag Mask(/TM) Load matching points before running.

Default Error

Status(/DES)

Specifies status to be stored if the digital status string cannot be found in the tag’s digital state set. Specify the index for the desired system digital state.

Write Delay(/WD) How long, in milliseconds, to wait between bulk writes to the PI archive. Used for performance tuning. Default: 10 ms

Write Size(/WS) Maximum number of values writing in a single bulk write to PI archive. Used for performance tuning. Default: 10240

Additional

Parameters

Any additional startup parameters not supported by PI ICU

Creating Trusts

If you are installing the UFL interface on a node separate from the PI Server, you must create

trusts for the following:

UFL interface

PI Interface Configuration Utility (ICU)

PI SDK (if you are using PI annotations, automatic point configuration, or digital set or

digital state creation.)

PI SDK buffering (if running against a High Availability PI Server)

To create each of these trusts using PI System Management Tools, connect to the PI Server

and perform the following steps:

Installing the UFL Interface

10

1. Click Security and choose Mappings & Trusts.

2. On the Trusts tab, right-click and choose New Trust… The Add Trust wizard is

launched.

3. Specify a meaningful name and description.

4. Configure settings as follows:

Program Type of Trust Application Name

Buffering PI-API application APIBE (PI Buffer Server) or pibufss (SDK buffering)

PI ICU PI-API application PI-ICU.exe

UFL interface PI-API application PI_UE

UFL interface PI-SDK application PI_UFL.exe

The PI identity you specify for the trust must have permission to update the PI Module

Database. For maximum security, specify the network path using a fully-qualified node name

or IP address plus netmask 255.255.255.255

Verifying Successful Startup

To display the message log, launch PI SMT and choose the Operation > Message Logs

option. Using PI ICU, start the interface, then watch the log for messages indicating success

or errors.

Configuring the Windows Service

To ensure that the UFL interface instance starts whenever the interface node is rebooted,

configure the instance as a Windows service, as follows:

1. In PI ICU, click Service.

2. Set Startup Type to Auto.

3. Click the Create button.

4. To start the service, click .

To verify that the service is created and is running, use the Windows Services control panel.

The PI UFL interface does not support automated failover. However, to ensure that data

continues to be written if an instance of the interface stops running, you can configure

multiple instances of PI UFL to read the same data files or POP3 server. (Multiple instances

cannot read the same serial port.) Configure the instances identically and enable the Send

data to PI Archive (/LB) option so that data is written directly to the PI archive. For

efficiency, enable the Read before Overwrite /RBO option, which prevents an instance from

writing an already-written value.


Chapter 4. Configuring Diagnostics

To track the status and performance of the interface you can create the following types of

diagnostic points:

Health monitoring points: Track various status and performance-related data

I/O rate point: Reports the rate at which the interface is updating points

Interface status point: Indicates whether the interface is writing data to points

Performance counter points: Tracked by the PI Performance Monitor

Note that health monitoring do not require the PI Performance Monitor, and they provide

equivalent functionality. The following sections describe these diagnostics in more detail.

Health Monitoring Points

The interface includes a spreadsheet, PI_UFL_Sample_HealthPoints.xlsx, which

you can use to bulk-create health monitoring points using the OSIsoft PI Tag Configurator

Excel plug-in, which is available from the OSIsoft Download Center. To define health

monitoring points, open the spreadsheet and edit it as required for your configuration, then

use PI Tag Configurator to export the tags to your PI Server. The following health tags can be

created.

Health Tag Description

UI_DEVSTAT Status of the source device, as follows:

Starting: The interface has started, initialized the plug-in, and is

waiting for the first scan class.

Good: The interface is receiving data.

Intf Shutdown: The interface was shut down.

UI_HEARTBEAT Indicates whether the interface is running. The value of this point is an integer that increments continuously from 1 to 15, then resets to 1. If the value stops changing, the interface has stopped running. This point is updated at least every second and no more often than every minute.

UI_IORATE The sum of the number of scan-based input values that the interface collects before it performs exception reporting. The interface updates this point at the same frequency as the UI_HEARTBEAT point. A stale timestamp for this

point indicates that the interface has stopped collecting data.

UI_MSGCOUNT The number of messages that the interface has written to its log file since start-up. In general, a large number indicates that the interface is encountering problems or a high debug level is set.

Configuring Diagnostics

12

Health Tag Description

UI_SCBVRATE The number of system digital state values that the interface has collected. The interface updates the point after completing a scan.

UI_SCINDEVSCANTIME The amount of time (in milliseconds) that the interface takes to read data from the device and fill in the values for the tags. Normally, the value of a

UI_ SCINDEVSCANTIME point is a fraction of the corresponding

UI_SCINSCANTIME point value. You can use these values to calculate the

percentage of time that the interface spends communicating with the device compared with the percentage of time communicating with the PI Server. If scans are being skipped, the UI_SCINSCANTIME and UI_SCINSCANTIME

points can help you identify where the delay is occurring. The interface updates the value of this point at the completion of the scan.

UI_SCINFO Scan class information, returned as a string that indicates the number of

scan classes, the update frequency of the UI_HEARTBEAT health point, and

the scan class frequencies. The interface updates the value of this point at startup and at each performance summary interval.

UI_SCINSCANTIME The amount of time (in milliseconds) that the interface takes to read data from the device, fill in the values for the tags, and send the values to the PI Server. The interface updates the value of this point at the completion of the scan.

UI_SCIORATE The number of events that the interface has collected. If the current value of

this point does not exceed the value of the UI_SCPOINTCOUNT point, the

interface executed the scan successfully. A stale timestamp indicates that the tags for the scan class are not receiving new data. The interface updates the point after completing a scan.

UI_SCPOINTCOUNT The number of tags in a scan class. The interface updates this point when it performs the scan.

UI_SCSCANCOUNT The number of scans that the interface has performed. The interface updates the value of this point at the completion of the associated scan. The interface resets this point to zero after the interval configured by the /perf start-up parameter.

UI_SCSKIPPED The total number of scans that the interface was unable to perform before the scan time elapsed and the interface began the next scheduled scan. The interface updates this point each time it skips a scan. The interface resets this point to zero after the interval configured by the /perf start-up

parameter.

The following health points are not implemented for the PI UFL interface

UI_OUTPUTRATE

UI_OUTPUTBVRATE

UI_TRIGGERRATE

UI_TRIGGERBVRATE


I/O Rate Point

An I/O rate point measures the rate at which the interface writes data to its input tags. You

can create one I/O rate point for each interface instance. The point contains a ten-minute

average of the rate at which the interface sends values to the PI Server, specified as values per

minute. At startup and shutdown, the interface sets the point to zero. During operation, it

updates the point every ten minutes.

To create the point using PI ICU:

1. Go to the IO Rate tab.

2. Check Enable IORates for this interface.

3. Modify default settings if desired.

4. Click Create.

5. To enable the point for the interface instance, click Apply.log

6. To restart the interface, click the Restart toolbar button:

To verify that the point is tracking the I/O rate, check the log for IORATE messages, which

contain the following fields.

Field Description

Event Counter Correlates an active I/O rate tag tag file with this instance of the interface. (/EC)

In File Indicates whether the I/O rate tag and the event counter are active.

Snapshot The current value of the I/O rate tag.

Interface Status Point

The PI Interface Status Utility tracks whether an interface is writing data to the PI Server by

monitoring a watchdog tag, a PI tag that is updated regularly if the interface is operating

properly. If you have installed and configured PI ISU, you can use PI ICU to configure an

interface status point for the PI UFL interface.

To configure an interface status point using PI ICU:

1. In the Interface field, choose the desired instance of the UFL interface.

2. Right-click the tag list field, as shown in the following figure, and choose Create.


14

3. Select a watchdog tag: click the Tag Search button. Select a tag that is updated regularly

when the interface is operating correctly.

4. In the Scan frequency field, specify how often the watchdog tag is to be read. For

optimal performance, choose a scan frequency lower than the majority of the scan rates

for this interface’s points. For example, if this interface scans most of its points every 30

seconds, choose a scan frequency of 60 seconds.

5. If the Tag Status field displays a status of Incorrect, right click it and select Correct.

6. Click Create.

7. To update the startup batch file, click Apply.

Scan Class Performance Points

A scan class performance point measures how long it takes for the interface to complete a

scan. Low scan completion times (close to zero) indicate that the interface is performing

optimally. High scan completion times indicate an increased risk of missed or skipped scans.

To prevent missed or skipped scans, you can create more interface instances. Because the

UFL interface supports only one scan class, you cannot spread scanning using multiple scan

classes with offsets.

To define a performance point, create a numeric point, assign it the UFL point source, and set

its ExtendedDescriptor attribute to [PERFORMANCE_POINT].

Right-click here and choose Create


Performance Counter Points

If you use the PI Performance Monitor interface to track interface performance, you can

create performance counter points for the UFL interface using PI Tag Configurator.

To create the points using PI Tag Configurator, perform the following steps:

1. Launch Microsoft Excel, browse to the directory where the UFL interface is installed,

and open PI_UFL_Sample_PerformanceCounters.xlsx.

2. Edit the spreadsheet to replace <Machine> with the name of your PI Server node and

to make any other desired changes.

3. Choose Add-Ins > PI SMT > Export Tags. The tags are created.

4. Verify that the Performance Monitor interface detects and starts updating the tags (use PI

SMT to check current values).

Total and Scan Class Performance Counters

The following counters can be defined both for totals and for a specified scan class.

Performance Counter Tag Description

point_count The number of points per scan class or the total number for the interface instance.

sched_scans_%missed The percentage of scans the interface missed per Scan Class or the total number missed for all scan classes since startup. A missed scan occurs if the interface performs the scan one second later than scheduled.

sched_scans_%skipped The percentage of scans that the interface skipped since startup. A skipped scan is a scan that occurs at least one scan period after its scheduled time.

sched_scans_this_interval The number of scans that the interface performed per performance summary interval for the scan class or the total number of scans performed for all scan classes during the summary interval.

Total-Only Performance Counters

The following counters track totals for the interface instance.

Performance Counter Tag Description

Device_Actual_Connections The number of foreign devices currently connected and working properly. This value cannot exceed Device_Expected_Connections.

Device_Expected_Connections The total number of foreign device connections expected to be working properly at run time.

Device_Status Indicates whether the required number of connections to foreign devices (as specified by the /PercentUp command line flag)

has been made. Contains 0 if the device status is good, 1 if the device status is bad.

up_time The amount of time (in seconds) that the interface has been running. At startup the value of the counter is zero. If the point reaches the maximum value for an unsigned integer, it is reset to zero.


16

log_file_msg_count The number of messages that the interface has written to the log file.

PI_Status Contains 0 if the interface is communicating with the PI Server or 1 if the interface cannot communicate with the server.

pts_added_to_interface The number of points added to the interface after the last successful startup

pts_edited_in_interface The number of edits to points for the interface’s point source and instance ID since last interface startup.

Points_Good The number of points that have sent a good current value to the PI Server since last interface startup.

Points_In_Error The number of points that have sent a system digital state value as a current value to the PI Server, indicating that a vlaue is not good.

pts_removed_from_interface The number of points that have been removed from the interface configuration.

Points_Stale_10min The number of good points that have not received a new value in the last ten minutes.

Points_Stale_30min The number of points that have not received a new value in the last thirty minutes.

Points_Stale_60min The number of points that have not received a new value in the last sixty minutes.

Points_Stale_240min The number of points that have not received a new value in the last 240 minutes (four hours).

The stale point counts are inclusive. For example, the ten-minute count includes all stale

points, the 30-minute count includes the 60-minute count, and so on.

Scan Class-Only Performance Counters

The following counters can be defined for specific scan classes.

Performance Counter Tag

Description

Device_Scan_Time The number of milliseconds that the interface takes to read the data from the foreign device and package the data to send to the PI Server. Does not include the amount of time required to send the data to the PI Server.

scan_time The number of milliseconds that the interface takes to read the data from the device and send the data to the PI Server.


Chapter 5. Loading Data into PI

To specify how the UFL interface parses incoming data, you create a configuration file (also

referred to as an INI file because of its .ini extension). The configuration file is composed of

sections that define how the incoming data is parsed into fields, processed, and written to PI

tags. By default, example files are installed in the following locations:

PIHOME\Interfaces\PI_UFL\Examples

The configuration file also specifies the nature of the data source, line endings, logging, and

other run-time settings. The following sections explain configuration files in detail. For

examples of the features and functions, refer to the examples appendix in this guide and to the

example files installed with the UFL interface.

Defining the Configuration File

Regardless of its source, incoming data is treated as a set of consistently-formatted lines,

referred to as messages. Messages are parsed into fields, which are assigned data types and, if

required, formats that describe the layout of the field (for example, for incoming DateTime

fields).

Blank lines are ignored. To embed comments, precede text with a straight quote (ASCII 39).

For example:

'------------------------------------------------------------

' Get QUANTITY DETAILS

'------------------------------------------------------------

To continue a line, end it with an underscore (ASCII 95). For example:

message1.filter = C1=="Line containing *" And _

C56=="DateTime*"

The maximum line length supported by the PI UFL interface is 10K (10240) characters.

The configuration file is composed of the following sections:

INTERFACE: Specifies the plug-in to be used to process incoming data, according to the

type of data (text file, serial port or POP3 server).

PLUG-IN: Specifies plug-in-specific settings.

SETTING: Configures logging settings, locale, and other operational settings.

FIELD: Defines and declares data types for the individual fields that receive data when

messages are parsed.

MSG: Defines the types of incoming messages to be parsed, assigning a name that is used

to define the section where the message is parsed.

Platform Details and Features

18

Per-message sections: For each message defined in the MSG section, these sections filter

incoming messages, parse them into fields, process the fields and write results to PI tags.

These sections can contain processing logic, redirecting to other sections and skipping

lines from the input stream.

For example, the following configuration file specifies the logic required to process a set of

text files that contain three types of messages.

[INTERFACE]

PLUG-IN=AsciiFiles.dll

[PLUG-IN]

IFM=D:\PIPC\Interfaces\PI_UFL\data\*.txt

ERR=BAD

IFS=N

NEWLINE=13,10

PURGETIME=10s

REN=_OK

[SETTING]

DEB=4

MAXLOG=10

MAXLOGSIZE=10

MSGINERROR=D:\PIPC\Interfaces\PI_UFL\logs\simple2.err

OUTPUT=D:\PIPC\Interfaces\PI_UFL\logs\simple2.log

LOCALE=en-us

[FIELD]

FIELD(1).NAME="Timestamp"

FIELD(1).TYPE="DateTime"

FIELD(1).FORMAT="dd-MMM-yyyy hh:mm:ss"

FIELD(2).NAME="value"

FIELD(2).TYPE="Number"

FIELD(3).NAME="Tag"

FIELD(4).NAME="Unit"

[MSG]

MSG(1).NAME="MSG_1"

MSG(2).NAME="MSG_2"

MSG(2).EPC="Float32" ' Automatically create tags

[MSG_1]

MSG_1.FILTER=C1=="HEADING 2:*"

Unit = ["* Unit (*) *"] ' Global variable Unit

[MSG_2]

MSG_2.FILTER=C1=="Date:*"

Timestamp=["Date: (*) Value: *"]

Value = ["* Value: (*) Tag: *"]

Tag = ["* Tag: (*)"]

' Tag = C43-C51 'Alternatively, Tag name starts at column 43

and ends at column 51

IF (Unit IS NOT NULL) THEN

Tag = Unit & "_" & Tag

StoreInPI(Tag,,Timestamp,Value,,)


ELSE

Print("Unit not read")

ENDIF

The following sections describe each configuration file section in detail.

[INTERFACE] Section

This section specifies the plug-in required to process incoming data. For each type of data

source, the PI UFL provides a plug-in in the form of a DLL containing the required logic. The

following plug-ins are provided:

ASCIIFiles.dll

Serial.dll

POP3.dll

Specify the plug-in as follows:

Plug-In = ASCIIFiles.dll

The UFL interface also includes logic for processing files in the format accepted by the PI

Batch File interface.

[PLUG-IN] Section

This section contains plug-in-specific settings. The following table describes the settings.

ASCII File Settings

Setting Description

ERR File extension to be assigned to files that caused errors during processing. The default error suffix is ERR.

Example:

ERR = BAD

IFM The input files to be processed. When the interface runs as NT service and the data files reside on a network drive, use UNC paths to specify the location.

Example:

IFM = C:\myfolder\Data\data*.txt

IFM = \\mynode\myshare\Data\*.txt

IFS=c Specifies the order in which input files are processed. Valid arguments are:

C: Creation date (default)

M: Modification date

N: File Name

Example:

IFS=C

NEWLINE Specifies line-end character(s). Default is CRLF (ASCII 13 and 10).

Example:

NEWLINE = "STOP" OR "END"

PFN Prepend File Name. To include the filename as the first line in the input stream, set to

TRUE. Default value is FALSE.


20

PFN_PREFIX

Used with the PFN keyword, precedes the filename with specified text.

Example:

PFN_PREFIX = "DATAFILE: "

PURGETIME

Specify the amount of time to wait before purging processed data files. The time specified is relative to the time on the interface node and is compared against the to-be-purged file processed time. Default is one day (1d). The minimum value is 1s (one second). Specify time as follows:

s seconds

m minutes

h hours

d days

Only renamed files that were processed without error are purged.

Example:

PURGETIME = 10m

RBP Rename Before Processing. The interface (optionally) renames the file, still on the source drive (referenced through the IFM keyword). This is to avoid duplicate processing of the same file in redundant scenarios ( two UFL instances working against the same source directory ).

Example:

RBP = TempFile1.txt

REN

Specifies the extension to be assigned to successfully-processed input files. Default is “_OK”. In addition, the file is assigned a suffix specifying the date and time it was

processed. The format of the date-time suffix is dd MMM yyyy_hh mm ss.nnn. The

format of the date-time suffix cannot be configured. Note that the renaming scheme changed after version 2.x of the UFL interface.

Example:

REN = SUCC

WORDWRAP

Breaks input line into lines of specified length. Overrides NEWLINE, if specified. Maximum setting is 10,240.

Example:

WORDWRAP = 11

Data file contents:

TagName1 1 TagName2 2 TagName3 3 TagName4 4

Resulting lines:

TagName1 1

TagName2 2

TagName3 3

TagName4 4

When specifying the NEWLINE clause, note the following:

If the lines in the input file are terminated in more than one way, specify the line endings

in double quotes and use the OR clause to specify valid line endings.

To specify ASCII line endings, use comma-separated number with no white space (for

example, 13,10).

You can specify line endings as ASCII values or strings, but not both. For example,

NEWLINE = "event end> 13,10" is invalid.

String comparisons are case sensitive.


Serial Port Settings

Setting Description

BITS Number of bits. Valid values: 4,5,6,7,8. Default value is 8.

Example:

BITS = 8

COM The serial port number. Default value is 1.

Example:

COM = 2

COMDATA Full path to a file in which the interface stores raw data read from the serial port. Intended for troubleshooting.

Example:

COMDATA = c:\UFLLogs\rawdata.txt

NEWLINE Specifies line-end character(s). Default is CRLF (ASCII 13 and 10). (Same considerations as ASCII plug-in.)

Example:

NEWLINE = "STOP" OR "END"

PARITY Specifies parity of incoming data. Valid values:

EVEN

ODD

NO (default)

MARK

SPACE

SPEED Port BAUD. Default is 9600 bps.

STOPBITS Number of stop-bits. Valid values:

0: 1 stop bit (default)

1: 1.5 stop bit

2: 2 stop bits

POP3 Settings

Setting Description

ATTACHMENT_PREFIX For use with MAIL_ATTACHMENT. Specifies text to be prepended to the

contents of the attachment. Default is [Attachment]:

Example:

ATTACHMENT_PREFIX = [Message Attachment]:

BODY_PREFIX For use with MAIL_BODY. Specifies text to be prepended to the contents of the email body. Default is [Body]:

Example:

BODY_PREFIX = [Message Body]:

DATE_PREFIX For use with MAIL_DATE. Specifies text to be prepended to the date. Default pattern is [Date]:

Example:

DATE_PREFIX= [Message Date]:


22

FILTER_FROM Process emails from specified address(es). Emails from other sources is ignored (but can be forwarded to the backup address). Separate multiple address using semicolons. If you omit this clause, all email from the specified server is processed.]:

Example:

FILTER_FROM = [email protected];[email protected]

FORWARD_TO Specify a backup email address. Useful when emails need to be available after being processed or in case of errors. The SMTP server and port number (through which the email is forwarded) are specified using the keywords SMTP_SERVER and SMTP_PORT. By default, email is not forwarded. If you omit the email address, email is forwarded to the sender.

Example:

FORWARD_TO= [email protected]

FORWARD_AS_UFLSTREAM Enables email forwarding. By default, email is not forwarded.

Example:

FORWARD_AS_UFLSTREAM = True

FROM_PREFIX For use with MAIL_FROM. Specifies text to be prepended to the sender. Default pattern is [From]:

Example:

FROM_PREFIX = [Message From]:

MAIL_ATTACHMENT To disable processing of attachments, set to FALSE. By default, attachments are processed.

Example:

MAIL_ATTACHMENT = True

MAIL_BODY To disable processing of the body of the email, set to FALSE. By default, email body is processed.

Example:

MAIL_BODY = True

MAIL_DATE Prepend Date. By default, the date when the email was sent is prepended to the beginning of the email body. To disable this feature, set to FALSE.

Example:

MAIL_DATE = False

MAIL_FROM Prepend Sender. By default, the sender’s email address is prepended to the beginning of the email body. To disable this feature, set to FALSE.]:

Example:

MAIL_FROM = False

MAIL_SUBJECT Prepend Subject. By default, the subject is prepended to the beginning of the email body. To disable this feature, set to FALSE.

Example:

MAIL_SUBJECT = False

PFN Prepend File Name. To include the filename of the attachment as the first line in the attachment, set to TRUE.

To assist parsing, the filename can be prefixed with a specified string pattern. Default value is FALSE.

PFN_PREFIX

Used with the PFN keyword, precedes the filename in the attachment with specified text.

Example:

PFN_PREFIX = "ATTACHED FILE: "

mailto:[email protected]


POP3_COMMAND_WAIT Number of millisecond to wait for the POP3 answer. Default 500 ms. Applicable when the POP3 server response times are long.

POP3_PASSWORD Specify the password for the POP3 user.

Example:

POP3_COMMAND_WAIT = 1000

POP3_PORT Specify the port number of the POP3 server. Default value is 110.

Example:

POP3_PORT = 110

POP3_SERVER Address of the POP3 server. You must specify either the direct IP address or the name of the POP3 server. Default value is localhost.

Example:

POP3_SERVER = mail.osisoft.com

POP3_USER Mandatory: Email account on the POP3 server.

Example:

POP3_USER = ufl

SMTP_PORT Specify the port number of the SMTP server. Default value is 25.

Example:

SMTP_PORT = 25

SMTP_SERVER Name or IP address of the SMTP server to which email is forwarded. See

FORWARD_TO for more details. If you omit this parameter, mail is

forwarded to the POP3 server from which email is being read.

Example:

SMTP_SERVER = mail.osisoft.com

SUBJECT_PREFIX For use with the MAIL_SUBJECT keyword. Specifies a string to be prepended to the subject line of incoming email. Default value is [Subject]:

POP3 Passwords

To access the POP3 email server, the interface requires the email user’s password. You can

specify the password in the configuration file using the POP3_PASSWORD parameter, but

specifying a password in clear text is insecure. As an alternative, you can run the interface

interactively and enter the password when prompted. The interface encrypts the password and

stores it in a file named POP3.PWD, in the same directory as the configuration file. After the

file has been created, you can run the interface as a service, and it will read the password

from the file.


24

BatchFL Settings

Setting Description

ADJUST Specifies the number of minutes to adjust the timestamp. For example, to add an hour to the timestamp, specify 60. To subtract an hour, specify -60. By default, timestamps are not adjusted.

Example:

ADJUST = 60

ALIAS = [E | I] The data file specifies an alias instead of a PI tag name. If ALIAS is specified, you must specify a point source (/PS). The interface searches

for the alias in the Extended Descriptor or Instrument Tag field

of points with the specified point source.

Valid values are E (Extended Descriptor) or I (Instrument Tag). By default, the interface uses the tag name.

Example:

ALIAS = E

DATETIME_FORMAT Specify time string format. See Date/Time Format on page 28 for

details.

Example:

DATETIME_FORMAT = dd-MMM-yyyy hh:mm:ss

DATETIME_MONTH_FORMAT Specify month format. See Date/Time Format on page 28 for details.

Example:

DATETIME_MONTH_FORMAT = Jan,Feb,Mar,Apr,May,Jun,

Jul,Aug,Sep,Oct,Nov,Dec

DIGITAL_SET If the POINT_TYPE is Digital, you must specify the name of an existing digital state set name. Default is System.

Example:

DIGITAL_SET= My_Digital_Set

ERR File extension to be assigned to files that caused errors during processing. The default error suffix is ERR.

Example:

Err = BAD

FIELD_SEPARATOR Specifies the field separator between tag name and timestamp, and timestamp and value. Default separator is a comma.

Example:

FIELD_SEPARATOR = |

IFM The input files to be processed. When the interface runs as a service and the data files reside on a network drive, use UNC paths to specify the location.

Example:

IFM = C:\myfolder\Data\data*.txt

IFM = \\mynode\myshare\Data\*.txt


IFS=c Specifies the order in which input files are processed. Valid arguments are:

C: Creation date (default)

M: Modification date

N: File Name

Example:

IFS=C

POINT_TYPE Specifies the data type to be used when the interface attempts to create a previously undefined point. By default, undefined points are not automatically created.

Example:

POINT_TYPE = Float32

PURGETIME

Specify the amount of time to wait before purging processed data files. The time specified is relative to the time on the interface node and is compared against the to-be-purged file processed time. Default is one day (1d). The minimum value is 1s (one second). Specify time as follows:

s seconds

m minutes

h hours

d days

Only files that were processed without error are purged.

Example:

PURGETIME = 10m

REMOVE_BLANKS By default, leading and trailing blanks are trimmed from strings. To disable trimming, set this option to TRUE.

Example:

REMOVE_BLANKS = True

REN

Specifies the extension to be assigned to successfully-processed input files. Default is “_OK”. In addition, the file is assigned a suffix specifying the date and time it was processed. The format of the date-time suffix is

dd MMM yyyy_hh mm ss.nnn. The format of the date-time suffix

cannot be configured. Note that the renaming scheme changed after version 2.x of the UFL interface.

Example:

REN = SUCC

SCALE Enable scaling: read the UserReal1 point attribute and use it to multiply the value in the data file. For numeric points only. By default, no scaling is performed.

SLEEP Specifies the number of seconds to pause between processing files, to throttle the rate at which the data files get processed. By default, there is no delay between files.

Example:

SLEEP = 10


26

[SETTING] Section

This section specifies various operational settings, described in detail below.

Setting Description

DEB Debug level. The interface maintains its own log file, where it redirects all kinds of messages, errors, as well as debug, or information messages (see the description of the OUTPUT keyword below). The higher the debug level the more detailed is the printout.

Example:

DEB = 4

LOCALE Specifies how the interface transforms the string representation of numbers to the native numeric form; that is, which locale it uses. The default Locale is English – United States. Different decimal separators can be handled. The list of all locale codes can be found at:

http://msdn2.microsoft.com/en-us/library/0h88fahh.aspx

You can specify locale using the long form, short form, or numeric identifier (LCID).

Example:

LOCALE = "German – Germany"

LOCALE = "de-de"

LOCALE = 1031

MAXLOG Maximum number of log files. The interface starts overwriting the oldest log files when MAXLOG has been reached. By default, log files are not overwritten.

Example:

MAXLOG = 10

MAXLOGSIZE Maximum size of log files in MB. When a log file reaches the specified size, the interface creates a new log file. Default is 20 MB. (In version 3.0.3.16 and previous versions, maximum size was 2G.)

Example:

MAXLOGSIZE = 10

MSGINERROR Specify the path and filename for the log file containing lines that were not successfully processed.

Example:

MSGINERROR = c:\ufl_logs\errors.txt

OUTPUT Specify the path and filename for the interface log file, which contains debugging output. Each time the interface starts, it appends a version number to any existing log and creates a new one. If you omit this option, errors are logged in the PI message log.

Example:

OUTPUT = c:\ufl_logs\PI_UFL.log


The following table describes debugging levels. Higher levels include the information from

lower levels. For typical operation, choose level 0 or 1, because higher levels can increase log

file size significantly.

Debug Level What’s Logged

0 (Default) No debug output.

1 Tasks that are normally performed once, such as startup and shutdown messages, points added to the interface’s cache, etc.

2 More detail than level 1.

3 Include raw data.

4 Include data about to be sent to PI Server.

5 Include read scan cycles start and end time, interface internal cache refresh cycles starts and ends times, etc.

6 Log lines read from input before processing.

[FIELD] Section

The statements in the section assign fields a name and data type. For date/time fields, you

must specify the format used by incoming data. The [FIELD] section is mandatory and must

follow the [INTERFACE], [PLUG-IN] and [SETTING] sections.

To assign field attributes, use “dot” notation, as follows:

FIELD(n).Name = "Valid-Field-Name"

FIELD(n).Type = Data-Type

FIELD(n).Format = DateTime Format

Field Names

To ensure that your configuration file is readable, assign descriptive names to incoming

fields. Field names must be composed of alphanumeric characters only (no punctuation). For

clarity, avoid assigning names that might be confused with the interface’s reserved words

(such as "FIELD", "MSG", "TIME", etc.). In addition, do not use the following characters in

field names: ` ' " * ? ; { } [ ] | \ . .

Data Types

The following types are supported:

DateTime (replaces Time type in version 2.x releases)

Time (duration as opposed to date/time)

String (default)

Int32 (integer type)

Number (float type)

DSTFlag (0 if standard time, 1 if Daylight Saving Time)

Values in strings are cast to numbers according to the LOCALE setting. Scientific

(exponential) notation is recognized.


28

Date/Time Format

To specify the format of incoming date/time and time (duration) fields, define a format string

in the form:

InputTimeFieldName.Format = "format" [, "monthlist]"

Enclose the format definition in double quotes. For example:

InputTimestamp.Format = "dd-MMM-yy hh:mm:ss", _

"Jan,Feb,Mar,Apr,May,Jun,Jul,Aug,Sep,Oct,Nov,Dec"

Use the following tokens:

Token Description

yy Year, two digits.

yyyy Year, four digits.

MM Month, two digits.

M Month, one or two digits.

MMM Month, in string format. Default is standard English three-character abbreviations, unless overridden using the optional month list parameter.

dd Day of the month, two digits.

d Day of the month, one or two digits.

hh Hour, two digits. By default a 24-hour clock is assumed, unless p or pp is used to specify

AM/PM.

h Hour, one or two digits.

m Minutes, one or two digits.

mm Minutes, two digits.

s Seconds, one or two digits.

ss Seconds, two digits.

n Tenths of a second.

nn Hundredths of a second

nnn Milliseconds

p A/P for AM/PM. In this case a 12-hour clock is assumed.

pp AM/PM. In this case a 12-hour clock is assumed.

Note: Month abbreviations must be comma delimited. The timestamp format string comparison is case-sensitive. Other evaluations are not case-sensitive.

The following standard formats are supported:

Format Description

SECONDS_GMT Number of seconds since 1970, in Universal Time Coordinated (UTC)

SECONDS_LOCAL Number of seconds since 1970, in local time.


Examples:

InputTimestamp.Format = "dd-MMM-yy hh:mm:ss",_


'German months

InputTimestamp.Format = "dd-MMM-yy hh:mm:ss", _

"Jan,Feb,Mär,Apr,Mai,Jun,Jul,Aug,Sep,Okt,Nov,Dez"

InputTimestamp.Format = SECONDS_GMT

Daylight Saving Time Flag

If the input contains an indicator for daylight saving time, define a DSTFlag field to detect

whether the input date/time field must be adjusted. The input field must be binary in nature.

In other words, it must assume one of two values, for example, “winter” or “summer,” or

“DST” or “Standard”. Default values are 0 and 1. When the field is parsed, the DSTTime

field is set to 0 for standard time or 1 for daylight saving time.

To define the format, specify the values as “standard,DST” as shown in the following

example:

FIELD(3).Name = "DSTField"

DSTField.Type = "DSTFlag"

DSTField.Format = "winter,summer"

To apply the value, evaluate the field as shown in the following example:

DSTOffset = "01:00:00"

If(DSTFlag == 1) Then

TimeStamp = TimeStamp – DSTOffset

EndIf

[MSG] Section

The PI_UFL interface filters incoming data to catch messages, which are assigned names.

For each message name you assign, you use the name to define a section that parses and

processes the message. For messages containing tag names, you can enable point creation and

specify the point’s data type or another point from which the new point inherits its data type

and attributes. If the point does not exist, the UFL interface creates it.

Assign names to messages as follows:

[MSG]

MSG(n).NAME = "MessageName"

For example:

MSG(1).Name = "HEADER"

MSG(2).Name = "DATA LINE"

Message names can contain alphanumeric characters, spaces and underscores. Names are not

case-sensitive. Names containing spaces must be specified in double quotes.


30

Point Creation

The EPC attribute enables point creation and specifies the data type of the point to be created.

If a field in the input message contains the name of a tag that does not exist, the tag is created.

The following PI point data types are supported:

Int16

Int32

Float16

Float32

Float64

Digital

String

Timestamp

For example:

[MSG]

MSG(1).Name = "Title"

MSG(1).EPC = "Float32"

As an alternative to assigning a data type, you can specify a tag from which a newly-created

tag inherits its data type by setting the EPC_INHERIT attribute. For example:

MSG(1).EPC_INHERIT = "Sinusoid"

By default, when creating digital tags, the UFL interface creates a state set and assigns it the

name of the tag plus “_SET” (unless the state set already exists). To override the default

name, set the DigitalSet attribute; for example:

MSG(1).DIGITALSET = "UFL"

You cannot use both MSG(n).EPC and MSG(n).EPC_Inherit for the same message

type.

If the UFL interface attempts to create a point and a point with the same name but a different

point source already exists, the following error is logged:

[-10550] Tag Already Exists in Table

The preceding error might also be logged if the UFL interface user has not been granted Read

privilege for an existing point, because the interface cannot check whether the point exists

before attempting to create it.

Parsing Messages Into Fields

For each message you define in the [MSG] section, you must define a section for parsing and

processing the contents of the message. The section heading is the name of the message or its

entry in the message array (for example, MSG(1)).


Filtering

To filter the desired messages, set the FILTER attribute. Messages are filtered by matching

text. Messages are processed only by the first section that catches them, so the order in which

you define message-processing sections is significant. For more control over the order in

which filtering is performed, use the SetNextMsg() action. For details, see

SetNextMsg() on page 58.

Filtering is case-sensitive. To define filters based on the contents of a message, use the

following tokens.

Token Matches…

? Any single character

* Zero or more characters

# Any single digit (0 — 9)

[character list] Any single character in character list. Must be enclosed in square brackets.

[!character list] Any single character not in the character list. Must be enclosed in square brackets.

\ To match characters that are used for filter tokens, for example, question marks, precede the character with a backslash.

The following example filter catches messages where the following is true:

The line doesn’t start with an exclamation point.

Starting at position 10, the line contains the text “TAG” followed by any number of

characters.

Starting at position 30, the line contains the text “VALUE” followed by any number of

characters.

MSG(1).Filter = NOT C1=="!*" AND C10=="TAG*" AND C30=="VALUE*"

The following data line matches the filter criteria:

' 1234...TAG=mytag............VALUE=10.0

The following filter catches lines that start with “State.City.A”, “ State.City.B”, or

“State.City.C”.

MSG(2).Filter = C1=="State.City.[ABC].*"

The next filter catches lines that do not begin “State.City.D”, “ State.City.E”, or

“State.City.F”.

MSG(3).Filter = C1=="State.City.[!DEF].*"

Extracting Data from Messages Into Fields

You parse a message into fields based on the position of the field, using masks and wildcards

to extract the desired data. For example, suppose your input is a .csv file containing data lines

like the following:

VIC1,"2008/02/21 16:00:00",7271.92,46.41,TRADE

The following statements extract data from the message and assign it to fields. Note the use

of the backslash to escape the double-quote delimiter:


32

LOCATION = ["(*),*,*,*,*"]

INTERVALDDATE = ["*,\"(*)\",*,*,*"]

DEMAND = ["*,*,(*),*,*"]

PRICE = ["*,*,*,(*),*"]

PERIODTYPE = ["*,*,*,*,(*)"]

To extract data based on starting and ending position, specify the range using the format “Cn

– Cn”. For more flexibility, you can use masks and wildcards in conjunction with position

specifiers. Following are examples of extracting data from messages into fields.

Example Description

FIELD(1) = C1 – C10 Extract the first ten characters from the input line

FIELD(2) = C11 – C11(",") Extract the field that starts at character 11 and ends before the next comma.

FIELD(3) = C11(",") – (",") Extract the field that starts after the first comma after character 11 and ends before the next comma.

FIELD(4) = C31 – C41("[;,:]") Extract the characters starting at position 31 up to (but not including) the first semi-colon, comma, or colon after position 41

FIELD(5) = C51 – C51("[!0123456789]") Extract characters starting at position 51 up to ' (but not including) the first non-numeric character after position 51

The following table describes the characters you use to specify masks.

Character Matches

? Any single character

* Zero or more characters

# Any single digit (0 — 9)

[character string] Any single character in character string. Must be enclosed in square brackets

[!character string] Any single character not in character string. Must be enclosed in square brackets.

( ) Indicates the data to be extracted into the field.

\ To match characters that are used for filter tokens, for example, question marks, precede the character with a backslash.

The combination of wildcards and parentheses is a highly flexible approach to extracting

fields from messages. For example, to extract fields from a comma-separated values (csv) file

containing data formatted as follows:

TagName, Timestamp, Value “Status”

You can use statements like the following, without needing to specify the numeric start and

end positions of the fields:

FIELD(1) = ["(*),*,*\"*\""]

FIELD(2) = ["*,(*),*\"*\""]

FIELD(3) = ["*,*,(*)\"*\""]


To extract a double-quoted field and strip the quotes, use backslash to escape the quotes:

FIELD(4) = ["*,*,*\"(*)\""]

Performing Calculations Using Fields

You can perform calculations using fields by specifying expressions in the configuration file.

The resulting value of an expression on the right hand side (of an assignment) is stored into

the field on the left hand side. The data types of all operands in the expression on the

assignment’s right hand side are implicitly converted as needed. For example, when two

operands are added using a “+” operator, both operands are interpreted as numbers.

Fields are NULL when declared; that is, their value is undefined.

The following sections describe the operations you can perform on data in fields. For

examples, see the example files installed with the interface and the example appendix in this

guide.

Arithmetic and Logical Operators

Operator Meaning Data Types Operands

* / Multiply and Divide Number,

Time

+ - Add and Subtract. Number,

DateTime,

Time

& String concatenation. String

AND Logical AND. Returns 1 if both operands are non-zero, else returns 0.

Number

OR Logical OR. Returns 1 if either operand is non-zero, else returns 0.

Number

Note: You can perform date calculations using arithmetic operators. For details, see Date Math on page 55.

Mathematical Functions

Operator Description Data Type of Operands

ABS Absolute value. Number ABS(x Number)

ACOS, ASIN, ATAN,

ATAN2, COS, COSH,

SIN, SINH, TAN,

TANH

Trigonometric functions.

Return value is in radians.

Number ACOS(x Number)

…

Number ATAN2(x Number, y Number)

CEILING Rounds a number with a fractional portion to the next highest integer.

Number CEILING(x Number)

EXP Exponential value. Number EXP(x Number)


34

FLOOR Largest integer less than or equal to the given numeric expression.

Number FLOOR(x Number)

LOG, LOG10 Logarithmic value. Number LOG(x Number)

PI 3.14 Number PI()

ROUND Round the value. Number ROUND(x Number)

String Functions

Operator Description Data Types Operands

CONCAT Concatenate two strings. String CONCAT(x String, y String)

INSTR Returns the position of the given occurrence of a specified substring. Positions start with 1. Returns 0 if specified substring is not found.

Int INSTR(x String, substring String,

start Int, occurrence Int)

LOWER All characters lower-case. String LOWER (x String)

LEFT Returns the leftmost n characters. String LEFT(x String, n Int)

LEN Number of characters excluding leading and trailing blanks. Int LEN (x String)

LTRIM Trim the leading blanks. String LTRIM (x String)

REPLACE Find the specified string and replace it with the third parameter.

String REPLACE (x String, findWhat String, replaceWith String)

RIGHT Returns the rightmost n characters. String RIGHT(x String, n Int)

RTRIM Trim the trailing blanks. String RTRIM (x String)

SPACE Character string consisting of n spaces. String SPACE (n Int)

SUBSTR String consisting of len characters starting at start position. String SUBSTR(x String, start Int, len Int)

TRIM Trim leading and trailing blanks. String TRIM (x String)

UPPER All characters upper-case. String UPPER (x String)

DateTime and Time Functions

The following functions extract a portion of a datetime or time value.

Operator Data Type of Operands

DAY Int32 DAY(x DateTime)

FRACTION

(Extracts the subseconds)

Float64 FRACTION(x DateTime)

Float64 FRACTION(x Time)

HOUR Int32 HOUR(x DateTime)

Int32 HOUR(x Time)

MINUTE Int32 MINUTE(x DateTime)

Int32 MINUTE(x Time)

MONTH Int32 MONTH(x DateTime)

MONTHNAME String MONTHNAME(x DateTime)


Operator Data Type of Operands

SECOND Int32 SECOND(x DateTime)

Int32 SECOND(x Time)

WEEK Int32 WEEK(x DateTime)

YEAR Int32 YEAR(x DateTime)

IF Statement

To define processing logic, specify the IF statement as follows:

IF <condition> THEN <expression(s)> [ELSE <expression(s)>]

ENDIF

Specify the condition as follows:

{[NOT] <predicate> | (<condition>)}

[{AND | OR} <condition>]

[, …]

Within the condition clause, specify the predicate as follows:

<predicate> ::=

<expression> { == | > | < | >= | <= | <> | != } <expression>

In the predicate, specify the expression as follows:

<expression> IS [NOT] NULL

Manipulating Data Using Actions

To manipulate the contents of fields, you specify actions using dot notation as follows:

MSG(n).Action = ActionName (Parameters)

You specify actions in the message-specific section that catches (filters) and processes

messages. The following sections describe the actions. For examples, see the example files

installed with the interface and the examples appendix in this guide

Action Description

DateTimeFromJulian(n) Converts a numeric Julian date to a PI timestamp. A Julian date represents an interval of time as days and fractions of a day since January 1. 4713 BC Greenwich noon.

DigCode(system_digital_state) Returns the number corresponding to the specified system digital state (string).

Now() Returns the current local timestamp in datetime format. For all messages read from a file, Now() returns the

same timestamp, the time when the file is opened for reading.

NowUTC() Returns the current local timestamp in UTC format. For messages from a file, NowUTC() returns the same

timestamp, the time when the file is opened for reading. .


36

NumberFromHex(HexNumString) Converts a string containing a hexadecimal number to decimal.

Print("string")

Print(fieldname)

Print specified string or value of specified field to log file.

SetNextMsg (MSG[, NumberOfMsgs]) Change order in which filters are applied to messages. By default, filters are applied in the order in which they are specified in the configuration file. This action redirects the specified number of following messages to the filter for the specified message. If you omit the number of messages, all subsequent messages are directed to the specified filter.

SkipFile()

Skip the rest of the lines that arrived in a batch of input stream lines, for example in a data file. SkipFile()

can be used when a certain message indicates that the

incoming data is invalid.

SkipLines(n) Skip the specified number of lines from the input stream.

Storing Data in PI Tags

The StoreInPI action sends the following to a specified PI tag:

Timestamp

Value

Status

Questionable flag

Annotation

The syntax for this action is as follows:

StoreInPI (Tag, InstrumentTag, Timestamp, Value, [Status], _

[Questionable][, Annotation])

Parameters for this action are as follows:

Parameter Description

Tag Specifies the target tag

InstrumentTag Alias for target tag.

Timestamp

Timestamp to be recorded with the value. Must be a valid datetime value, accurate for the interface node system time. If you omit this parameter, the current system time is recorded.

Value

The value to be recorded in the target point. For digital tags, you can specify the state as a string or its corresponding numeric value as an integer.

Status

(optional)

Tag status specified as a number corresponding to an entry in the PI System digital set. Set to 0 for valid values, positive value for digital tags, negative for all other data types.

Questionable

(optional)

0 for valid values, 1 if there is an issue with the quality of the value being sent.


Annotation

(optional)

Annotation to be recorded with this value. The annotation is stored using the PI variant data type that best corresponds to the type declared for the field, as follows:

String: VT_BSTR

Number:VT_R8

DateTime: VT_DATE

Return Value Returns 0 if the operation was successful, otherwise it returns an error code from the corresponding PI API or PI SDK call. To store the return value in a field, use the following syntax:

Result = StoreInPI( Tag, ,Timestamp, Value, , )

If you attempt to write a NULL value to a tag, the value that is written depends on the data

type of the tag, as follows:

Tag Data Type Value Written for NULL

String or Digital Empty string

Float16,Float32,Float64,Int32 0

Timestamp 01-Jan-1970

If you require different handling for NULL values, define the desired logic in the

configuration file.

Parsing Standard File Types

The following sections tell you how to define the logic required to parse common data

formats. There is no difference between the logic required to process ASCII data from a serial

port and ASCII data from a text file.

CSV Files

To extract fields from a .csv (comma-separated values) file, use the comma as the field

delimiter. For example:

TagName = ["(*),*,*"]

Timestamp = ["*,(*),*"]

Value = ["*,*,(*)"]

XML Files

NOTE: The PI UFL interface can process XML files, but for applications that are best addressed using the OPC XML-DA specification, use the PI Interface for OPC DA XML.

To process an XML file that is not reliably terminated with standard line endings, use the

NEWLINE keyword to specify terminating tags. For example:

NEWLINE = "</TZ>" OR "</TS>" OR "</PV>"

Because XML files are highly structured, you can control processing sequence reliably using

the SetNextMsg action.


38

To parse incoming XML data like the following, break the data into messages based on tags,

as shown in the configuration file that follows.

Example Incoming XML Data

<?xml version=”1.0” encoding=”UTF-8” ?>

<MS ID=”EXAMPLE”>

<MP UOM=”KG/H” FCSID=”36”>

<TZ>GMT+1</TZ>

<M Q=”ok” ST=”300”>

<TS DST=”no”>2004,01,22,12,00,00</TS>

<PV>17940</PV>

</M>

</MP>


<TZ>GMT+1</TZ>

<M Q=”ok” ST=”300”>

<TS DST=”no”>2004,01,22,12,00,00</TS>

<PV>52320</PV>

</M>

</MP>


<TZ>GMT+1</TZ>

<M Q=”ok” ST=”300”>

<TS DST=”no”>2004,01,22,12,00,00</TS>

<PV>1618776</PV>

</M>

</MP>

</MS>

Example Configuration File

The following example illustrates how simple XML data can be parsed. The

[XML_LINE_MQ] section extracts the PI status from the <M> tag’s ST attribute and adjusts

it by subtracting 300. The [XML_LINE_TS] section adjusts the incoming timestamp for

Daylight Saving Time if required. Note that, in the [XML_LINE_MP] section, double quotes

are escaped so they can be parsed as data. [INTERFACE]

PLUG-IN = ASCIIFiles.dll

[PLUG-IN]

ERR = BAD

IFM = C:\PIPC\Interfaces\PI_UFL\Data\*.xml

IFS = N

PURGETIME = 1d

[SETTING]

DEB = 4

MAXLOG = 10

MAXLOGSIZE = 20

MSGINERROR = C:\PIPC\Interfaces\PI_UFL\Logs\errors_xml.out

OUTPUT = C:\PIPC\Interfaces\PI_UFL\Logs\pi_ufl_xml.out

'-------------------------------------------------------------

[FIELD]

FIELD(1).NAME = "TAG_ID"

FIELD(2).NAME = "TIMEZONE"


FIELD(3).NAME = "TIMESTAMP"

FIELD(3).TYPE = "DateTime"

FIELD(3).FORMAT = "yyyy,MM,dd,hh,mm,ss"

FIELD(4).NAME = "DST"

FIELD(4).Type = "DSTFlag"

FIELD(4).Format = "no,yes"

FIELD(5).NAME = "UOM"

FIELD(6).NAME = "STATUS"


FIELD(7).NAME = "QUALITY"

FIELD(8).NAME = "VALUE"

FIELD(9).NAME = "TIMEONEHOUR"

FIELD(9).TYPE = "Time"

FIELD(9).FORMAT = "hh:mm:ss"

'-------------------------------------------------------------

' Five messages are recognized based on tagging:

[MSG]

MSG(1).NAME = "XML_LINE_MP"

MSG(2).NAME = "XML_LINE_TZ"

MSG(3).NAME = "XML_LINE_MQ"

MSG(4).NAME = "XML_LINE_TS"

MSG(5).NAME = "XML_LINE_PV"


'-------------------------------------------------------------

' TAG_ID and Unit of Measure

' Use backslash to escape double quote for parsing

[XML_LINE_MP]

XML_LINE_MP.FILTER= C1=="*<MP*"

UOM = ["*\"(*)\"*\"*\"*"]

TAG_ID = ["*\"*\"*\" (*)\"*"]

TAG_ID = "XML-" & TAG_ID

'-------------------------------------------------------------

' Time Zone Info

[XML_LINE_TZ]

XML_LINE_TZ.FILTER = C1=="*<TZ>*"

TIMEZONE = C1(">")-("<")

'-------------------------------------------------------------

' Quality and Status

[XML_LINE_MQ]

XML_LINE_MQ.FILTER = C1=="*<M Q=*"

QUALITY = ["*\"(*)\"*\"*\"*"]

STATUS = ["*\"*\"*\"(*)\"*"]

' 300 means OK => transform it to Status = 0 for PI

STATUS = STATUS-300

'-------------------------------------------------------------

' Timestamp Info

[XML_LINE_TS]

XML_LINE_TS.FILTER = C1=="*<TS*"

DST = ["*\"(*)\"*"]

TIMESTAMP = ["*\"*>(*)<*"]

TIMEONEHOUR = "01:00:00"

' Adjust for Daylight Saving if required

If(DST == 1) Then

TIMESTAMP = TIMESTAMP – TIMEONEHOUR

EndIf

'-------------------------------------------------------------


40

' Process Value

[XML_LINE_PV]

XML_LINE_PV.FILTER = C1=="*<PV>*"

VALUE = ["*>(*)<*"]

StoreInPI(TAG_ID,,TIMESTAMP,VALUE,STATUS,)

POP3 Server

The following example extracts data from incoming email directed to the user “ufl”. Data is

extracted from the body of the email, parsed and written to the PI Server.

The incoming data is formatted as follows:

Tagname: sinusoid, Timestamp: 01-Jun-2013 09:00:00, Value: 50,

Example Configuration File

' POP3.ini

[INTERFACE]

PLUG-IN = POP3.dll

[PLUG-IN]

POP3_SERVER = pop3.osisoft.com

POP3_USER = ufl

SMTP_SERVER = smtp.osisoft.com

FORWARD_TO = [email protected]

FORWARD_AS_UFLSTREAM = true

FILTER_FROM =

[email protected];[email protected]

MAIL_FROM = true

FROM_PREFIX = [From]:

MAIL_DATE = true

DATE_PREFIX = [Date]:

MAIL_SUBJECT = True

SUBJECT_PREFIX = [Subject]:

MAIL_BODY = true

BODY_PREFIX = [Body]:

MAIL_ATTACHMENT = true

ATTACHMENT_PREFIX = [Attachment]:

PFN = true

PFN_PREFIX = [Attached File Name]:

[SETTING]

DEB = 4

MAXLOG = 10

MAXLOGSIZE = 10

MSGINERROR = C:\PIPC\Interfaces\PI_UFL\Logs\pop3.err

OUTPUT = C:\PIPC\Interfaces\PI_UFL\Logs\pop3.out

'-----------------------------------------------------

[FIELD]

FIELD(1).NAME = "Tagname"


mailto:[email protected];[email protected]


FIELD(1).TYPE = "String"

FIELD(2).NAME = "Value"

FIELD(2).TYPE = "Number"

FIELD(3).NAME = "Timestamp"


FIELD(3).FORMAT = "dd-MMM-yyyy hh:mm:ss",_


[MSG]

' Only one message type

MSG(1).NAME = "DataLine"

[DataLine]

' Any line that contains the Tagname: pattern is considered a

' valid message

DataLine.Filter = C1=="*Tagname:* "

' Parse out three variables:

Tagname = ["*Tagname: (*),*"]

Timestamp = ["*Timestamp: (*),*"]

Value= ["*Value: (*),*"]

' Send the events to PI Archive

StoreInPI(Tagname,,Timestamp,Value,,)


Chapter 6. Configuring UFL Using UFLDesigner

UFLDesigner is a graphical tool that enables you to configure the processing of incoming

data. It is provided as an alternative to editing the configuration file manually, and ensures

that the resulting configuration file is syntactically correct. You can also use UFLDesigner to

validate manually-created configuration files. UFLDesigner can be launched from the PI

Interface Configuration Utility (ICU) on the UFL tab, when you are configuring the UFL

interface, as shown in the following figure.

To create a configuration file using UFLDesigner, perform the following steps:

1. Launch PI ICU.

2. Choose Interface > New Windows Interface Instance from BAT file… The Open

Interface Configuration File dialog is displayed.

3. Browse to the folder where the UFL interface is installed, choose PI_UFL.bat_new,

and click Open. The Select Host PI Server dialog is displayed.

4. Choose the PI Server that you want to use and click OK to dismiss the dialog.


5. On the UFL tab, browse to the INI subfolder in the folder where the UFL interface is

installed, choose the file named Sample.ini, and click Open to dismiss the dialog.

6. On the UFL tab, click Launch UFLDesigner.exe. UFLDesigner is launched. (Ignore any

“Sample.ini not valid” errors.)

7. In UFLDesigner, click New Ini File. The New INI dialog is displayed.

8. Specified the type of data source and the file to be processed.

Specify the remaining settings as follows. For details about how settings are specified, refer

to “Defining the Configuration File” on page 17.

General Tab

PLUG-IN: Settings specific to the type of input to be processed.

SETTING: Operational settings.

Variables Tab Define fields and variables required for processing logic.

Message Types Tab Filter incoming data and assign names for dispatching to data extraction logic.

.Data Extraction Tab Parse incoming data into fields. This tab contains features that enable you to define and preview how incoming data is parsed

Actions Tab Manipulate contents of fields and specify processing logic.

To add elements to each tab, click and proceed as prompted. When you are finished

configuring, save your setting


Appendix A. Platform Details and Features

PI_UFL is not based on the OSISoft UniInt framework. Notable aspects of the UFL include

the following:

Point source is not required

Can store values to PI annotations

Automatically creates new PI points and digital states and state sets

Data-source-specific logic is isolated in separate plug-in DLLs

Supported Features

(Features that are starred have additional explanation following this table.)

Feature Support

Interface part number PI-IN-OS-UFL-NTI

Automatically creates PI points* Yes

Point builder utility No

ICU control Yes

PI point data types Float16 / Float32 / Float64 / Int16 / Int32 / Digital / String / Timestamp

Sub-second timestamps Yes

Sub-second scan classes No

Automatically incorporates PI point attribute changes

Yes

Exception reporting* Yes

Outputs from PI No

Inputs to PI Scan-based

Supports questionable bit Yes

Supports multi-character point source Yes

Maximum point count Unlimited

Uses PI SDK* Yes

PINet string support No

Source of timestamps* System time or data from data source

History recovery* Yes


Feature Support

UniInt-based*

Disconnected dtartup*

SetDeviceStatus*

No

No

Yes

Failover* Yes, two independent interface instances.

Vendor software required on interface node or PINet node

No

Vendor software required on foreign device

No

Vendor hardware required No

Additional PI software Included with interface*

Yes

Device point types Not applicable

Serial-based interface* Yes

Exception Reporting

The following startup flags, when enabled, suppress standard exception reporting:

/lb

/lbs

/rbo

Uses PI SDK

The UFL interface requires the PI SDK and PI API to be installed on the interface node.

Source of Timestamps

When writing values to PI points, you can set timestamps using data from the data source or

the system time on the interface node.

SetDeviceStatus

The device status health point, designated by the text [UI_DEVSTAT] in the

ExtendedDescriptor attribute, represents the status of the data source as follows:

Status Description

Starting The interface has started and initialized the plug-in, and is waiting for the first scan class.

Good The interface is communicating properly and receiving data from its data source.

Intf Shutdown The interface was shut down.

Failover

No automation for failover. You can run redundant copies against the same data source (text

files or POP3 server, not serial ports).


46

Additional PI Software

PI UFL includes UFLDesigner, a GUI tool for configuring INI files.

Serial-Based Interface

Note that server-class machines often have inferior serial ports. Server-class machines are not

required for most interfaces and are not recommended when reliable serial port connections

are required.

Platform

The UFL interface is designed to run natively on the following Microsoft Windows 32-bit

operating systems and in emulation mode on 64-bit versions. The following platforms are

supported:

Windows XP

Windows 2003 Server

Windows Vista

Windows 2008

Windows 2008 R2

Windows 7

Windows 8

Windows 2012

No 64-bit build is available.


Appendix B. Installation and Configuration Checklist

If you are familiar with installing OSISoft interfaces, use the following checklist as a quick

reference. If you require detailed installation steps, refer to Installing the UFL Interface.

To install and configure the UFL interface, perform the following steps:

Download and run the UFL installation kit on the interface node.

Confirm that you can connect to the PI server: Launch PI System Management

Tools (SMT), choose File >Connections, and attempt to connect.

Test the API connection to the PI Server: In the %PIPC%\bin directory, issue

the apisnap PISERVERNODE command.

Test the SDK connection to the PI Server: Choose Start Menu > PI System >

AboutPI-SDK and use the File > Connections option to connect to the PI

Server.

Launch PI Interface Configuration Utility (ICU) and define an instance of the PI

UFL interface.

Using PI SMT, configure trusts that enable PI_UFL.exe to access the server

node. If necessary, configure firewall exceptions.

Configure the interface as a service and verify that it starts successfully. Reboot

the interface node and confirm that the interface service restarts.

If you intend to use digital tags, define the appropriate digital state sets. Add the

ON and OFF states to the system state set, if they are not already present.

Build PI tags to receive data from the interface. Note the following:

The only location code used is Location5, which configures exception

reporting.

ExDesc is used only for health points.

Convers defines the coefficient to be applied to PI numeric tags.

InstrumentTag defines the TagName alias.

PointSource is optional. If specified, the UFL interface loads the

corresponding points during startup.

Test the interface using the example INI files, which are located in the

Examples folder in the installation directory. The INI files contain instructions

for their usage. Verify that data is imported successfully.

Installation and Configuration Checklist

48

The following are useful but not required:

Configure diagnostics:

Configure diagnostic points:

scan class performance

performance counter

health monitoring

I/O rate

interface status

Install diagnostic software:

PI Server node: PI Interface Status Utility

Interface node: PI Performance Monitor Interface

Configure redundant instances of the interface to ensure that, if one instance fails,

the other continues collecting data


Appendix C. Command Line Parameters


/am=#

Optional

Archive mode. If the snapshot bypass feature is enabled (/lb), you can configure archive mode by setting this flag to the following values:

3: (ARCNOREPLACE) Add unless event(s) exist at same time (PI 2.x).

4: (ARCAPPEND) Add event regardless of existing events.

5: (ARCREPLACE) Add event, replace if event at same time. (Default)

6: (ARCREPLACEX) Replace existing event (fail if no event at time).

7: (ARCDELETE) Remove existing event. 8: (ARCAPPENDX) Add event regardless of existing

events, with no compression.

This startup parameter does not apply when values sent to the PI Server include annotations, because such values are sent using the PI SDK. If annotations are required and you need this feature, set Location5 for the target tags.

/cf=xxx.yyy

Required

Specifies the configuration file, including full path.

/des=#

Optional

Default error status, specified as the index of the desired state from the PI system digital set (specified by MSG(n).DIGITALSET). This status is stored in PI when the digital status string cannot be translated.

/disablecounters

Optional

Disable writing to performance counters.

/f=HH:MM:SS

Or

/f=SS

Required

The /f parameter defines the time period between scans in terms

of hours HH, minutes MM, and seconds SS.

For example, to define a one minute scan class: /f=00:01:00

Unlike other interfaces, the UFL interface supports only one scan class. If you specify multiple /f flags, only first one is recognized.

/host=host:port

Required

The IP address or node name and port of the PI Server. Default is 5450. Examples:

/host=marvin

/host=marvin:5450

/host=206.79.198.30

/host=206.79.198.30:5450

/imt

Optional

Ignore missing tags, do not create them.

Command Line Parameters

50


/lb

Optional

Write events to archive in bulk for efficiency. By default, events are sent one at a time. Must be enabled to set archive mode (/am).

/lbs

Optional

Write events to snapshot in bulk for efficiency. By default, events are sent one at a time. Must be enabled to set archive mode (/am).

/perf=#

Default: 8 hours

Optional

Specifies (in hours) how often performance summary information is written to performance counters.

/ps=x

Optional

Specifies the point source for the points updated by the interface. Not required by UFL interface. If configured, the interface loads only PI points with the specified point source.

/rbo

Optional

Read archive before writing value, to see if event already exists at same timestamp. Write new value only if value is different from existing value. To enable this option, points must have Location5

set to 1, and neither the /lb or /lbs flags can be configured.

Does not work if values include annotation, which are written using the PI SDK.

/runonce

Optional

Process existing data and exit. The ASCII file plug-in processes specified files and exits. For the POP3 plug-in, read and process emails and exit. Not applicable for serial port plug-in.

/tm=xxx*

Or

/tm="xxx xxx*"

Optional

Tag mask. The interface loads all points matching this tag mask prior to run-time operation. Useful when using the InstrumentTag attribute to identify target tags to store data in,

and you must limit the write operations to a subset of tags. Wildcard characters are * or ?.

/uht_id=#

Optional

Allows different instances of the UFL interface working with a different set of Health Tags. The match between the number defined by this start-up parameter occurs through Location3 of the given set of Health Points.

/utc

Optional

Incoming timestamps are UTC.

/wd=#

Optional

Write delay between bulk writes to the PI archive, specified in miliseconds. Default is 10ms. Used to balance the workload on the PI Archive and the network. See also /ws=# below.

/ws=#

Optional

Write size: maximum number of values written in one (bulk) call to the PI Archive. Default is 10240 events per bulk. Used to balance the workload on the PI Archive and the network, for example, when loading files covering a long time periods.


Appendix D. UFL Examples

Besides the examples in this appendix, the UFL interface includes example configuration and

data files, which are installed in the Examples folder in the directory where you install the

interface. Before attempting to configure and run them, modify the paths (and possibly the

timestamp formatting) in the configuration files to reflect your interface node’s configuration.

The installed examples include the following:

Example4MsgLgNt illustrates processing of data from a serial port.

Example5BatchFl01 shows how to process the BatchFL data file structure using the

ASCII files plug-in, as do the following two examples. Example5BatchFl04 shows

how to construct a configuration file for processing BatchFL files.

Example8Pop3 illustrates processing of data from a POP3 email server.

The following sections contain examples that illustrate how you construct configuration files

and use UFL statements to process various types of input data.

Calculation Examples

Simple Expressions with Arithmetic Operators

[FIELD]



[MSG(1)]

' Data file content:

' 001, Value: 1.23

' Create a tag name using the '&' operator

' Scale the value by a factor of 100

FIELD(1) = C1 – (",")

FIELD(1) = "TAG_" & FIELD(1)

' extract the value and scale it

FIELD(2) = C12 – NEWLINE

FIELD(2) = 100 * FIELD(2)

Mathematical Functions

[FIELDS]



[MSG(1)]

UFL Examples

52


' Value1: 1.23; Value2: 2.61

FIELD(1) = ["*(*);*:*"]

FIELD(2) = ["*:*;*(*)"]

' Apply ROUND()

FIELD(1) = ROUND(FIELD(1))

FIELD(2) = ROUND(FIELD(2))

String Functions

[FIELDS]


[MSG(1)]

' Replace string data with an error

FIELD(1) = C10 – NEWLINE

FIELD(1) = REPLACE(FIELD(1), "Invalid string part", "OK")

Sub-Milliseconds

[FIELDS]


FIELD(1).Format = "dd-MMM-yyyy hh:mm:ss.nnn"


[MSG(1)]

' Data file content: 01-Jul-2006 08:00:00.1234; 123

' PI supports time precision up to 15 microseconds.

FIELD(1) = C1 – (";")

' extract the number after the semicolon:

FIELD(2) = ["*;(*)"]

IF Statement

Example 1

[FIELDS]



[MSG(1)]

' Data file content: 1;2

FIELD(1) = ["(*);*"]

FIELD(2) = ["*;(*)"]

IF (FIELD(1) > FIELD(2)) THEN

FIELD(2)=2*FIELD(2)

ELSE


FIELD(2)=FIELD(1)

ENDIF

Example 2

[FIELDS]



FIELD(3).Type = "Time"

[MSG(1)]


' 25-Jan-2007;01-Nov-2007;01:00:00

FIELD(1) = ["(*);*;*"]

FIELD(2) = ["*;(*);*"]

FIELD(3) = ["*;*;(*)"]

IF (FIELD(1) > FIELD(2)) THEN

' Add one hour

FIELD(1) = FIELD(1) + FIELD(3)

ENDIF

Example 3

[FIELD]




[MSG(1)]


' Tag1; 23-Oct-2007 01:00:00; 1

FIELD(1) = ["(*);*;*"]

FIELD(2) = ["*;(*);*"]

FIELD(3) = ["*;*;(*)"]

' Only store value if tag name is set

IF (FIELD(1) IS NOT NULL) THEN

StoreInPI(FIELD(1),,FIELD(2),FIELD(3),)

ENDIF

Example 4

[FIELDS]

FIELD(1).Name = "TimeVar"


FIELD(1).Format = "m"

FIELD(2).Name = "TimeOffset"


FIELD(2).Format = hh:mm:ss"

FIELD(3).Name = "DateVar"

UFL Examples

54


FIELD(3).Format = "yyyymmdd"

FIELD(4).Name = "TimestampVar"


FIELD(5).Name = "TagNameVar"

FIELD(6).Name = "ValueVar1"


FIELD(7).Name = "ValueVar2"


' …


' 200,TagName1,kWh,30,

' 300,20071201,,1,1.2,1.1,1.12,1.01,…

[MSG(1)]

MSG(1).NAME = "DataDetails"

MSG(2).NAME = "Values"

' …

[Values]

Values.FILTER = C1=="300*"

TimeOffset = "00:30:00"

' Example of multiple statements executed in an IF()

' statement

IF (TimeVar == TimeOffset) THEN

TimestampVar = DateVar + TimeVar

StoreInPI(TagNameVar,,TimestampVar,ValueVar1,,)

TimestampVar = TimestampVar + TimeVar

StoreInPI(TagNameVar,,TimestampVar,ValueVar2,,)

TimestampVar = TimestampVar + TimeVar

ENDIF

Example 5

[FIELD]




[MSG(1)]


' Tag1; 23-Oct-2007 01:00:00; 1

FIELD(1) = ["(*);*;*"]

FIELD(2) = ["*;(*);*"]

FIELD(3) = ["*;*;(*)"]

' Nested IF

IF (FIELD(1) IS NOT NULL) THEN

StoreInPI(FIELD(1),,FIELD(2),FIELD(3),)

ELSE


IF(FIELD(2) IS NULL) THEN

StoreInPI("ErrorTag",,,FIELD(3),)

ENDIF

ENDIF

Date Math

' Use "+" to concatenate fields to compose a timestamp.

Timestamp = Date+DaysTime+HoursTime+MinutesTime

' Use "-" to conditionally adjust for Daylight Saving Time.

TimeOffset = "01:00:00"

IF((Day + Time) >= "25-Mar-2007 03:00:00" AND (Day + Time) <

"28-Oct-2007 03:00:00") then

Time = Time - TimeOffset

endif

Advanced Calculation Examples

Examples using more structured input data files are shown in the next sections.

Data File Example

S 05.07.200314:40:21Pt=422 Reaktor-B 0303301905

D 13 NGScalib 9000 30.00 34.50

C Al1 7.36881 % Al 2.4380 10.0 1.0191 0.0000 8000

C P 41.15004 ppm P 0.0707 30.0 1.0095 0.0000 8000

C Ca 2.19745 % Ca 4.3559 10.0 1.0004 0.0000 8000

C Pb 21.69290 ppm Pb 0.1271 100.0 0.9978 0.0000 8000

C Si* 98.03407 % Si 8000

Configuration File Example

The configuration file below defines two message names, “S_Line” and “C_Line”. A

message is recognized if the line starts with an “S” or a “C”. If the message is caught by the

filter, the : InstrumentTag, PI_Timestamp, Value and InstrumentTagPrefix

fields are extracted. InstrumentTagPrefix is set using the value (“Reaktor-A” or

“Reaktor-B”) at position 28. In the [C_Line] section, the InstrumentTag field is

composed of a combination of a prefix and the characters starting in column 3 up to (but not

including) the first space after column 3, as shown in the following statement:

InstrumentTag = InstrumentTagPrefix & C3 – C3(" ")

The target tag is specified by setting the InstrumentTag parameter. (Note that the tag

name parameter is blank.) Finally, the value is sent to the PI Server by issuing the

StoreInPI statement.

' ------------------------------------------------------

' XRF.ini

' ------------------------------------------------------

[INTERFACE]

UFL Examples

56

PLUG-IN = ASCIIFiles.dll

[PLUG-IN]

ERR = BAD

IFM = C:\PIPC\Interfaces\PI_UFL\Data\*.xrf

IFS = N

PURGETIME = 8h

[SETTING]

DEB = 4

MAXLOG = 20

MAXLOGSIZE = 10

MSGINERROR = C:\PIPC\Interfaces\PI_UFL\Logs\errors_xrf.out

OUTPUT = C:\PIPC\Interfaces\PI_UFL\Logs\PI_UFL_xrf.out

'-------------------------------------------------------------

[FIELD]

FIELD(1).NAME = "InstrumentTag"

FIELD(2).NAME = "InstrumentTagPrefix"

FIELD(3).NAME = "PI_Timestamp"


FIELD(3).FORMAT = "dd.MM.yyyyhh:mm:ss"



FIELD(5).Name = "Resource"

'-------------------------------------------------------------

[MSG]

' File consists of two messages.

MSG(1).NAME = "S_Line"

MSG(2).NAME = "C_Line"


'-------------------------------------------------------------

[S_Line]

' Filter

S_Line.FILTER = C1==”S*” AND C28 == "Reaktor*"

' Variables

PI_Timestamp = C3 – C21

Resource = C28 – C37

' Logic

IF(Resource == "Reaktor-A") THEN

InstrumentTagPrefix = "T42_C100A_PRFA_BETT_"

ELSE

IF(Resource == "Reaktor-B") THEN

InstrumentTagPrefix = "T42_C100B_PRFA_BETT_"

ELSE

InstrumentTagPrefix = "UNDEFINED_"

ENDIF

ENDIF

'-------------------------------------------------------------

[C_Line]

' Value lines:

C_Line.FILTER = C1==”C*”

' Variables


InstrumentTag = InstrumentTagPrefix & C3 – C3(" ")

Value = C8-C16

' Action

StoreInPI(,InstrumentTag, PI_Timestamp, Value,,)

Action Examples

The following examples illustrate the use of actions to modify data and processing logic.

AppendLines()

Suppose the input file contains the following lines:

BATCH: B1;

05-Feb-07 12:00:00;

Mixture1

UNIT: U1;

05-Feb-07 12:10:00;

Blue

The configuration file filters for “BATCH” and “UNIT” and appends the following two lines,

as shown in the following example.

[MSG]

MSG(1).Name = "Batch_MSG"

MSG(2).Name = "Unit_MSG"

[Batch_MSG]

Batch_MSG.Filter = C1 == "BATCH*"

Batch_MSG.Action = AppendLines(2)

Batch = ["*(*);*;*"]

TimeStamp = ["*:*;(*);*"]

Value = ["*:*;*;(*)"]

StoreInPI(Batch,,TimeStamp,Value,,)

[Unit_MSG]

Unit_MSG.Filter = C1=="UNIT*"

Unit_MSG.Action = AppendLines(2)

Unit = ["*(*);*;*"]

TimeStamp = ["*:*;(*);*"]

Value = ["*:*;*;(*)"]

The resulting lines look like this:

BATCH: B1; 05-Feb-07 12:0:00; Mixture1

UNIT: U1; 05-Feb-07 12:10:00; Blue

UFL Examples

58

SetNextMsg()

' Data file content: Name, Timestamp, Value

' Tag1, 05-Feb-07 12:00:00, 1

' Tag1, 05-Feb-07 12:10:00, 2

[FIELD]

FIELD(1).NAME = "TagName"

FIELD(2).NAME = "Timestamp"

Timestamp.TYPE = "DateTime"

Timestamp.FORMAT = "dd-MMM-yy hh:mm:ss"



[MSG]

MSG(1).Name = "Description"

MSG(2).Name = "Events"

[Description]

Description.Filter = C1=="Name, Timestamp, Value"

' Check the next couple of lines in the context of MSG(2)

' until there is a line that does not satisfy the filter

Tag.Action = SetNextMsg ("Events",)

[Events]

Events.Filter = C1 == "*,*,*"

FIELD(1) = ["(*),*,*"]

FIELD(2) = ["*,(*),*"]

FIELD(3) = ["*,*,(*)"]

StoreInPI (TagName,,Timestamp,Value,,)

SkipFile()

In this example, the entire file is skipped because the first message matched the section that

invoked SkipFile() function. The file is renamed as though it were processed normally.


' Invalid Sample

' Name, Timestamp, Value

' Tag1, 05-Feb-07 12:00:00, 1

' Tag1, 05-Feb-07 12:10:00, 2

[MSG]

MSG(1).Name = "FileValidation"

MSG(2).Name = "MSG1"

[FileValidation]

FileValidation.Filter = C1=="Invalid*"


SkipFile()

[MSG1]

SkipLines()

' Data file contents: Name, Timestamp, Value

[MSG]

MSG(1).Name = "MSG1"

[MSG1]

MSG1.Filter = C1=="*,*,*"

FIELD(1) = ["(*),*,*"]

FIELD(2) = ["*,(*),*"]

FIELD(3) = ["*,*,(*)"]

IF (FIELD(3) < 0) Then

SkipLines(1)

Else

StoreInPI(FIELD(1),, FIELD(2), FIELD(3),,)

EndIf

StoreInPI

Example 1

' Write a value of FIELD(1) to the tag named test:001

' using current time

StoreInPi ("test:001",,,FIELD(1),,)

Example 2

' Write the value of FIELD(1) to the tag 'test:001'

' using current time. If the value exceeds 200, indicate

' the value is bad (-255 represents the code

' from the PI system digital Set)

FIELD(1) = ["*,(*),*"]

IF( FIELD(1) > 200 ) THEN

FIELD(2) = -255

Else

FIELD(2) = 0

Endif

StoreInPi ("test:001",,,FIELD(1),FIELD(2),)

Example 3

' Write the full PI data record. Because an annotation is

' included, the interface uses the PI SDK to write the value.

[FIELD]

UFL Examples

60

FIELD(1).NAME = "PI_TAG"


FIELD(2).NAME = "PI_TIMESTAMP"


FIELD(2).FORMAT = "yyyy-MM-dd hh:mm:ss"

FIELD(3).NAME = "PI_VALUE"


FIELD(4).NAME = "PI_STATUS"


FIELD(5).NAME = "PI_QFLAG"


FIELD(6).NAME = "PI_ANNOTATION"


FIELD(7).NAME = "RESULT"


[MSG]

MSG(1).Name = "Msg1"

[Msg1]

Msg1.Filter = C1=="-"

' Field filters

Result = StoreInPI(PI_TAG,, _

PI_TIMESTAMP, _

PI_VALUE, _

PI_STATUS, _

PI_QFLAG, _

PI_ANNOTATION)

IF( RESULT <> 0) Then

StoreInPI("UFL_Error_Tag",,,Result,,)

EndIf


Appendix E. Migrating from the Batch File Loader Interface

Version 3.x of the PI UFL includes a utility for migrating from the PI BatchFL interface, and

the UFLDesigner GUI include supporting features as well. However, the PI_UFL interface

does not support use of a tag number instead of a tag name or alias in the data files (/TN

parameter). If you require this feature, do not migrate to PI UFL.

The migration utility does the following.

Removes the PI BatchFL interface instance and delete associated services (optional but

recommended)

Creates and registers an instance of the PI UFL interface.

Creates a configuration (INI) file for the PI UFL interface.

After conversion, launch PI ICU and configure the new instance of the PI UFL interface as

required. To ensure that the interface restarts when the interface node is rebooted, configure it

as an automatic service.

If you configured an event counter tag for the BatchFL interface, you can rename it for use

with the PI UFL interface as follows:

1. In PI ICU, select the new PI UFL interface instance.

2. Click IO Rate.

3. Click Rename and specify the desired name for the point.

If you choose to keep the instance of the BatchFL interface, stop its service and change the

type of service to “Manual”, to ensure that it does not compete for resources with the newly-

installed PI UFL interface.

Note: Unlike the BatchFL interface, the PI UFL interface does not support out-of-order data handling. To disable exception reporting on a per-tag basis, set Location5 to a non-zero value.

Migrating from the Batch File Loader Interface

62

Creating the Startup File

To create a batch startup file for PI UFL based on the BatchFL startup file:

1. Navigate to the PI UFL interface installation directory and double click the

BatchFL_to_UFL.exe file. The BatchFL to UFL dialog box is displayed.

2. In the dialog box, browse to the BatchFL batch startup file. The dialog displays the

source command line and the corresponding UFL startup command.

3. Click the Convert to UFL button. The utility displays the results of the conversion; for

example:

4. To dismiss the dialog, click OK.

To make further changes to the configuration, launch PI ICU and select the instance created

by the utility.

Post Conversion Steps

Due to differences in features between the BatchFL and UFL interfaces, you might need to

adjust the configuration of the UFL interface as described in the following sections.

If the BatchFL interface instance is removed

If you enabled an event counter and IO rate tag for the BatchFL interface, you can rename the

tag so it is clearly associated with the new UFL interface instance.To rename the tag, perform

the following steps:

1. In PI ICU, select the new PI_UFL interface instance.

2. Display the IO Rate tab.

3. Click the Rename button. PI ICU displays a dialog box where you can rename the

tag.

4. Rename the IO rate tag to indicate its association with the PI UFL interface. For

example:

Old name: sy.io.xyz.BatchFL1

New name: sy.io.xyz.PI_UFL1

If the BatchFL interface instance is retained

Do not run the BatchFL and UFL interfaces simultaneously if both interface instances

contend for the same data sources and write to the same PI tags.


If you retain the instance of the BatchFL interface, you must manually create an IO Rates tag

and event counter for the new PI_UFL interface and manually configure the UFL interface to

run as a service.

Handling Out-of-Order Data

Unlike the BatchFL interface, the UFL interface does not handle out-of-order data. If you

need to handle such data, you must set Location5 of the tags in question to a non-zero

value, and you cannot enable the /LB or /LBS options.


Appendix F. Glossary

Buffering: Temporary storage of the data that PI interfaces collects and forwards to the PI

Servers. Buffering enables the interface to continue collecting data if the PI Server is

unavailable.

ICU: The PI Interface Configuration Utility is the primary application that you use to

configure PI interfaces. Install the ICU on the computer where the interface runs. OSIsoft

strongly recommends that you use the ICU for interface management tasks, rather than

manually editing configuration files.

ICU control: A plug-in for the ICU that enables it to configure a particular OSISoft

interface.

Interface node: A computer where an OSISoft interface runs.

N-way buffering: The ability of a buffering application to send the same data to every PI

Server in a PI Collective.

PI API: A library of functions that enable applications to communicate and exchange data

with the PI Server.

PI Collective: Two or more replicated PI Servers that collect data concurrently. Collectives

are part of the High Availability environment. When the primary PI Server in a collective

becomes unavailable, a secondary collective member node seamlessly continues to collect

data and provide data access to your PI clients.

PI SDK: A library of functions that enable applications to communicate and exchange data

with the PI Server. Some PI interfaces require the PI SDK.

PI Server node: The computer on which PI Server programs run.

PI SMT: (PI System Management Tools) OSISoft’s graphical tool for configuring PI

Servers.

PIHOME: The directory where PI 32-bit client applications reside. Interfaces are installed in

a subdirectory of PIHOME.

PIHOME64: The common location for PI 64-bit client applications.

Local PI message log: The file to which OSIsoft applications, including interfaces, write

informational and error messages. To view the log, use PI ICU.

Point: A value tracked by the PI Server. Also called a “tag.”

Service: “On Microsoft Windows operating systems, a Windows service is a long-running

executable that performs specific functions and it is designed not to require user intervention.

Windows services can be configured to start when the operating system is booted and run in

the background as long as Windows is running, or they can be started manually when

required.” [Wikipedia]. OSISoft interfaces can be configured as services, to ensure they

restart when the interface node is rebooted.


Appendix G. Technical Support and Resources

For technical assistance, contact OSIsoft Technical Support at +1 510-297-5828 or

[email protected]. The OSIsoft Technical Support website offers additional contact

options for customers outside of the United States.

When you contact OSIsoft Technical Support, be prepared to provide this information:

Product name, version, and build numbers

Computer platform (CPU type, operating system, and version number)

Time that the difficulty started

Log files at that time

Details of any environment changes prior to the start of the issue

Summary of the issue, including any relevant log files during the time the issue occurred

The OSIsoft Virtual Campus (vCampus) website has subscription-based resources to help you

with the programming and integration of OSIsoft products


PI Interface for Universal File and Stream Loading...

Documents

Transcript of PI Interface for Universal File and Stream Loading...