What was new

New in OmniMark 9.1.0

New in the libraries of OmniMark 9.1.0

OmniMark 9 introduces OMXMLWRITE library can be used to convert a markup event stream to a well-formed XML instance. The library exports two functions, xml.writer and xml.written.

New in OmniMark 9.0.1

New in the OmniMark Studio for Eclipse 9.0.1

The bytecode export wizard now offers only the launch configurations that have their XVC filename set.

During debugging, the variable view now highlights shelves and shelf items that have been modified.

New in OmniMark 9.0.0

OmniMark Studio for Eclipse now requires Eclipse version 3.4.1

New in the language of OmniMark 9.0.0

To facilitate the streaming of markup processing, markup source and markup sink have been introduced. A markup source creates a markup stream, which is a stream of data interspersed with markup events. A markup sink consumes a markup stream. The built-in SGML and XML parsers do not trigger markup rules directly any more: they parse the input string source into a markup source, replacing each markup tag by the appropriate markup-event.

The do markup-parse action has been generalized to process any markup source.

The built-in variable #content has been introduced to represent the content of a markup-region-event that has triggered the current rule.

The built-in variable #current-markup-event represents the markup-point-event or markup-region-event that has triggered the current rule.

A rule can reproduce the markup stream it is processing by outputting #content or by signalling the catches #markup-start, #markup-end and #markup-point with a markup event as an argument.

For a discussion on how the above features can be used, please refer to markup processing control, linking chains of streaming markup filters, and validating markup.

The sgml-dtds and xml-dtds shelves can now be used as other shelves: key of, remove, repeat over, and other shelf operations can be applied to them. Only copy, set, and new are still disallowed.

New in the OmniMark 9.0.0 compiler

A compile-time warning is now emitted when a markup-error is found to belong to a group other than group #implied. group declarations do not affect markup-error rules.

The %c format item can now be used in a shelf initializer inside a do sgml-parse, do xml-parse, and do markup-parse scope.

New in the libraries of OmniMark 9.0.0

A new RELAX NG support library, OMRELAXNG has been introduced, to enable programmers to validate input against a RELAX NG schema.

The OMMARKUPUTILITIES library has been added, containing new functionality for processing markup streams.

The new OMUNICODE library provides easy access to Unicode character properties.

The new OMURI library provides the ability to validate and manipulate URI strings.

New in OmniMark 8.2.0

New in the OmniMark 8.2.0 compiler

Actions signal throw and signal rethrow have been added for communicating out-of-band information between coroutines.

A string sink function call can now be used with a referents-allowed modifier. Any referent output to the string sink function must belong to a nested-referents scope entirely contained within the scope of the string sink function call.

The syntax define input function now triggers a compiler warning. The preferred syntax is define string source function.

Attribute handling and XML namespace processing has been optimized out if it is not needed by the program.

submit can now be used in a markup-error rule.

do xml-parse and do sgml-parse can now be used in find-start, find-end, external-text-entity, and markup-error rules.

sgml-in and sgml-out can now be used in markup-error rules.

A global variable declaration can now be prefixed by the keyword domain-bound.

The restriction on using output-to action on a string sink coroutine has been relaxed. Now output-to is only disallowed on a string sink function call itself, not on a reference to the created string sink.

New in the OmniMark 8.2.0 Runtime

OmniMark now outputs a function call stack trace after a run-time error.

The "%c" format item is now streaming: submit "%c", for example, will run the content-processing code concurrently with the find rules.

New in the OmniMark 8.2.0 libraries and external API

A new process execution library module, OMPROCESS, has been introduced. It provides a mechanism to execute a program outside of OmniMark, to control the input to this program, and process the output from this program. OMPROCESS supercedes OMSYSOUT, which is, however, still available for backwards compatibility.

The function db.is-null has been added to the OMDB function library.

The OMUTF8 module now exports a byte-order mark constant.

New in OmniMark 8.1.0

The behavior of the next group is action has changed in coroutines: the action now modifies the current group of the parent coroutine, unless it is protected by a save groups.

The behavior of coroutines has changed. A string source coroutine now terminates when the last reference to it is dropped, whereas string sink coroutines receive a value-end on their #current-input and continue running normally.

Content modifiers can now be specified on #current-output.

The keyword output can no longer be used as a synonym for #main-output.

The is attached test can now be applied to value string sink arguments.

The flush action can now be applied to #current-output.

string sink functions are no longer required to consume their entire #current-input.

Multiple instances of #current-output can now be specified in a single string sink expression.

The drop and take operators now stream better. As a consequence, they do not rewind their input in the failure case.

OmniMark's run-time memory footprint has been reduced.

The -ftrace command-line option now tracks coroutines as well as nested function calls.

A new warning class, performance, has been added. This class is used to identify uses of language constructs that may negatively affect the performance of a user's program.

A new warning option, -warning-summary, has been added. It is used to suppress a particular class of warnings, generating instead a summary upon termination.

New in OmniMark 8.0.2

Call stack information is now displayed in the debug information view in Studio for Eclipse.

New in OmniMark 8.0.1

Single-item string constants are inlined at compile time.

#current-input in a string sink function can be passed as a string source argument to a function.

The -term option can now be used without specifying any input file.

do select action now accepts the as alias specification syntax.

New in OmniMark 8

stream shelves are now passable as write-only arguments to a function.

The void action has been introduced. It allows an expression to be evaluated purely for its side-effects.

A catch can now take modifiable and write-only arguments, as well as read-only and value arguments.

Pattern matching operations have been optimized to improve run-time performance.

Certain markup rules have been optimized to improve run-time performance.

The submit action can now be performed from a markup-error rule.

@<item number> and item <item number> have been deprecated as syntax for item indexers; [<item number>] should be used instead. A warning is now emitted when the deprecated forms are encountered.

^ <key value> and key <key value> have been deprecated as syntax for key indexers; {<key value>} should be used instead. A warning is now emitted when the deprecated forms are encountered.

The domain-free modifier is now deprecated. Where it appears, it has no effect. All streams are domain-free. A warning is emitted when the modifier is encountered.

The set action now evaluates its right-hand side argument first, then its left-hand side. All other actions and expressions are evaluated left-to-right, depth-first.

Element names in element context tests (parent is, ...) can now be expressions. The tilde (~) is used to introduce an expression that generates an element name.

The string data type has been introduced. It should be used instead of stream when the shelf item is always attached to a buffer.

  • The value-stream argument type (both for functions and for catches) has been deprecated in favour of the value string argument type. A warning is emitted when value stream is encountered.
  • The remainder stream argument type has been deprecated in favor of the remainder string argument type. A warning is emitted when remainder stream is encountered.
  • The stream function type has been deprecated in favor of the string function type. A warning is emitted when the definition of a function of type stream is encountered.
  • stream type tests (is attached, is closed, has name, ...) on value stream arguments have been deprecated: they are essentially dead code, and should be removed. A warning is emitted when these tests are encountered.

The input and external source function types have been deprecated in favour of the new string source function type. A warning is emitted when the deprecated forms are encountered.

The external output function type has been deprecated in favour of the new external string sink function type. A warning is emitted when the deprecated form is encountered.

The string sink function type has been introduced for internal functions, as well as external functions. A function of type string is executed as a coroutine with the context that feeds data to it.

The value source external function argument type has been deprecated in favour of the new value string source function argument type. A warning is emitted when the deprecated form is encountered.

The value string source function argument type has been introduced for internal functions, as well as external functions. A function of type string source is executed as a coroutine with the context that is feeding data to.

The value output external function argument type has been deprecated in favour of the new value string sink function argument type. A warning is emitted when the deprecated form is encountered.

The value string sink function argument type has been introduced for internal functions, as well as external functions.

The keyword prefix-function was never documented as an alternative to the function keyword when defining an overloaded function. It has been removed from the language. Programs that use the keyword prefix-function will no longer compile.

A new form of module has been introduced to better support circular dependencies, and to allow separation of interface from implementation.

The operator forms of new and new? have been generalized: wherever a shelf item reference can occur, the operators can be used.

Literal values for shelves can now be supplied wherever shelf references can be specified, in addition to their old use following the initial keyword of a shelf declaration.

A private overriding of a function can now be made on a private type.

elsewhere can now be used on global shelf declarations.

The syntax for the predeclaration of record types has been generalized.

Predeclared record types can now be extended.

An overriding function definition can now appear before the definition it overrides.

Functions of type input now co-routine in all contexts. (They used to be limited to co-routining only when invoked from a markup parse.)

The share name for a shared module can now be an arbitrary (constant) string.

The new shelf reference operator can be followed by a ?, which allows the operation to proceed if the shelf item being created already exists. The new? form of the new operator requires a key, and does not accept an insertion point.

An optional function argument can be passed down to a function that takes an optional argument, whether or not the argument was specified.

The construct unanchored <pattern> is deprecated; use any** <pattern> instead.

New in the libraries of Omnimark 8

The omxerces library has been updated to use Xerces 2.7.0

The omdb_sedna extension to the omdb library has been introduced. It allows OmniMark programs to access a SEDNA XML database server via the omdb interface.

Support for Oracle 8 client installations has been dropped, as has the corresponding db.open-oci8 function

Support for Oracle 9i and 10g client installations has been added on supported platforms, as have the corresponding db.open-oci9i and db.open-oci10g functions.

Dynamic (or scrollable) cursor support has been added to the OCI backends for Oracle 9i and 10g.

The default cursor type is now static instead of dynamic, which is a change of behavior for the ODBC backend.

db.move-dynamic-record has been created which provides the functionality of the old db.move-record; db.move-record no longer takes optional 'by' or 'to' parameters. This was done so that any OmniMark code which assumes that cursors are dynamic by default will no longer compile (instead of providing an error at runtime). Any such code needs to be changed as follows:

  1. change the move-record call to move-dynamic-record
  2. add the optional 'cursor db.dynamic-cursor' to the db.query or db.compile-statement call which leads to the creation of that result set

The new Oracle 10g datatypes binary_float and binary_double are now supported.

The new Oracle 9i datatype XMLType is now supported in db.table operations by treating it as a CLOB. In queries, compiled statements and procedures, the programmer is responsible for converting to and from XMLType in their SQL statements.

New in OmniMark Studio for Eclipse 8.0

Ported work with Eclipse 3.1.

Added support to run/debug/compile from OmniMark programs and project files selected in Navigator view.

A number of small preferences additions added for the editing view.

New in OmniMark 7.1.2

The omxerces library has been updated to use the latest release of the Xerces parser from the Apache Group, Xerces 2.4.0.

A new form of the open action has been added: open new. Therefore, instead of

          local stream s variable initial-size 1
  
          new s{"one"} before [1]
          open s{"one"} with referents-allowed as buffer

the following shorter form can be used

  
          open new s{"one"} before [1] with referents-allowed as buffer

The syntax of set new when applied to a stream shelf with modifiers and an insertion point has been updated to conform with other uses of insertion points and modifiers in the language. The old form has been deprecated and may be dropped in future releases of the language.

New forms of using output as and output-to has been added: stream attachments (files, external output functions, and referents) can now be specified, as well as stream references. Therefore, instead of

          local stream s 
  
          open s with referents-allowed as file "foo.txt"
          using output as s
             output "Hello, World!%n"
          close s

the following shorter form can be used

          using output as file "foo.txt" with referents-allowed
             output "Hello, World!%n"

A new stream modifier has been added: append. This opens the corresponding attachment in appending mode:

          local stream s 
  
          open s with append as file "foo.txt"

The old idiom for opening an attachment in appending mode, which specified an attachment in the reopen action, e.g.,

          reopen s as file "foo.txt"

has been deprecated and may be dropped in future releases of the language. The append modifier can be used with the open and output-to actions, and the using output as prefix.

The http and https protocols in the omvfs library have been extended to allow specification of proxy servers as well as user and password information that is not passed as part of the URL. (eg. http://user:[email protected]:port/path/to/file/filename )

New in OmniMark Studio 7.1.2

OmniMark Studio now displays records in the variables tables.

A new tab has been added to the project options dialog, which allows you to specify which classes of warning will be reported.

OmniMark Studio 7.1.2 ships with a number of sample programs designed to demonstrate some of the things that can be done with OmniMark. By default these are installed in the samples sub-directory.

New in the packaging of OmniMark 7.1

OmniMark 7.1 introduces OmniMark Studio for Eclipse, a cross-platform development environment, which also allows you to take advantage of a rich set of other development tools that have been built for the Eclipse platform. OmniMark Studio for Eclipse is available for all supported platforms.

You can once again run programs from source code in your OmniMark Server or Desktop Engine, using the -s command. A version of the engine, known as VM, which only runs compiled scripts, is still available.

New in the language of OmniMark 7.1

OmniMark now includes records. A record is a user-defined data type that may contain multiple fields of different types.

You can specify an alias for the shelf that you are repeating over. The alias name is specified using the as keyword following the name of the shelf. Specifying an alias is required when you iterate over a shelf that is a field of a record.

You can use the conditional operator ->...|... to select one of two values based on a condition.

You can now write overloaded functions, which are two or more functions with the same name but which operate on data of different types.

You can now write infix functions, where the name of the function goes between its two arguments. Thus the function works like an operator.

Compiler warnings have been categorized and enabled by default. You can control warnings using new command-line options:

  • -warning-error promotes the class of warning to error
  • -warning-ignore tells the compiler to ignore the specified class of warnings
  • -warning-report reports the warning and compiling continues.
Classes of warnings are:
  • ambiguity (a statement is potentially ambiguous )
  • deprecated (you have used a deprecated keyword, and should use the modern equivalent)
  • dead-code (a section of code cannot be reached)
  • pattern (a pattern can match zero characters)
  • redundancy (the same declaration or modifier is used more than once)
  • uninitialized data-access (a shelf has not been explicitly initialized or assigned a value, yet its value is used)

Constant shelves can now be used in places where constant values used to be required. For example:

      constant integer months-per-year initial {12}
      global stream months size months-per-year

Function arguments can now be declared write-only. If an argument is declared write-only, the following conditions apply:

  • The function call must pass an actual existing shelf, not a constant or the result of a calculation.
  • The function can write to the shelf.
  • The function cannot read from the shelf.
  • The function cannot address individual fields of an item on the shelf, it can only add or remove whole items.
  • The type of the shelf must match the declared type or be a type from which the declared type is an extension.
  • If the function call specifies a particular item on the shelf, that item is treated as the default item inside the function, but the whole shelf is still available.

New in the libraries of OmniMark 7.1

The OMRTF library allows you to parse RTF documents, much as you parse SGML or XML documents.

The OMVFS library now has functions that enable you to access files using the HTTP, HTTPS and FTP protocols.

New in the language of OmniMark 7.0.2

The Web Services Broker allows you to host web services and direct web service requests.

You can use #schema to retrieve a schema for an XML document or to test if an external text entity represents an XML Schema.

The join operator || is now streaming in all contexts. In earlier versions, join was buffered in some contexts.

OmniMark 7.0.2 supports two new "content is ..." tests:

  • content is empty-tag returns true when the element being asked about was specified using an empty element tag. For XML this is a tag of the form "<foo/>".
  • content is declared-empty returns true when the content model of the element being asked about is empty.

Module interfaces have been created for the following libraries:

  • OMDB, the database access library,
  • OMLDAP, the ldap support library,
  • OMNETUTIL, the network utilities library,
  • OMTCP, the TCP/IP client and server library, and
  • OMUTIL, the utilities library.

The functionality that was previously found in OMUTIL has been reorganized into three modules:

  • UTF8PAT, the UTF-8 processing library, has been renamed to OMUTF8
  • OMNETUTL, the network utilities library, has been renamed to OMNETUTIL (to make it consistent with OMUTIL).

New in the language of OmniMark 7

  • Modules. A module is a self-contained unit of OmniMark code. It can have its own include files, macros, rules, groups, globals and functions. It has control over what names its "clients" (includers) see, and the client has control over what names it uses for the things it gets from the module. It is useful to think of a module as being like an "include" file, but with control over the names defined and used within it. By convention, OmniMark modules are placed in files whose names end with the ".xmd" suffix, in the same way that OmniMark programs use ".xom" and OmniMark include files use ".xin". For more detailed information about OmniMark Modules, please see the Modules page.
  • Quoting user-defined names. You can put a name inside quotes (single or double), so long as you immediately precede the first quote by a hash sign:
        set #"a" to #"b" #"+" 1
       
    
  • Constants. A constant is like a global except that it is immutable-- once its initial value is set, it can never be changed. For more information on constants, please see the global, local, constant page.
  • Value-source and value-output. Value-source and value-output external function parameter types have been added to the language. These aid in writing stream external filter functions.
  • The initial value of optional function arguments can now be dynamic values.The default initial value is calculated each time the function is called and the default value is needed. This means that default values can now depend on global and constant values.
  • Support for using #current-input in external functions has been added to the external function api.
  • OmniMark now supports filter functions: external functions that can filter while data is being read, or can filter while data is being written. For more information on this feature, please see Filter Functions.
  • It is no longer necessary to specify size for a multi-item fixed-sized global, constant or local shelf if you provide initial values.
  • Input functions have been generalized. An input function is now declared as being of the type "input" rather than being treated as an input function because of the way it is called in an XML or SGML parse. This means that input functions are usable in all contexts and can be usefully used in place of stream returning functions. See Functions and Input functions and the markup parserfor details.

New in the libraries of OmniMark 7

  • An XSLT processing support library, omxslt, is available. The omXSLT library allows OmniMark programs to perform XSLT transformations on input and output data. It also allows you to feed XSLT transformations into OmniMark's internal XML parser or the pattern-matching engine.
  • An external buffer library, omiobuf, is available.
  • Six new filter function libraries have been created which repackage the international character set functions of omefio so that they have streaming capability.These are omffbase64 (base 64 conversion filters), omffbig5, (big 5 conversion filters), omffeuc (EUC conversion filters), omffjis (JIS conversion filters), omffsjis(Shift JIS conversion filters) and omffutf16 (utf-16 conversion filters).
  • The MQSeries access library, ommqi, provides an interface between OmniMark and IBM MQISeries.
  • Some existing libraries are also provided as modules, in addition to the old include file form: omfloat, omtrig, ombessel, ombcd, omvfs, and omxerces.

New in OmniMark Studio 7

  • A new debug window has been added to display information about constants.
  • Small modifications have been made to the shelf details window.

New in the language and libraries of OmniMark 6.5

The following features have been added:

  • The external markup parser interface provides W3C schema support.
  • Namespace handling is facilitated through new markup rules.
  • The omxerces Xerces external markup parser library makes it possible to parse XML using the Xerces markup parser with rules written in OmniMark.
  • Hash-quoted names: you can now have names (variable, function and otherwise) in OmniMark programs such as #"foo". They may contain any characters that are legal for strings, and any of the static format items (such as %n, %%, etc.)
  • Input functions: functions can be defined with type "input". An input function is used like a stream function, but its return value is the output it generates. Input functions have no return statement. For example:
      define input function foo
      as
         output "some stuff%n"
         output " more stuff%n"
    
  • Support for Linux has been upgraded to Redhat 8
  • The omfloat floating point number support library is now available on HP/UX 11.00
  • The omvfs virtual file system library has two new functions: vfsChangeOwner and vfsChangePermissions
  • All OMXs now have Display subfunctions

New in OmniMark Studio 6.5

  • Catch parameters now appear in local variable tables.
  • Small modifications have been made to the shelf details window.

New in the language and libraries of OmniMark 6.1.3

The following features have been added:

OMDB library

  • OMDB now supports OCI8 and OCI8i database interfaces.
  • The ability to stream data into database fields was added. This means that large data can be inserted into databases without having to buffer all of it in OmniMark.

Installers

  • The authorization key request function has been integrated into the OmniMark product for Unix and Linux machines. It is no longer shipped as a separate utility.

New in the language and libraries of OmniMark 6.1

The following features have been added:

omutil library

  • the omutil - miscellaneous system utilities library now contains a suite of sorting functions that allow you to sort items of a shelf by item or by key:
    • UTIL_SortIntegersByKey
    • UTIL_SortIntegersByValue
    • UTIL_SortStreamsByKey
    • UTIL_SortStreamsByValue
    • UTIL_SortSwitchesByKey

HTTP library

  • All the HTTP functions are now available in one library, omhttp - http support. This includes both client and server functions.

FTP library

  • All the FTP functions are now available in one library, omftp - ftp support. This includes both client and read/write functions.

HPUX

  • HPUX is now fully supported as a platform for OmniMark.

New in OmniMark Studio for Eclipse 6.1

OmniMark Studio for Eclipse now does the following

  • Displays a dialog box that tells you how to get correct authorization if there is an authorization failure (licensing failure) in OmniMark Studio for Eclipse

For further information, see the OmniMark Studio for Eclipse Windows help file.

New in the language and libraries of OmniMark 6.0.2

The following features have been added:

omtcp library

  • There is a new interface to omtcp which uses exceptions. The previous TCP/IP server and client support library has been renamed as omtcp-old. The include file omtcp.xin gives you access to both libraries.

New in OmniMark Studio for Eclipse 6.0.2

OmniMark Studio for Eclipse now does the following

  1. Lets you choose Highlight Currently Matched Data in the Parsing Sources Datascope
  2. Responds correctly when you choose Save Project as Args File from the File Menu (it creates an Args file that contains the -load option)

New in the language and libraries of OmniMark 6.0.1

The following features have been added:

omdb library

  • A new function has been added: dbRecordSetMove function, which allows multiple differently-structured result sets.
  • The "where" argument of dbTableUpdate and dbTableDelete functions is now mandatory.

omvfs library

  • Full support for local paths, including drive letters on Microsoft Windows, has been added.
  • Full support for RFC 1738 and 1808 compliant (encoded) URLs for file protocol has been added.
  • The functions vfsMove and vfsCopy have been enhanced to support moving items into an existing directory.
  • Error messaging has been enhanced.
  • vfsRename has been deprecated (use vfsMove).

New in OmniMark Studio for Eclipse 6.0.1

OmniMark Studio for Eclipse now does the following

  1. Has a reload operation on the File menu to replace a file in the edit window with one that has been saved on disk.
  2. Lets you use arguments files (.xar files) in Studio projects
  3. Warns you when saving a file if file on disk has been changed
  4. Tells you when a file has changed on disk, whenever you bring the file's edit window to the front or whenever your program in Studio processes the file

New in the language and libraries of OmniMark 6.0

The following features are new in version 6.0:

** and ++ occurrence indicators

For years, OmniMark programmers have been writing and using code like the following:

     ((lookahead not <terminating-pattern>) any)* <terminating-pattern>

Now you can write:

     any** <terminating-pattern>

See pattern matching for details.

The ++ occurrence indicator has also been added. It is identical in function and form to the ** occurrence indicator except that it is required to match one or more characters from the given character set before the terminating pattern.

In other words, if the terminating pattern is the next sequence in the input, ++ will fail whereas ** would have succeeded.

repeat for loop

OmniMark now has a repeat for loop. You can write loops that repeat for each allowed value of a control variable:

  repeat for integer i from 12 to 144 by 6
     output "d" % i || "%n"
  again

repeat to

A new looping construct, repeat to has been added. It allows you to repeat a loop a specified number of times.

Pattern variables in lookahead

You can now capture the pattern matched in a lookahead to a pattern variable:

  find lookahead any => next-char
     output "The next character in the input is '"
         || next-char
         || "'.%n"

Zero-length pattern matching and the null modifier

You can allow a pattern to match zero characters, even where OmniMark's restriction on two consecutive zero-length pattern matches in the same scan would normally prevent it, by prefixing the pattern with null.

Instance keyword now optional when parsing well-formed XML

The keyword instance is now optional when you are parsing well-formed XML using do xml-parse.

For example, the following routine will parse well-formed XML:

  do xml-parse
       scan file my-instance
       output "%c"
    done

If you are parsing XML using a DTD, however, you need to specify "do xml-parse document".

New indexing forms

Item indexing of a shelf has been replaced by '[ numeric-expression ]'

For example:

     set lookuptable[5] to "epsilon"

Key indexing of a shelf has been replaced by '{ string-expression }'

For example:

     set lookuptable{"epsilon"} to 5

These new forms are supported in the following contexts:

  1. Indexing a shelf, attribute list, or referents
  2. do markup-parse with dtds { ... }
  3. new <shelf-name> after/before [...] to <value>
  4. new <shelf-name> after/before {...} to <value>
  5. set new <shelf-name> after/before [...] to <value>
  6. set new <shelf-name> after/before {...} to <value>
  7. set <shelf-name> [...] to <value>
  8. set <shelf-name> {...} to <value>

positional patterns

Positional patterns are now allowed in more contexts.

New pattern matching synonyms

A \ inside character class means "except"

The symbols |= and =| are synonyms for value-start and value-end.

Take and Drop in pattern matching

The new take and drop operators enhance OmniMark's string manipulation.

New modifier on #main-output

You can now specify that the stream #main-output has domain-free.

Identity operator

The identity operator (~)is used to distinguish shelf expression from pattern expressions within a pattern.

Overloaded operators and dynamic initialization

OmniMark has been adding overloaded operators over the past few releases. This work is not yet finished. However, new libraries, notably the Binary Coded Decimal library, are taking advantage of the work done to date.

As part of this ongoing work, the latest visible functionality is the addition of dynamic initialization of variables. Dynamic initializers can be applied to both global and local variables.

Numbers with underscores

Underscores are now allowed in numbers, including integers and BCD numbers, but not in floating point numbers.

For example, you can specify an integer with underscores:

  process
     local integer x
     set x to 1_000_000
     output "Number is %d(x)"
  ; The output will be "Number is 1000000"

You can also specify a BCD number with underscores:

  include "ombcd.xin"
  process
     local bcd y
     set y to 1_000_000
     output "BCD Number is " || "<,NNZ.ZZ>" % y
  ; The output will be "BCD Number is 1,000,000.00"

Less casting

When a literal of an externally-defined numeric type is used, it no longer requires explicit type casting where the intended type of the numeric literal is clear.

assert

You can now add assert statements to your OmniMark programs.

#args

#args is now a synonym for #command-line-names

log-message change

log-message is now part of the language and is no longer part of the builtins library. You no longer need to include "builtins.xin" to use log-message.

Utilities library changes

The global stream shelf, UTIL_Env, has been added to the utilities library. This shelf is populated with all of the environment variables. Using this shelf makes it easy to find HTTP parameter variables.

BCD and floating point libraries (ombcd and omfloat) changes

New versions of the BCD and floating point libraries have been released, and the operations allowed in both libraries are largely identical.

Both the floating point and the BCD libraries now use these arithmetic operators and the comparison operators in the same way:

  • + (add)
  • - (subtract)
  • * (multiply)
  • / (divide)
  • modulo (find the remainder)
  • = (equal-to)
  • < = (less-than-or-equal-to)
  • < (less-than)
  • >= (greater-than-or-equal-to)
  • > (greater-than)
  • != (not-equal-to)

The absolute value function, abs, has also been extended to the floating point library.

The new power operator, **, has been added to both the floating point library and the BCD library.

The floating point library now implements the same rounding operations as the BCD library:

  • ceiling (round up)
  • floor (round down)
  • round
  • truncate

Both the floating point library and the BCD library now support the same exponential functions:

  • exp (raise e to any power)
  • log (calculate the natural logarithm)
  • log10 (calculate the base 10 logarithm)
  • sqrt (calculate the square root)

Both libraries define the constant e.

Formatting in BCD and floating point

The format operator, %, has been extended to the floating point library so that floating point numbers can be formatted in the same way as BCD numbers.

The format operator supports both template formatting and the "d" format command. The "d" format modifiers ("u", "l", "f", and "k") are now supported for floating point numbers.

The "d" format command supports the new "e" format modifier for both floating point and BCD numbers. This modifier formats a number in exponential notation.

Trigonometry library (omtrig)

The OmniMark trigonometry library is a collection of trigonometric functions based on the C math library. It serves as a "wrapper" around a particular math library. You can use the functions in this library to write OmniMark applications that can perform double-precision floating point arithmetic.

The functions in this library are the following:

  • acos (arc cosine)
  • asin (arc sine)
  • atan (arc tangent, in radians))
  • atan2 (arc tangent, in degrees)
  • cos (cosine)
  • cosh (hyperbolic cosine)
  • hypot (hypotenuse)
  • sin (sine)
  • sinh (hyperbolic sine)
  • tan (tangent)
  • tanh (hyperbolic tangent)

Bessel library (ombessel)

There is a new Bessel function library, ombessel, that includes the Bessel functions j0, j1, jn, y0, y1 and yn.

What's been deprecated

item and key in all contexts where they serve as an indexer. Use [...] and {...} respectively instead.

@ and ^ are deprecated in all contexts.

The following functions and constants are now deprecated (but still supported for backwards compatibility): FP_add, FP_sub, FP_mul, FP_div, FP_mod, FP_eq, FP_le, FP_ge, FP_lt, FP_gt, FP_ne, FP_pow, FP_ceil, FP_floor, FP_nearest, FP_trunc, FP_exp, FP_log, FP_log10, FP_sqrt, FP_s, FP_d, FP_v, FP_e, FP_pi, FP_cos, FP_sin, FP_tan, FP_acos, FP_asin, FP_atan, FP_atan2, FP_cosh, FP_sinh, FP_tanh, FP_hypot.

The omvfile library has been removed from the product. Use omvfs instead.

The omfsys library has been deprecated. Use omvfs instead.

Incompatibilities

External libraries compiled for previous versions of OmniMark are not compatible with OmniMark 6. This includes both libraries supplied by OmniMark Technologies and third party libraries. Older libraries must be recompiled with the OmniMark 6 external function developers library before they can be used with OmniMark 6.

With the introduction of dynamic initializers, OmniMark no longer supports multiple global variable definitions. In earlier versions the following was legal:

     global integer message-count
     global integer message-count

Now this is not legal code. One of the declarations must be removed.

New in OmniMark Studio for Eclipse 6.0

OmniMark Studio for Eclipse now does the following

  1. Displays data in two new datascopes in three viewing modes: Text, Raw and Dump
  2. Lets you drag and drop files onto the Edit window
  3. Expands variables to show their shelf items on click of plus sign
  4. Provides Global and Local variables windows (instead of an "All Visible Variables" window)
  5. Lets you drag and drop variables from Global or Local variables to the Watched variables table
  6. Displays a variable's details on double-click or from Run menu
  7. Displays a referent's details on double-click or from Run menu
  8. Constrains display of a variable to certain items or ranges on right-click for "Display only"
  9. Lets you select which shelf item of a variable to watch
  10. Lets you compress variable display on item or on key
  11. Throttle now has a Stop button
  12. Edit preferences window has a variables tab to customize variables tables
  13. Details of variable allows display of Text, Raw or Dump mode for streams, OMX variables, and pattern variables
  14. Log window has right-click option to "Clear log window on run"
  15. Provides a redesigned Project options dialog box
  16. Traps OmniMark run-time errors and throws
  17. Stops on the first breakable line or runs to completion
  18. Colors the syntax in other programming languages
  19. Supports block operations for editing files
  20. Displays the cursor's row and column position on the status bar
  21. Lets you set bookmarks anywhere in your file
  22. Lets you split a document into multiple views
  23. Lets you dock and undock toolbars
  24. Provides tabbed edit windows (window menu has been removed)
  25. Provides a drop-down menu of recent files after the Open file and Open project toolbar buttons
  26. Provides a drop-down menu of open projects after the Start Debugging button on the Debug toolbar
  27. Lets you close the Project manager from a close button (and maximize the Edit window)

New in OmniMark 5.4

The following features are new in version 5.4:

well-formed XML

Well-formed XML parsing has been added.

New in OmniMark 5.3

The following features are new in version 5.3:

General

OmniMark 5.3 introduces the Binary Coded Decimal (BCD) data type. BCD is the correct data type for performing many mathematical calculations and all types of financial calculations.

OmniMark's BCD implementation supports large numbers, limited only by available memory and 16 decimal places of accuracy. In addition, new formatting operations provide a way to customize the representation of BCD numbers according to the local conventions of most countries.

To use the BCD data type, you must import the module "ombcd.xmd". Using BCD is syntactically similar to mathematical operations on counters, although in some circumstances the BCD number must be "cast".

The IDE (the predecessor to OmniMark Studio for Eclipse) now accommodates BCD numbers.

OmniMark language

Heralds removed

Heralds have been removed from the language to make room for "casting". Also removed are the declaration declare heralded-names and the command-line option "-heralds".

V2 programs will no longer compile. To make them compile, you need to declare all variables and remove the words counter, stream, buffer, and switch that precede variables in your program.

Operator syntax

Operator recognition has also been changed to use longest match. For example, prior to OmniMark 5.3 the style of the following find rule was legal:

           find "a"+=>x

In OmniMark 5.3, the sequence "+=>" is still recognized as "a"+ followed by =>, but a warning is issued stating that this is deprecated. In a future version of OmniMark this will be reported as an error, unless "+=>" is defined as an operator. To remove the warning, write the code like this:

           find "a"+  => x

#program-error location argument

The format of the location argument of the #program-error catch has changed from "on line <line-number> in file <file-name>:%n" to "line <line-number>, file <file-name>".

Integers

OmniMark introduces two new integer data types called integer and int32. Integer will replace the counter data type in all cases except bit operations and "network long" calculations, which will be handled by int32. The default value for both integer and int32 is 0.

The introduction of the integer data type is a phased release. Programmers should use integer instead of counter, and begin phasing out the use of counter in existing programs.

Warnings on deprecated keywords

The several deprecated keywords now cause warning messages. The deprecated keywords, and their replacements, are as follows:

Libraries

LDAP

LDAP is an acronym for "Lightweight Directory Access Protocol", a client-server protocol to retrieve and manage directory information. The OmniMark LDAP legacy library is introduced. It contains functions that let you create connections to different LDAP data sources and let you display, create, modify, and delete entries in those sources. In 7.0.1, this library is replaced by the newer LDAP library.

New in OmniMark 5.2

The following features are new in version 5.2:

OmniMark language

is catchable

You can now test to see if a catch is available before initiating a throw using the is catchable test.

is thrown

You can now test to see if a throw is in progress using the is thrown test.

using input as

The using input as keyword has been added to allow you to change the current input scope without initiating a scan.

with utf-8

The with utf-8 modifier has been added to do xml-parse and do sgml-parse to allow you to specify how the parser handles character entities greater than 127.

HTML 4.0

You can now process HTML 4.0 as SGML. The SGML parser in OmniMark 5.2 now supports large character values and other SGML features used in the HTML 4.0 prolog. You will need to use an appropriate HTML 4.0 SGML declaration when parsing HTML 4.0 files.

Case-sensitive XML parsing

XML parsing is now case sensitive.

Namecase general and namecase entity obsolete

OmniMark now reads the NAMECASE GENERAL and NAMECASE ENTITY declarations from the SGML declaration. The OmniMark namecase general and namecase entity declarations are now obsolete.

For this reason it is now a compile-time error to have a namecase general no declaration in a program that uses attribute aliases. If you get this error, check carefully for any ambiguity in your use of attribute aliases before removing namecase general no from your program.

SGML quantities

The OmniMark SGML parser now accepts SGML files that use element names longer than eight characters without the need to supply an SGML declaration. A number of other SGML quantities also have larger default values.

Empty pattern matching

It is now permissible, in all situations, to write a pattern that matches zero characters. This is particularly useful for detecting the end of nested structures when you are doing nested pattern matching.

include-guard

The new include-guard makes it easier to ensure that a file is not included twice in your source code.

Large file support

On Solaris 2.6, OmniMark will now correctly handle files that contain more than 2 gigabytes of data.

Library changes

Extended Database Xpack

The new Database Extensions library in this OmniMark Developer's Network Xpack offers support for stored procedures and transaction processing.

HTTP library

The HTTP library now supports proxy servers.

Run-time environment

Project files as arguments files

You can now use OmniMark IDE (the predecessor to OmniMark Studio for Eclipse) project files as arguments files with OmniMark VM, OmniMark C, and OmniMark C/VM.

Library, include, and temporary file paths

Library, include, and temporary file paths can now be set globally using environment variables and/or Windows registry entries. In Windows, suitable default values are set during installation. See Developing and running OmniMark programs in OmniMark 4.0 documentation, or the OmniMark Engine documentation for OmniMark 6.0 onwards.

-noenvargs command-line option

You can use -noenvargs, or its short form, -noea, to tell OmniMark to ignore the library, include, and temporary file path settings in the environment.

New in OmniMark 5

The following features are new in version 5:

OmniMark 5 includes several new language features and development tools.

Language features

Catch & Throw

You can use catch and throw to manage the execution flow in your OmniMark programs. Catch and throw is a powerful addition to the flow handling features of OmniMark, allowing you to make major redirections of program flow in a safe and structured way.

Buffered, Unbuffered, Flush

Buffered and unbuffered are two new stream modifiers that allow you to control the buffering characteristics of the input and output streams your OmniMark program uses. The ability to declare the #main-input stream as unbuffered is one of the key features that allow you to do CGI programming with OmniMark. Accompanying buffered and unbuffered is the new flush keyword, which lets you force a buffered stream to send the data it contains.

Nested pattern matching

The new keyword #current-input has been added. This lets you use the technique of nested pattern matching to simplify the processing of data that contains nested structures.

Pattern matching functions

You can now write pattern matching functions, functions that participate in the pattern matching process. This greatly enhances the power and flexibility of OmniMark pattern matching.

Direct support for CGI Programming

OmniMark now supports the #! directive used by UNIX shells and some web servers to associate a script with the appropriate runtime. This greatly eases the writing of CGI programs in OmniMark.

Default variable shelf size is now 0

The default size of a shelf declared variable is now 0. It was 1 in previous versions. 0 is the appropriate default in almost all programming situations.

Command-line option changes

-sb command-line option added

The new -sb command-line option is a combination of "-s" and "-brief". You can use this option to specify the name of the OmniMark program to execute, and to suppress the OmniMark banner information normally output when you run an OmniMark program from the command line. This command-line option was included to use in CGI programs.

-expand command-line option removed

The -expand command-line option has been removed from this release. This was done to address security concerns when using OmniMark to run CGI programs. The functionality provided by -expand is no longer required now that the IDE (the predecessor to OmniMark Studio for Eclipse) is available.

Libraries

Database

A new high-level database library is included in this release. It is much easier to use than the ODBC library, which is still available for use in applications that require its low level functionality.

CGI

Developing CGI applications with OmniMark is a simple process made simpler with the addition of the CGI library. This library provides routines to retrieve and decode the information that the web server sends using the CGI protocol.

OMX components

We have renamed opaque objects OmniMark eXtension (OMX) components. OMX components extend the functionality of OmniMark the same way Active X and other component types are used to extend other programming languages. OMX components perform the same essential function as other component technologies: they encapsulate complex functionality and allow the OmniMark programmer to easily deal with complex systems such as FTP servers, databases, and TCP/IP connections. In keeping with the streaming nature of the OmniMark language, many OMX components are capable of acting as sources and sinks. You can read from or write to the OMX just as you would to a file or buffer. This greatly simplifies many aspects of network programming.

Development tools

Integrated Development Environment

The OmniMark IDE (available on Windows 95/98/NT) is a graphical user interface application that includes a full suite of editing and debugging tools.

The IDE editing environment:

  • automatically color-codes your OmniMark syntax
  • allows you to set breakpoints in your code for debugging
  • allows you to quickly and easily set up and modify your program execution parameters

The IDE debugging tools let you:

  • watch your input data as it is processed
  • view the values of your program variables as they are dynamically updated during debugging
  • step incrementally through your code in a variety of ways so you can quickly identify and repair trouble spots
  • jump instantly to error-causing lines when OmniMark encounters compile-time errors

OmniMark VM

OmniMark VM is a stand-alone version of the OmniMark virtual machine. You can use it to run compiled OmniMark byte codes files (*.xvc).

New in OmniMark 4.0

The following features are new in version 4.0:

OmniMark 4.0 includes numerous new features and changes that have been implemented to extend functionality and increase program flexibility.

XML document processing

OmniMark 4.0 supports XML parsing, a variant of SGML parsing that supports the key features of the Extensible Markup Language (XML) specification (as defined in WD-xml-970807). The addition of this feature allows OmniMark to successfully parse a valid XML document. With the addition of this feature, some changes and additions were made to the OmniMark language. Most significantly, a set of generic "markup" keywords was introduced to generalize, as much as possible, OmniMark functionality that was previously specific to the SGML parser. With this generalization, programmers can interact with a parser without having to specify which one, SGML or XML.

Opaque types

<Opaque data types have been added, allowing programmers to define their own data types to be used with external functions.

The "no-default-io" declaration

This declaration has been added to simplify the process of writing and testing server programs. OmniMark programs without default input/output sources and destinations are expected to be servers with their own explicitly defined and controlled inputs and outputs. Using the no-default-io declaration allows you to run an OmniMark CI program in a simulated virtual machine environment.

Format items and the format operator

The format operator has been added to increase the flexibility of format item usage. This feature reduces the amount of coding required in complex output formatting tasks. With the implementation of this operator, format modifiers can also be dynamically generated rather than having to be statically coded into a program.

Changes to "find" and "submit"

With this release of OmniMark, find rules and the submit action can be used almost anywhere. The most important effect of this change is that submit can now be used in the body of process rules.

New date format item

An "=t" has been added to the format items that can be used with the date and compiled-date keywords. This format item produces the offset in hours and minutes from Greenwich Mean Time.

New command-line options

An "-x"command-line option has been added, which is described in the OmniMark Engine documentation. This option allows programmers to specify the path of external function libraries from the command line rather than having to "hard code" them into an OmniMark program. This option can also be used to increase cross-platform portability of OmniMark programs that use external function libraries. The file-name extension of the external function library (which is different on most platforms) can also now be specified on the command line rather than in the OmniMark program itself.

The "-stats" command-line option has also been added. When specified, this option causes an OmniMark program to generate and output data regarding memory usage and execution times.

External functions changes

Two changes have been made to external functions. External functions now accept remainder and optional arguments. Additionally, external functions support generic functions that make use of various return types, argument counts, and argument types determined at run-time. The external functions API has also undergone some changes.

Function libraries

The following have been added to the OmniMark function libraries suite. Please note that some of these libraries can be used only in OmniMark 4.0 and beyond because they utilize opaque types:

  • #builtin. The external functions in this library provide access to functionality in the run-time environment.
  • Date and time. This collection of functions defines a set of operations on dates and times. They are built around a standard normalized form for the date/time, a 19-character string in "YYYYMMDDhhmmss+ZHZM" format.
  • Encoded file I/O. This external function library allows you to do a variety of character encoding conversions on file input or output.
  • File System Utilities. This collection of external functions provides access to files and directories from within an OmniMark program.
  • Floating Point. This is a collection of basic arithmetic, trigonometric, Bessel, exponential, and miscellaneous math functions based on the C math library.
  • FTP. This external function library provides FTP functionality to OmniMark programs.
  • HTTP. This is a collection of OmniMark internal functions that provides access to Hypertext Transmission Protocol (HTTP) messages and communication.
  • I/O Exception Object. This external function library allows you to create and manipulate an "io-exception" opaque object. This type of opaque object can be used extensively in the Encoded File I/O library.
  • Mail. This set of external functions allows you to send and receive electronic mail through OmniMark programs.
  • Network Utilities. This library is a set of functions that provide miscellaneous network-related services.
  • OCI. This external function library provides access to Oracle databases from within an OmniMark program, allowing you to access all of the Oracle Call Interface (OCI) functions to view and manipulate data within the Oracle database.
  • ODBC. This library gives you access to ODBC sources from within an OmniMark program using your existing ODBC-compliant database software drivers. The functions provided in this library allow you to access all of the ODBC 3.0 functions used to view and manipulate data within a data source.
  • SGML Open Catalog. This function library provides the services described in the SGML Open Technical Resolution 9401:1997, Amendment to TR 9401.
  • legacy System Utilities. This external function library provides miscellaneous services not available in other OmniMark function libraries.
  • Legacy TCP/IP library. This library allows you to make TCP connections to the outside world from within an OmniMark program via the Transmission Control Protocol/Internet Protocol (TCP/IP). (Superceded by the newer TCP/IP library)

New in OmniMark 3

The following language features have been added in OmniMark versions 3 and 3.2: