Aviation Message Archiver

An application that reads aviation message files, parses some basic properties and stores messages along with parsed metadata in a database.

Overview

Aviation Message Archiver is a Spring Boot service application. It monitors configured input directories for aviation message files. Whenever new files appear, it scans for messages in files, parses some basic properties like message (issued) time, validity period (when applicable), station / location indicator. It stores each message in a PostGIS (PostgreSQL) database.

Feature overview

Configurable archival process.
Automatic message type recognition.
Supports TAC (experimental) and IWXXM message formats.
Supported file formats:
- WMO GTS meteorological message as specified by WMO Doc. 386.
  - Multiple meteorological messages may be included in a file.
  - A supported COLLECT document is also supported as message content.
- COLLECT 1.2 document
- A meteorological bulletin starting with a WMO GTS abbreviated heading line, followed by messages or a COLLECT document
- A single message (not recommended for TAC messages)
Supported database engines:
- PostGIS (PostgreSQL)
- H2 (for testing)
Traceable logging. Support for structured JSON logging.
Built on Spring Boot and Spring Integration.

Supported message types and formats

Supported message types and formats are listed in the table below. Generally, this application supports all

TAC format message types that are supported by fmi-avi-messageconverter-tac library TAC_TO_GENERIC_AVIATION_WEATHER_MESSAGE_POJO conversion and
IWXXM format message types that are supported by fmi-avi-messageconverter-iwxxm library IWXXM_STRING_TO_GENERIC_AVIATION_WEATHER_MESSAGE_POJO conversion.

Message type	TAC	IWXXM 2.1	IWXXM 3.0	IWXXM 2023-1
METAR	1)	+	+	1)
SPECI	1)	+	+	1)
TAF	1)	+	+	-
SIGMET	-	+	+	+
AIRMET	-	+	+	+
Volcanic Ash Advisory	1)	n/a	+	-
Tropical Cyclone Advisory	-	n/a	+	-
Space Weather Advisory	1)	n/a	+	-

+ Complete support
- Unsupported
1) Experimental

Getting started

The next steps guide you to test the application with example configuration using H2 (in-memory) or PostGIS database engine.

After cloning the code repository, build the application with Maven.
```
mvn package
```
Set up the database engine.
- H2: is automatically set up at application startup, no actions needed.
- PostGIS: Database is easily set up with Docker or Podman. Use credentials specified by spring.datasource.* properties in the application.yml configuration for profile local & postgresql & !openshift.
```
docker run \
  -p 127.0.0.1:5432:5432 \
  --env POSTGRES_USER=avidb_agent \
  --env POSTGRES_PASSWORD=secret \
  --env POSTGRES_DB=avidb \
  --name avidb \
  docker.io/postgis/postgis:14-3.2
```
In the local mode used in this guide, the application will automatically initialize the schema at startup.
Prepare an SQL script to populate the avidb_stations table. This is optional for testing the application, but all messages will be rejected without a matching location indicator in the icao column of avidb_stations table.
- H2: See schema-h2.sql for the schema, and h2-data/example/avidb_stations.sql for an insertion template.
- PostGIS: See schema-postgresql.sql for the schema, and postgresql-data/example/avidb_stations.sql for an insertion template.

Start the application. Replace

$AVIDB_STATIONS_SQL with a path to the file created in previous step, or omit the spring.sql.init.data-locations property.
$DB_ENGINE with h2 or postgresql.

java \
  -Dspring.profiles.active="local,example,$DB_ENGINE" \
  -Dspring.sql.init.data-locations="\${example.spring.sql.init.data-locations.$DB_ENGINE},file://$AVIDB_STATIONS_SQL" \
  -jar target/aviation-message-archiver-1.4.1-SNAPSHOT-bundle.jar

Check some actuator endpoints to see that the application is running and healthy.
- info: http://localhost:8080/actuator/info
- health: http://localhost:8080/actuator/health
Copy some message files in the input directories specified by the production-line.products[n].input-dir properties in the application.yml configuration file.
After an input file is processed, the application moves it to one of target directories specified by the production-line.products[n].archive-dir and production-line.products[n].fail-dir properties in the application.yml configuration file. The processing identifier is appended to the file name.

Connect to the database.

H2: You can access the H2 database console at http://localhost:8080/h2-console/login.jsp with default connection settings and credentials.
PostGIS: Use psql or any appropriate client with connection information provided in the database setup step.

Look at the archived and rejected message tables in the database for any messages. E.g.

SELECT *
FROM avidb_messages AS messages
         LEFT JOIN avidb_message_iwxxm_details AS iwxxm ON iwxxm.message_id = messages.message_id
ORDER BY message_time DESC;
SELECT *
FROM avidb_rejected_messages AS r_messages
         LEFT JOIN avidb_rejected_message_iwxxm_details as r_iwxxm
                   ON r_iwxxm.rejected_message_id = r_messages.rejected_message_id
ORDER BY message_time DESC;

Logging

Application logging is designed to enable tracking of file processing and possible errors during the process. This section describes implemented mechanisms supporting this goal.

Processing identifier and phase

Processing identifier is a unique string within a single Java virtual machine that identifies a single processing of a single input file. It connects all separate log entries of the file processing. Processing identifier 0 or null denotes that a processing identifier is not available or applicable on the log entry. See FileProcessingIdentifier class for details.

Processing phase describes the phase in the processing chain. Processing phase nul or null denotes that the processing phase information is not available or applicable on the log entry. See ProcessingPhase enum for list of phases.

When using unstructured logging, the processing identifier and phase are logged before logging level in format:

[<processing-id>:<phase>] <level> ...

Processing context

Most log entries contain processing context information, referring to file being processed, bulletin of the file and message within the bulletin. The format of the processing context in a log message is:

<fileReference:bulletinReference:messageReference>

which can be extracted to:

<productId/fileName:bulletinIndex(bulletinHeading)@bulletinCharIndex:messageIndex(messageExcerpt)>

where

productId: identifier of the product specifying the file under processing
fileName: name of the file under processing
bulletinIndex: index of bulletin within file, starting from 0
bulletinHeading: heading (GTS or collect identifier) of the bulletin under processing
bulletinCharIndex: character index of bulletin within file, starting from 0
messageIndex: index of message within bulletin, starting from 0
messageExcerpt: excerpt from beginning of a TAC message, or IWXXM message gml:id

Unavailable or inapplicable fields and separators are omitted.

See ReadableLoggingContext interface for more information.

Processing statistics

When file processing is finished, the application logs statistics of the processing in the following format:

M{X:n,...,T:n} B{X:1,...,T:n} F{X}

where

M{} contains statistics counted as messages within the file.
B{} contains statistics counted as bulletins within the file.
F{} contains the overall result of the file processing.
X:n contains the count of items having the specified processing result. Any processing results with zero (0) count are omitted.
- X is the abbreviated one-letter name of the processing result. See ReadableFileProcessingStatistics.ProcessingResult enum for possible values.
- n is the count of items resulting with the preceding processing result.
T:n contains the total count n of processed items.

Aggregated results denote the worst processing result of a message within the aggregation context. E.g. a bulletin is considered rejected when it contains at least one rejected message, but no failed messages.

See ReadableFileProcessingStatistics interface for more information.

Structured logging

Structured JSON logging can be enabled by activating the logstash runtime profile in the startup command. E.g.

java -Dspring.profiles.active=<other profiles...>,logstash ...

Application configuration

Application configuration properties are collected in a YAML file called application.yml. The provided configuration file is a base configuration, acting as an example. You can use it as a base for your own application configuration file. In your custom configuration file you need to add and/or override only changed or forced properties in your own configuration file, because the provided base configuration file is loaded as well. See External Application Properties in Spring Boot reference documentation for instructions on how to apply your custom configuration file.

Runtime behavior is controlled using Spring profiles which are activated by the application launch command. Profiles declared in the provided configuration are described in the application.yml file.

The most relevant part of the configuration file is the production line configuration under production-line property. Its contents are described below.

Note: Invalid configuration does not necessarily raise an error on startup, but may be silently ignored.

Products

Aviation product model describes basic information like input and output directories and accepted file name patterns. This application can be configured for one or more products. Template of products configuration:

production-line:
  products:
    - id: <product id>
      route: <route name>
      input-dir: <input directory>
      archive-dir: <archived files directory>
      fail-dir: <directory for failed files>
      files:
        - pattern: <file name regex pattern>
          name-time-zone: <zone of timestamp in file name>
          format: <message format>
        - ...
    - ...

See application.yml file for a documented configuration example. More detailed documentation on individual properties can be found in the AviationProduct model class.

Message processors

Message processor is a concept of small components that are designed to do a single action on each message at a certain phase of the archival execution chain. The message processor instances used in the archival process are configurable in the application configuration, and their activation may be conditional based on message properties.

Currently, the application supports two kind of message processors: message populators and post-actions. See corresponding chapters for more details.

Message populators

Message populators are small components that are responsible for populating the archive data object with message data parsed from the input file. Each message populator focuses on a single responsibility, and together all configured populators construct the complete message entity to be archived. Message populators also decide whether a message is considered

eligible for archival, thus will be stored in the message database table after all populators are executed,
rejected, thus will be stored in the rejected message database table after all populators are executed,
discarded, thus will be ignored immediately and logged at info level,
failed, thus will be ignored immediately and logged at error level.

Message populator configuration specifies which populators are executed and in which order. A message populator executed later in the execution chain may then override values set by previously executed populators. The configuration applies to all products.

Template of message populators configuration:

production-line:
  message-populators:
    - # Name of message populator to execute (mandatory)
      name: <populator name>
      # Optional activation conditions. The specified populator is executed only when all of
      # provided activation conditions are satisfied. Omit to execute unconditionally.
      activate-on:
        <activation property>:
          <activation operator>: <activation operand>
          ...
        ...
      # Map of populator-specific configuration options. Some of them may be mandatory.
      # May be omitted, when no configuration options are given.
      config:
        <config property name>: <config property value>
        ...
    - ...

Message populator name is generally by convention the same as the class simple name. For example, name of fi.fmi.avi.archiver.message.processor.populator.MessageDataPopulator class is MessageDataPopulator in the configuration.

A base configuration is provided in the application.yml file as an example.

Bundled message populators

This application comes with handful of bundled message populators. Some of them, like MessageDataPopulator, play an essential role in the archival process. The provided application.yml has an example configuration of these. Others, like FixedProcessingResultPopulator or StationIcaoCodeReplacer, are provided for customized message handling, along with the possibility for conditional activation. One message populator, StationIdPopulator, cannot be configured, but is implicitly active.

Available populators are listed below in alphabetical order by name. These are declared for use in the MessagePopulatorFactoryConfig class.

BulletinHeadingDataPopulator

Populate properties parsed from input bulletin heading.

name: BulletinHeadingDataPopulator
config:
- bulletin-heading-sources (optional) - List of bulletin heading sources in preferred order.
  
  Possible values are specified in BulletinHeadingSource enum.
  
  Example:
```
bulletin-heading-sources:
  - GTS_BULLETIN_HEADING
  - COLLECT_IDENTIFIER
```

FileMetadataPopulator

Populate properties available in input file metadata.

name: FileMetadataPopulator
config: ~

FileNameDataPopulator

Populate properties parsed from file name.

name: FileNameDataPopulator
config: ~

FixedDurationValidityPeriodPopulator

Set validity period to a fixed duration period starting from message time.

name: FixedDurationValidityPeriodPopulator
config:
- validity-end-offset (mandatory) - Validity period length from message time as an ISO 8601 duration (java.time.Duration).
  
  Example:
```
validity-end-offset: PT24H
```

FixedProcessingResultPopulator

Set processing result to specified value.

name: FixedProcessingResultPopulator
config:
- result (mandatory) - Processing result code name.
  
  Possible values are specified in ProcessingResult enum.
  
  Example:
```
result: FORBIDDEN_MESSAGE_STATION_ICAO_CODE
```

FixedRoutePopulator

Set route to specified value.

name: FixedRoutePopulator
config:
- route (mandatory) - Route name.
  
  Possible values are specified in the production-line.route-ids identifier mapping property.
  
  Example:
```
route: DEFAULT
```

FixedTypePopulator

Set message type to specified value.

name: FixedTypePopulator
config:
- type (mandatory) - Message type name.
  
  Possible values are specified in the production-line.type-ids identifier mapping property.
  
  Example:
```
type: METAR
```

MessageContentTrimmer

Trim whitespaces around message content.

name: MessageContentTrimmer
config: ~

MessageDataPopulator

Populate properties parsed from message content.

name: MessageDataPopulator
config:
- message-type-location-indicator-types (optional) - Message type-specific list of location indicator types in order of preference for reading the station ICAO code.
  
  Available message types are specified in the map property production-line.type-ids. Available location indicator types are specified in GenericAviationWeatherMessage.LocationIndicatorType enum.
  
  Example:
```
message-type-location-indicator-types:
  - AIRMET:
      - ISSUING_AIR_TRAFFIC_SERVICES_REGION
  - METAR:
      - AERODROME
  - SIGMET:
      - ISSUING_AIR_TRAFFIC_SERVICES_REGION
  - SPECI:
      - AERODROME
  - SPACE_WEATHER_ADVISORY: []
  - TAF:
      - AERODROME
  - TROPICAL_CYCLONE_ADVISORY: []
  - VOLCANIC_ASH_ADVISORY: []
```
- default-location-indicator-types (optional) - Default list of location indicator types in order of preference for reading the station ICAO code.
  
  Only used when the message type-specific list is not configured. Available location indicator types are specified in GenericAviationWeatherMessage.LocationIndicatorType enum.
  
  Example:
```
default-location-indicator-types:
  - AERODROME
  - ISSUING_AIR_TRAFFIC_SERVICES_REGION
```

MessageDiscarder

Discard message.

name: MessageDiscarder
config: ~

MessageFutureTimeValidator

Deprecated and will be removed in a future release. Instead, use FixedProcessingResultPopulator with a negative message-age condition operand value:

- name: FixedProcessingResultPopulator
  activate-on:
    message-age:
      is-less-than: -PT12H
  config:
    result: MESSAGE_TIME_IN_FUTURE

Reject message if message time is too far in the future.

name: MessageFutureTimeValidator
config:
- accept-in-future (mandatory) - Maximum duration as an ISO 8601 duration (java.time.Duration) from current time to accepted message time.
  
  Example:
```
accept-in-future: PT12H
```

MessageMaximumAgeValidator

Deprecated and will be removed in a future release. Instead, use FixedProcessingResultPopulator with message-age condition:

- name: FixedProcessingResultPopulator
  activate-on:
    message-age:
      is-greater-than: PT36H
  config:
    result: MESSAGE_TOO_OLD

Reject message if message time is too far in the past.

name: MessageMaximumAgeValidator
config:
- confname (mandatory) - Maximum duration as an ISO 8601 duration (java.time.Duration) until current time to accepted message time.
  
  Example:
```
maximum-age: PT36H
```

ProductMessageTypesValidator

Reject message if message type is not one of the valid types configured for the product.

name: ProductMessageTypesValidator

config:

confname (mandatory) - Product-specific list of valid message types.

Available products are specified under list property production-line.products. Available message types are specified in map property production-line.type-ids.

Example:

product-message-types:
  example1:
    - METAR
    - SPECI
  example2:
    - TAF
    - SIGMET
    - AIRMET
    - TROPICAL_CYCLONE_ADVISORY
    - VOLCANIC_ASH_ADVISORY
    - SPACE_WEATHER_ADVISORY

StationIcaoCodeReplacer

Replace all occurrences of regular expression pattern in the station ICAO code with the provided replacement.

name: StationIcaoCodeReplacer
config:
- pattern (mandatory) - Pattern to find in station ICAO code.
  
  Pattern syntax is specified in java.util.regex.Pattern class.
  
  Example:
```
pattern: "^XY([A-Z]{2})$"
```
- replacement (mandatory) - Replacement string.
  
  Replacement syntax is specified in java.util.regex.Matcher.replaceAll(java.lang.String) method.
  
  Example:
```
replacement: "XX$1"
```

StationIdPopulator

Set the numeric station id matching station ICAO code, and reject the message if such ICAO code cannot be found in the database stations table.

StationIdPopulator is implicitly added in the end of the message populator execution chain, and it cannot be omitted nor configured in the middle of the execution chain.

name: ~
config: ~

Post-actions

Post-actions are components that execute a single action after the message has been archived in the database. Post-actions are not applied on messages that were discarded or failed in an earlier message processing phase (e.g. message population phase). In other words, only messages that have been stored successfully in the database either as archived or rejected will be processed by post-actions. The input file is marked as finished only after all post-actions have finished (with success or failure) on all messages of the file. Post-actions cannot change the final processing status (archived, rejected, failed) of an input file.

The following characteristics are unspecified, and current implementation may change any time without a prior notice.

processing order of messages
execution order of post-actions
sequential or parallel execution

Post-action configuration specifies which actions are executed. The configuration applies to all products. The execution of each post-action is isolated, and a failing post-action will not affect other post-actions.

Bundled post-actions

This application comes with some bundled post-actions. In addition to post-action configuration, you may want to set up conditional activation to, for example, restrict execution to messages that have been archived successfully, skipping rejected messages.

Available populators are listed below in alphabetical order by name. These are declared for use in the PostActionFactoryConfig class.

ResultLogger

A simple example post-action, that outputs a static info-level log message suffixed by message context information.

name: ResultLogger
config:
- message (mandatory) - The log message.
  
  Example:
```
message: Message archiving process finished.
```

SwimRabbitMQPublisher

This post-action publishes messages to a RabbitMQ broker, complying with the MET-SWIM (Meteorological System Wide Information Management) guidelines. Currently, this post-action implements the MET-SWIM CP1 requirements.

name: SwimRabbitMQPublisher
config:
- id (mandatory) - An unique identifier string of the instance to distinguish it from other instances.
  
  Example:
```
id: swim-example
```
- publisher-queue-capacity (mandatory) - Number of messages accepted in the processing queue.
  
  In case of broker disruptions, the incoming messages are held in a queue, until they are sent to the broker. When the queue is full, all subsequent incoming messages will be silently dropped. The queue is not persisted.
  
  Example:
```
publisher-queue-capacity: 50000
```
- publish-timeout (optional) - Timeout for a single post-action run on a single message as an ISO 8601 duration (java.time.Duration).
  
  When this timeout is reached, the publishing action is considered as failed and eligible for retry.
  
  Default value: PT30S
  
  Example:
```
publish-timeout: PT30S
```
- connection (mandatory) - Connection configuration section.
  
  The connection is established lazily upon publishing the first message to the broker. In case the connection has been closed since publishing the last message, it will be automatically re-established.
  
  For connection properties, a recommended convention is to use indirect values. Recommended format for properties is post-action.<post-action-name>.<post-action-id>.<property-name>.
  
  Example:
```
post-action:
  SwimRabbitMQPublisher:
    swim-example:
      uri: amqp://example.rabbitmq.host:5672/%2f
      username: exampleuser
      password: examplepassword
```
  The examples under the connection configuration properties documentation below refer to the properties of the example above.
  - connection.uri (mandatory) - RabbitMQ service uri.
    
    Example:
```
connection:
  uri: ${post-action.SwimRabbitMQPublisher.swim-example.uri}
```
  - connection.username (mandatory) - Connection username.
    
    Example:
```
connection:
  username: ${post-action.SwimRabbitMQPublisher.swim-example.username}
```
  - connection.password (mandatory) - Connection password.
    
    Example:
```
connection:
  password: ${post-action.SwimRabbitMQPublisher.swim-example.password}
```
- topology (mandatory) - Topology configuration section.
  
  Please refer to the RabbitMQ documentation on details of the configuration properties.
  - create (mandatory) - Specifies whether the application should create any of topology elements when (re-)establishing the connection.
    
    The account used to connect to the broker must have privileges to create such topology elements. When the value is set to NONE, no topology elements will be created. In this case the configured exchange must exist for publishing to succeed. The broker must also route the message to at least one queue, otherwise the broker will respond with RELEASED status, which is considered a publication failure leading to retries. An easy way to always have a queue to route to is to configure a debug queue and set this value to ALL or QUEUE_AND_BINDING.
    
    Possible values: ALL, EXCHANGE, QUEUE_AND_BINDING, BINDING, NONE
    
    Example:
```
topology:
  create: ALL
```
  - routing-key (mandatory) - Routing key to be used on publication.
    
    Example:
```
topology:
  routing-key: aviation-message-archiver-example-routing
```
  - exchange (mandatory) - Exchange configuration section.
    - name (mandatory) - Exchange name.
      
      Example:
      topology: exchange: name: x.aviation-message-archiver
    - type (optional) - Exchange type.
      
      Possible values: DIRECT (default), FANOUT, TOPIC, HEADERS
      
      Example:
      topology: exchange: type: TOPIC
    - auto-delete (optional) - Defines whether the exchange is deleted when last queue is unbound from it.
      
      Possible values: true, false (default)
      
      Example:
      topology: exchange: auto-delete: false
    - arguments (optional) - Exchange arguments.
      
      Example:
      topology: exchange: arguments: x-example-arg-1: x-example-arg-1-value x-example-arg-2: x-example-arg-2-value
  - queue (mandatory) - Queue configuration section.
    - name (mandatory) - Queue name.
      
      Example:
      topology: queue: name: q.aviation-message-archiver
    - type (optional) - Queue type.
      
      Possible values: QUORUM, CLASSIC (default), STREAM
      
      Example:
      topology: queue: type: QUORUM
    - auto-delete (optional) - Defines whether the queue get automatically deleted when its last consumer unsubscribes.
      
      Possible values: true, false (default)
      
      Example:
      topology: queue: auto-delete: false
    - arguments (optional) - Queue arguments.
      
      Example:
      topology: queue: arguments: q-example-arg-1: q-example-arg-1-value q-example-arg-2: q-example-arg-2-value
- retry (mandatory) - Retry configuration section.
  - initial-interval (optional) - Initial delay before first retry is attempted as an ISO 8601 duration (java.time.Duration).
    
    Default value: PT0.5S
    
    Example:
```
retry:
  initial-interval: PT0.5S
```
  - multiplier (optional) - The back off time multiplier as a decimal number.
    
    Default value: 2.0
    
    Example:
```
retry:
  multiplier: 2.0
```
  - max-interval (optional) - The maximum back off delay as an ISO 8601 duration (java.time.Duration).
    
    Default value: PT1M
    
    Example:
```
retry:
  max-interval: PT1M
```
  - timeout (mandatory) - The maximum time to retry since first attempt as an ISO 8601 duration (java.time.Duration).
    
    Use zero duration (PT0S) for infinite retry.
    
    Example:
```
retry:
  initial-interval: PT3H
```
- message (optional) - Message configuration section.
  
  Please refer to the MET-SWIM guidance on details of the configuration properties.
  - priorities (optional) - List of message priority descriptors.
    
    The priority of the message is determined by the first priority descriptor matching the message. If none of the descriptors match, the message will have the default priority 0. This is also the situation in case the priorities list is empty. If the priorities section is omitted, the default priority descriptors are used (see the example below).
    
    Example - the default priority descriptors:
```
message:
  priorities:
    - type: SIGMET
      priority: 7
    - type: SPECI
      priority: 6
    - type: TAF
      status: AMENDMENT
      priority: 6
    - type: TAF
      priority: 5
    - type: METAR
      priority: 4
    - priority: 0
```
    A priority descriptor consists of condition properties and the priority value that is given to messages matching all the conditions.
    - type (optional) - Message type condition.
      
      When omitted, messages of any type matches this condition.
      
      Possible values are specified in the production-line.type-ids identifier mapping property.
    - status (optional) - Report status condition.
      
      When omitted, messages of any report status matches this condition.
      
      Possible values: NORMAL, AMENDMENT, CORRECTION
    - priority (mandatory) - The priority a matching message will have.
      
      Possible values: an integer between 0-9 (inclusive)
  - encoding (optional) - Message encoding.
    
    Possible values: IDENTITY (default), GZIP
    
    Example:
```
message:
  encoding: GZIP
```
  - expiry-time (optional) - Message expiry time.
    
    This property value is an ISO 8601 duration (java.time.Duration) from the AMQP message creation time. It is used to set the absolute-expiry-time AMQP property.
    
    When omitted, the absolute-expiry-time property is omitted from the message.
    
    Example:
```
message:
  expiry-time: PT12H
```

Conditional message processor activation

Message processor configuration may include an activation condition, in which case the message processor is executed only when the provided condition is satisfied. An activation condition consists of one or more activation expressions, that consist of activation property and one or more activation operator and activation operand pairs. When multiple activation expressions and/or operator-operand pairs are specified, they are combined with AND operator, thus all of them must be satisfied to activate the message processor.

Template for activation condition configuration:

activate-on:
  <activation property>:
    <activation operator>: <activation operand>
    ...
  ...

Conditional execution is provided by ConditionalMessagePopulator and ConditionalPostAction classes.

Activation property

Activation property is the name of the aviation message data property that is applied as the first operand of activation operator.

The following activation properties are declared in MessageProcessorConditionPropertyReaderConfig:

archival-status: status of message archival to database. See ArchivalStatus for available states.
format: populated message format name. See format-ids in Identifier mappings.
message-age: duration between populated message time and current processing time in ISO 8601 format (java.time.Duration). The value can be negative, which allows referring to the future.
processing-result: populated result code of message processing. See ProcessingResult for available codes.
product-id: identifier of the product this file belongs to.
route: populated route name. See route-ids in Identifier mappings.
station: populated station ICAO code.
type: populated message type name. See type-ids in Identifier mappings.
<heading>-data-designator: input bulletin heading data designators (TTAAii).
<heading>-originator: input bulletin heading location indicator / originator (CCCC).

The <heading> specifies whether to read GTS or COLLECT heading. It may be one of:

gts: GTS heading
collect: collect identifier
gts-or-collect: GTS heading, or if not present, collect identifier
collect-or-gts: collect identifier, or if not present, GTS heading

E.g. gts-or-collect-data-designator.

Activation operator and operand

In an activation expression, the activation operator is applied to the activation property value (as first operand for the operator) and specified activation operand (as second operand for the operator). Activation operand type and possible values depend on the activation operator and activation property.

Activation operator may be one of:

presence: test for presence of activation property.

Activation operand is one of:
- PRESENT: activation property value must be present. This is the default when presence is omitted.
  
  For example, omitting presence is equivalent to:
```
activate-on:
  type:
    presence: PRESENT
```
- EMPTY: activation property value must not be present
- OPTIONAL: activation property value may or may not be present, aka presence condition is always satisfied
is: test whether activation property is equal to activation operand.

This is mutually exclusive with is-any-of.

For example, following activation condition is satisfied when message format is IWXXM.
```
activate-on:
  format:
    is: IWXXM
```
is-any-of: test whether activation property is equal to any of activation operand list.

This is mutually exclusive with is.

For example, following activation condition is satisfied when message type is either METAR or SPECI.
```
activate-on:
  type:
    is-any-of:
      - METAR
      - SPECI
```
is-not: test whether activation property is not equal to activation operand.

This is mutually exclusive with is-none-of.

For example, following activation condition is satisfied when message format is not IWXXM.
```
activate-on:
  format:
    is-not: IWXXM
```
is-none-of: test whether activation property is not equal to any of activation operand list.

This is mutually exclusive with is-not.

For example, following activation condition is satisfied when message type is neither METAR nor SPECI.
```
activate-on:
  type:
    is-none-of:
      - METAR
      - SPECI
```
matches: test whether activation property matches regular expression provided as activation operand.

For example, following activation condition is satisfied when originator parsed from GTS bulletin heading, or collect identifier when GTS heading is not present, starts with 'XX':
```
activate-on:
  gts-or-collect-originator:
    matches: "XX[A-Z]{2}"
```
does-not-match: test whether activation property does not match regular expression provided as activation operand.

For example, following activation condition is satisfied when originator parsed from GTS bulletin heading, or collect identifier when GTS heading is not present, does not start with 'XX':
```
activate-on:
  gts-or-collect-originator:
    does-not-match: "XX[A-Z]{2}"
```
is-greater-than: test whether a comparable activation property is greater than provided activation operand value.

For example, following activation condition is satisfied when message age is greater than 24 hours:
```
activate-on:
  message-age:
    is-greater-than: PT24H
```
is-greater-or-equal-to: test whether a comparable activation property is greater or equal to provided activation operand value.

For example, following activation condition is satisfied when message age is greater or equal to 24 hours:
```
activate-on:
  message-age:
    is-greater-or-equal-to: PT24H
```
is-less-than: test whether a comparable activation property is less than provided activation operand value.

For example, following activation condition is satisfied when message age is less than 24 hours:
```
activate-on:
  message-age:
    is-less-than: PT24H
```
is-less-or-equal-to: test whether a comparable activation property is less or equal to provided activation operand value.

For example, following activation condition is satisfied when message age is less or equal to 24 hours:
```
activate-on:
  message-age:
    is-less-or-equal-to: PT24H
```

Activation operators are provided by GeneralPropertyPredicate class.

Identifier mappings

Internally this application uses static values to represent message type and format. These internal values differ from the identifier of equivalent database entity, and are not bound to the name in the database. Therefore, internal values need to be mapped to database values in the application configuration. In addition to message type and format, message route is a similar property, but it has no internal semantics. However, it is mapped in the configuration for consistency with other similar properties.

The following mappings must exist under production-line application configuration property:

route-ids: Map route name, preferably same as database column avidb_message_routes.name, to database column avidb_message_routes.route_id.
format-ids: Map GenericAviationWeatherMessage.Format.name() to database column avidb_message_format.format_id.
type-ids: Map MessageType.name() to database column avidb_message_types.type_id.

See the provided application.yml for an example.

Spring Boot configuration properties

Many of the properties in application.yml configuration file control the behavior of Spring Boot features. Look at the Spring Boot Reference Documentation for more information on these. Some of related sections are:

License

MIT License. See LICENSE.

Name		Name	Last commit message	Last commit date
Latest commit History 480 Commits
.github/workflows		.github/workflows
src		src
.gitattributes		.gitattributes
.gitignore		.gitignore
CHANGELOG.md		CHANGELOG.md
LICENSE		LICENSE
README.md		README.md
pom.xml		pom.xml

Folders and files

Latest commit

History

Repository files navigation

Aviation Message Archiver

Overview

Contents

Feature overview

Supported message types and formats

Getting started

Logging

Processing identifier and phase

Processing context

Processing statistics

Structured logging

Application configuration

Products

Message processors

Message populators

Bundled message populators

BulletinHeadingDataPopulator

FileMetadataPopulator

FileNameDataPopulator

FixedDurationValidityPeriodPopulator

FixedProcessingResultPopulator

FixedRoutePopulator

FixedTypePopulator

MessageContentTrimmer

MessageDataPopulator

MessageDiscarder

MessageFutureTimeValidator

MessageMaximumAgeValidator

ProductMessageTypesValidator

StationIcaoCodeReplacer

StationIdPopulator

Post-actions

Bundled post-actions

ResultLogger

SwimRabbitMQPublisher

Conditional message processor activation

Activation property

Activation operator and operand

Identifier mappings

Spring Boot configuration properties

License

About

Resources

License

Uh oh!

Stars

Watchers

Forks

Releases 7

Packages 0

Uh oh!

Uh oh!

Contributors

Uh oh!

Languages

Packages