Setting Up a Standard DI Environment

For help upgrading an existing DI environment, see Upgrading a DI Environment.

AEMO recommends setting up your first Data Interchange (DI) environment in pre-production. Once you have a successful pre-production installation (used for testing), you can repeat the installation in production to receive live market data. Be careful to always keep the pre-production and production data entirely separate.

A standard DI environment is a single data feed, from one Participant ID folders on the participant file server, to a single participant database. For example, you can have your pre-production environment connected to the participant file server pre-production environment and your production environment connected to the participant file server production environment.

AEMO’s production and pre-production environments are independently operated, so the set of components shown in Figure 8 are duplicated for production and pre-production. To set up a standard Data Interchange environment, follow the steps in Table 5.

Do not point your pre-production environment to the AEMO production environment or vice versa.
All participants are encouraged to use the pre-production environment to test implementation and to train their participant users, before attempting any changes or operations on the production systems.

Figure 8 Standard Data Interchange implementation

Table 5 Steps to set up a standard Data Interchange environment

Step

Task

Component

References

1

Download and extract a DI bundle for your specific database (Oracle or SQL Server)

Data Interchange Bundle

Software Releases.

Release Documents

2

Create an RDBMS instance, conforming to the MMS Data Model:

  • Install the RDBMS.
  • Install the latest MMS Data Model structure (target tables). Do not make any changes to the MMS Data Model structure.

Electricity Data Model (MMSDM)

Gas Data Model (GASDM)

Data Model GUI Installer Guide

Data Model Report

Table to File Report

3

Set up Data Subscription Services:

  • Subscribe to files. AEMO recommends subscribing to the full data model set.

Data Subscription web application

Data Subscription

4

Install and configure the Participant Data Replication Loader (pdrLoader)

pdrLoader

Installation

5

Run the pdrLoader application:

  • In Windows, double-click pdrLoader.bat file in the Lib folder or start the Windows service.
  • In unix-like environments, execute the pdrLoader.sh file in the Lib folder.

pdrLoader

Participant Data Replication Loader (pdrLoader)

6

Copy the Data Model configuration for pdrLoader

pdrLoaderConfiguration

pdrLoader Configuration Note

7

Install and configure the pdrMonitor:

  • Extract and install the pdrMonitor software on a local PC running Microsoft Windows to monitor and control the operation of the pdrLoader.
  • Set the tables you want to populate to Active.

pdrMonitor

Participant Data Replication Monitor (pdrMonitor)

8

Activate the files to load into your database

pdrMonitor

Participant Data Replication Monitor (pdrMonitor)

9

Populate your database with data:

  • If you don't have access to AEMO's systems yet and you want to test your implementation, you can find CSV files on AEMO's website.
  • Load the baseline data to the your pdrLoader folder. (baseline data excludes historical data).
  • Load the optional historical data.

You can request historical data on CD from AEMO’s Support Hub

pdrLoader input folder

/MarketData/Baseline

10

Install the Participant Data Replication Batcher (pdrBatcher)

pdrBatcher

Participant Data Replication Batcher (pdrBatcher)

11

Run the pdrBatcher application:

  • In Windows, double-click pdrBatcher.bat file in the Lib folder or start the Windows service.
  • In unix-like environments, execute the pdrBatcher.sh file in the Lib folder.

pdrBatcher

Participant Data Replication Batcher (pdrBatcher)