Introduction

Audience

AEMO Australian Energy Market Operator provides this information as a service targeting business analysts and IT staff in Registered Participant See Relevant Rules or Procedures companies.

Objective

The EMMS Energy Market Management System (formerly MMS); software, hardware, network and related processes. - Technical Specification - Data Model v5.2 - May 2023 (Release) describes the changes to the Data Model The definition of the interface to participants of data published by AEMO for gas or electricity. A database conforming to the Data Model can contain a local copy of all current participant-specific data recorded in the main database. The Data Model includes database tables, indexes, and primary keys. v5.2 according to projects planned by AEMO from a participant perspective.

Status

Version	Status
	The Data Model design is still uncertified but nearing completion.
2.01	The Data Model design is still uncertified but nearing completion. Changes in this version of the technical specification are for participant information only. There are no changes to the Data Model design. AEMO is seeking feedback on considerations for future releases, see Increasing data volumes
2.00	The Data Model design is in the testing phase and almost complete. Some changes may occur according to testing and participant feedback
1.00	This technical specification is for participant review only. The design may change as the technical requirements are streamlined. Please provide feedback in AEMO’s website contact us form

Release dates

Scheduled for implementation in:

• Pre-production AEMO’s test system available to participants:     20 April 2023
• Production AEMO’s live system:        23 May 2023

Version numbers

AEMO releases new versions of this document as the technical requirements are streamlined.

Incremental version numbers such as 1.01, 2.01 and so on mean there is a minor change to the technical specification.

Major version numbers such as 1.00, 2.00 means there are substantial changes to the technical specification. Participants must carefully review these changes, detailed Changes in this version.

Changes in this version

This version includes the following changes:

MMS Data Model v5.2 release update

In error, AEMO introduced several unrequired columns in the upgrade and apologise to participants for the inconvenience caused to their pre-production Data Model databases during the Data Model v5.2 upgrade.

Table	Column	Reason
P5MIN_UNITSOLUTION	RAISE1SECACTUALAVAILABILITY LOWER1SECACTUALAVAILABILITY	Unrequired
P5MIN_REGIONSOLUTION	RAISE1SECAPCFLAG LOWER1SECAPCFLAG PRE_AP_RAISE1_PRICE PRE_AP_LOWER1_PRICE RAISE1SECACTUALAVAILABILITY LOWER1SECACTUALAVAILABILITY	Unrequired
PREDISPATCHPRICE	RAISE1SECAPCFLAG LOWER1SECAPCFLAG RAISE1SECROP LOWER1SECROP PRE_AP_RAISE1_PRICE PRE_AP_LOWER1_PRICE	Unrequired

If you have already applied the Data Model v5.2 upgrade, run the relevant SQL Server or Oracle delete script published to the Releases > Data Model folder manually to remove the unrequired columns from the impacted tables. You do not need to run the updated Data Model script.

Depending on how much data is retained online in the impacted tables, dropping columns in Oracle takes some time because it requires a full table scan.

If you haven’t yet applied the Data Model v5.2 upgrade, updated Data Model scripts are available in the Releases > Data Model folder for you to apply when you are ready. The manual delete script is only required if you already applied the scripts released 20 April 2023.

Modified table: DUDETAILSUMMARY

• The ADG_ID will populate a NULL value. AEMO will notify participants when we decide to populate this field.
• Removed the Important table changes message. This change is handled in a future release. AEMO will notify participants when this happens.

DUDetailSummary can be misleading and is a frequent point of confusion for participants. To overcome this, rather than reporting incremental changes to DUDETAILSUMMARY, we are supplying the full history of DUDETAILSUMMARY each day and modifying the transaction_type from INSERT-UPDATE to DELETE-INSERT. DUDetailSummary includes an End_Date, which is actually the Start_Date of the next record as defined when the data set was created. This gives the From Date-To Date model. DUDetailSummary was created to assist participants comprehension of the complex interrelations between the component tables. Such a presentation is subject to changes in Registration, particularly retrospective changes occurring as registration information is clarified after the initial registration. The consequence of retrospective changes is further records can be created and replicated to participants. This creates ambiguity as participants can see two records indicating an overlap for the same DUIDs. Note that the presentation is an external construct and the internal AEMO tables are always unambiguous. The operation to change the transaction type clears and reloads the DUDETAILSUMMARY table in a single database transaction. In the event the operation fails, the current data remains without impact to participants. Participants always have the opportunity to repopulate this table from the BASELINE reports.

Sparse model details

Previously, AEMO introduced the opt-in Sparse Model concept for some Data Model tables and requested participant feedback. This version adds more clarity for the Sparse model. See Increasing data volumes section.

BodyTableText_1_1

In v2.04

Current version

Considerations AEMO are considering the following options for participant provision of sparse offers and bids and can discuss this in the next MSUG. Report overlap For the NEXT_DAY_OFFER_ENERGY and NEXT_DAY_OFFER_FCAS reports, provide an overlap where the LATEST report version is sparse, and the LEGACY report version is the full version. Hard change Implement a hard change over to the sparse model when FFR is delivered. This change requires participants to alter the way their Energy Offer data is interpreted in their participant systems. For this option, we advise participants to modify their analytical and offer systems prior to the June 2024 Data Model Release, ensuring downstream systems remain compliant with the sparse representation. Participants can implement this logic in their systems ahead of time as the interpretation of a fully qualified offer (as published now) is completely compatible with the sparse offer representation. Participant sparse bidding Implement the sparse model for participants bidding into the Market. Participant feedback required We are asking for participant feedback for the following: Implement the report overlap in the May 2023 Release. Implement the report overlap in the June 2024 Release. Implement a hard change in the June 2024 Release when FFR is delivered. Participants use the sparse model for bidding into the Market.

See Sparse model details section

Rule and procedure changes

Details	Effective
Standalone Power Systems	30 May 2023
Integrating energy storage systems into the NEM	9 August 2023
Fast frequency response market ancillary service	9 October 2023
Enhancing information on generator availability in MT PASA | AEMC > ERC0338 National Electricity Amendment (Enhancing information on generator availability in MT PASA) Rule 2022 No. 7 (aemc.gov.au)	9 October

Related technical specifications

Title	Description
EMMS Technical Specification August 2023	Details of the FFR project changes
EMMS Technical Specification October 2023	Details of the MT PASA project changes
EMMS – Release Schedule – Data Interchange – April 2023	Details of changes to the Data Interchange products: Data Subscription and pdrLoader

Related documents

These guides and resources are updated according to this technical specification and published for the pre-production Release Date.

Title	Description	Published
Data Interchange Install Bundles	All the components for a new installation of Data Interchange	23 May 2023
Data Interchange Online Help	Online help provides the details about the updated screens and other details for the Markets Portal Help using Data Interchange and Data Model components	20 April 2023
Data Interchange Upgrade Software	Software components for upgrading an existing Data Interchange environment	20 April 2023 for pre-production environments
Data Model Reports	Explains the packages, tables, and reports in the Electricity and Gas Data Models	23 May 2023
Market Data NEMweb	Data Model, Current, and archived reports	23 May 2023
Release Documents	Release Notes and Technical Specifications	20 April 2023
Software Releases	Data Interchange component software	23 May 2023

Access to the Data Interchange Online Help does not require a MarketNet connection.
Access to the Markets Portal Help requires a MarketNet connection. Please speak with your company’s Participant Administrator if you require access.

Approval to change

AEMO requests approval to proceed from all participant change controllers by close of business Wednesday, 8 March 2023.

Market systems user group meetings

The Market Systems User Group (MSUG Market Systems User Group) is an industry user group established to discuss NEM wholesale and retail IT systems releases. Its purpose is to facilitate the continuing improvement of AEMOs IT systems by seeking feedback and collaboration from participants.

MSUG meetings are open to all interested parties, with invitations sent to all included on the distribution list. If you have a technical question for a project and want to attend the MSUG ask your company’s support team to include your email address in their AEMO Help Desk Bulletin (CRM) distribution list.