Acknowledgement and validation

About

MSATS Market Settlement and Transfer Solutions. The procedures published by AEMO under clause 7.2.8 of the National Electricity Rules, which include those governing the recording of financial responsibility for energy flows at a connection point, the transfer of that responsibility between market participants, and the recording of energy flows at a connection point. validates and responds to ALL Transactions with an acknowledgment of receipt (ACK An acknowledgement file from AEMO with a filename extension of .ACK. The file contains messages to acknowledge the success of another file.). Acknowledgement Transactions depend on the interface used: API Application Programming Interface; a set of clearly defined methods of communication between various software components., Batch A right type assigned to a participant user by their participant administrator to access a Batch (file) application, or web. This section describes validation, ACK files and how to acknowledge one for each interface.

MSATS to participant

MSATS does the following when submitting Transactions to participants:

  1. Sends a zip file to your Participant Outbox.

If you do not have a back-end system to process incoming files, AEMO Australian Energy Market Operator recommends you save the file and look at the contents before acknowledging.

To save the zip file, right-mouse click, select Save Target As, and save it to your local drive.

  1. Once you acknowledge the file, MSATS moves the zip file into your Participant Archive.

Access to the Participant Archive depends on the rights assigned to you by your company’s Participant Administrator Creates and maintains access to AEMO systems for their Participant ID users..

  1. MSATS deletes the zip file from your Participant Outbox.

Validation

Transactions Initiated by participants undergo several validations prior to moving to the Requested A status point of a Change Request. status.

Level 1

MSATS performs the following 1st level validations:

  1. The user ID nominated in the <SecurityContext> element of the file is permitted to perform each type of batch transaction submitted.

  2. The XML is well formed (meaning that it meets the rules defined for writing XML).

  3. The file is valid according to the rules specified in the aseXML schema.

  4. The schema and transaction versions are supported by MSATS.

  5. The number of transactions in the file do not exceed the transaction limits for the transaction group imposed by MSATS.

Level 2

For each accepted Transaction See Relevant Rules or Procedures, MSATS performs 2nd level validations and processes the request. These second level validations include business-level validations such as checking:

  1. The initiator of the Transaction can or is acting in a Role that is entitled to submit a transaction for the nominated change reason code.

  2. All required fields for this change reason code, as required by the Field Validation Rules, are provided.

  3. Only fields valid for this change reason code, as required by the Field Validation Rules, are provided.

  4. The Change Reason Code is valid for use in the jurisdiction in which the NMI is located.

Subsequent change request validation

After the Change Request A transaction submitted by participants in MSATS whenever they want to create or update data held in MSATS. is submitted, any subsequent Change Requests submitted by the Initiating participant is validated as follows:

  1. The NMI on the subsequent Change Request is checked against the NMI on the initial Change Request.

  2. The Participant ID on the subsequent Change Request is checked against the Participant ID on the initial Change Request.

Validation checks

Validation A process to test the veracity and integrity of metering data. checks in order of priority are:

  1. Codes and dates comply with the codes and rules look-up tables in the MSATS Web Portal > Administration > Codes and Rules Maintenance.
    The following data is validated:

  1. Change Request ID
  1. Jurisdiction
  2. Role ID
  3. NMI Status Code
  4. Read Type Code
  5. Change Request Code
  6. TNI Code
  7. DLF Code
  8. Metering Installation Type Code
  9. Parent Name
  10. Child Name
  11. Proposed Change Date

  1. Change Reason Codes and Field Validation Rules comply:

  1. Change Reason Codes, see Change reason codes.
  1. Field Validation Rules: RI, OI, RQ, RD, RA.

  1. NMI characters against the NMI Checksum.

  2. The Initiating participant is an active participant and can act in the Role to initiate the Transaction.
    The following data is validated:

  1. Participant ID
  1. Participant Status
  2. Participant Roles

  1. The Proposed Change Date and the Actual Change Date are within the range allowed by the Change Reason Code.

  2. The Proposed Change Date, the Actual Change Date, and the Actual End Date against the Timeframe Rules.

  3. Information regarding Embedded Networks:

  1. The codes comply with the MSATS Web Portal > Administration > Codes Maintenance > Embedded Network Identifier Codes.
  1. The Parent and Child Connection Point names are not identical for the same NMI.
  2. The Child NMI is checked against the Parent NMI.
  3. There are no circular relationships.
  4. Prevent Local Retailer changes on a Child NMI.
  5. If a Parent NMI is not active, there are no active Child NMIs.

Validation explanations

Validations and explanations summarises the validations MSATS performs when checking a Change Request The way information is returned from an API. In a request, the client provides a resource URL with the proper authorization to an API server. The API returns a response with the information requested. in PVAL status. If any of these validations fail, the Change Request moves to REJ status.

Table 108 Validations and explanations

Validation

Check

Actual Change Date

If the Change Reason Code is one submitting an ACTUALCHANGEDATE (CR 1500), if the participant submitting is where the Data Request was sent. Use the initiating Request ID to identify the original Change Request

If the ActualChangeDate is within the Retrospective Period

Changes to Change Requests

If the participant submitting the updated Change Request is the same participant who submitted the original Change Request and the original Change Request status is not CAN, REJ, or COM

For Change Request lifecycle details, see page Page 1

Checksum

If the NMI Checksum is valid for the NMI supplied

Date

For a Prospective Change:

  • The Proposed Change Date is tomorrow or later

  • The Proposed Change Date is not after the Prospective Period

For a Retrospective Change:

  • The Proposed Change Date is today or earlier

  • The Proposed Change Date is not prior to the Retrospective Period

  • If an ActualEndDate is submitted, ensure it is either today or prior to today

Embedded Network

If the Embedded Network Code is valid for the period covered

The EmbedNetParent and EmbedNetChild fields in the CATS_NMI_DATA table are not the same

If a Child NMI is being created, a Parent NMI has been identified for the Embedded Network

The change does not produce an Embedded Network circular relationship

It is not a change of LR on a Child NMI (unless the Transaction is creating the Child NMI)

It does not create an orphan (i.e. if a parent is removed, there are no more parents or children)

Field Completeness

For the Change Reason Code:

  • All fields in the Field Validation Rules with a Data Source Code of RI are supplied

  • All fields have a Data Source Code of either RI or OI

Field-level

  • If it includes fields held in lookup tables and the value supplied for each field is a valid code on the lookup table

  • If it was valid for the whole period covered by the Change Request

Initiating Party

  • If the participant submitting the Change Request is currently active and was active from the period covered by the Change Request

  • For a Change Request submitted by a New Role, if the participant submitting the Change Request is entitled to act in that Role for the period covered by the Change Request and the participant submitting the Change Request is the participant nominated in that New Role

  • For a Change Request submitted by a Current Role, if the participant submitting the Change Request is acting in that Role for the entire period covered by the Change Request

Jurisdictional Rules

  • For the combination of the Jurisdiction and NMI Classification on the NMI Master Record or Change Request and for the Change Reason Code on the Change Request, if there is a Jurisdictional rule. In the MSATS Web Portal, see Administration > Rules Maintenance > Jurisdictional Parameters

  • If the NMI already exists (i.e. there is a NMI Master Record) and there is a NMI Classification Code, Jurisdiction Code, or both, the NMI Classification Code and Jurisdiction Code take precedence in determining whether Jurisdictional rules exist

MDM

Some of these validations also apply at the time it is about to move to COM status. The context for this validation is if the it completes, the configuration as of the Day after the Actual Change Date or, for the period covered by the Proposed Change Date, must not break these rules:

  • The NMI must have a valid NMI Classification Code, Jurisdiction Code, DLF Code, NMI Status Code, TNI Code, LR, FRMP, RP, ROLR, MDP, and MPB

  • It must not cause any gaps in the NMI’s history (on any of the master tables)

  • Every time a Datastream record is created or updated, it must have a Datastream Status Code, that is NOT NULL for the ADL, a valid value in DataStreamType and a valid Profile Name

  • Every time a Datastream is created or updated there must be an active NMI covering the period when the Datastream is active

  • Every time a Meter is created or changed it must always have a Metering Installation Type Code and a Meter Register Status Code

  • The Profile Name assigned to a Datastream must be valid in that NMI's Jurisdiction

  • If the NMI Classification Code is Small, the ADL must be <=2000

  • If the Datastream Type is C, the Profile Name cannot be NOPROF

New NMI

If the Change Reason Code is 2000, 2001, 2020, 2021, 2100, 2500, or 2501, there isn’t an existing record on the CATS_NMI_ DATA table for this NMI

RA

If, in the Field Validation Rules there are RA fields, there is a participant to send the Data Request for the missing data to on the NMI Master Record for the period covered

RQ

If, in the Field Validation Rules there are RQ fields and no data on the NMI Master Record for those fields, there is a participant to send the Data Request for the missing data to on the NMI Master Record for the period covered

Two-stage batch validation process

Whenever you submit a Transaction by Batch it goes through two levels of validation, MSATS conveys:

  1. First level validation in an .ACK file placed in your Participant Outbox.

If you submit the Batch file using the Web Portal you also see it on the interface.

If the Transaction passes the first level validation, it passes to second level validation.

  1. Second level validation in a Response Transaction.

The type of Response The information returned by an API after a request is made. Responses are usually in JSON or XML format. Transaction you receive depends on the submitted Transaction:

Type

Response

Change Request (CR)

Change Request Response (CRR)

Objection (OBJ)

Objection Response (OBJR)

NMI Discovery Request (NMID)

NMI Discovery Response (NMIR)

Report Request (RPTD)

Report Response

Meter Data Notification (MDN)

Meter Data Response

  1. The Response Transaction contains the result of your request, Request IDs, or the results of the validation. For example, Change Request, Objection, Meter Data Notifications.

Acknowledgement

ACK files

The ACK file includes:

  1. The results of the 1st level validation.

  2. The status: Accept or Reject.

  3. The Receipt IDs: one for the message and one for each Transaction.

Different Transactions have different content in their ACK files.

Change requests

For Change Requests, the numeric part of the Receipt ID corresponds to the MSATS assigned Request ID. In Transaction ACK file, the Receipt ID is CATS Customer Administration and Transfer Solution. A set of procedures, principles and obligations made under the National Electricity Rules as part of Market Settlement and Transfer Solutions (MSATS), and applicable to NMI (National Metering Identifier) small and large classifications.-CR8335806, so 8335806 is the Request ID.

You can use the Request ID to search for the Change Request and check the status of the Transaction.

Figure 147 Transaction ACK file

NMI discovery

For NMI See Relevant Rules or Procedures Discovery the receipt ID identifies the corresponding file in your Participant Outbox (see NMI discovery ACK Page 1 and Participant outbox filename).

Figure 148 NMI discovery ACK

Figure 149 Participant outbox filename

Objections

For Objections, the numeric value of the Receipt ID corresponds to the MSATS assigned Objection A type of transaction raised in relation to a Change Request. ID. InObjection Transaction ACK the Receipt ID is CATS-OBJ19735, so the Objection ID is 19735. You can use the Objection ID to search and check the status of the Objection.

Figure 150 Objection Transaction ACK

Reports

For reports, the numeric value of the Receipt ID corresponds to the MSATS assigned Report A data report that loads into a data model table. Identified by its type, subtype, and version. For example: BILLING,BILLINGASPAYMENTS,2 Request ID. In Report ACK the ID is CATS-215731, so the Report Request ID 215731.

Figure 151 Report ACK

Acknowledgement – API

Sending and receiving acknowledgements using an API differs according to the type of API you use. For details, see Guide to NEM Retail MSATS and B2B systems B2M Business-to-Market APIs > Participant Implementation.

Participants can nominate the outbound protocol for B2M messages by transaction group, so acknowledged messages can be your Participant Hub Queue and your B2M Outbox.

Acknowledgement – batch

When you submit a Transaction using the File Interface it is validated and an Acceptance or Rejection response returned with the Change Request Response (CRR) Transaction Type Code As described in the CATS Procedures, a code used in MSATS to identify a need to change CATS Standing Data..

To acknowledge a zip file in your Participant File Server The publishing point from AEMO systems to participant systems. Each participant is allocated an account and access to private and public areas. Participants are responsible for interfacing with the participant file server. Outbox, write an ACK file with the same name as the zip file. For help, see Creating and submitting batch transactions on How you use MSATS B2M.

Acknowledgement - web portal

To acknowledge a zip file in the web portal:

  1. Login to the web portal, click Data Load Import > Participant Outbox.

  2. Select the relevant zip files and click Acknowledge Selected.

  1. MSATS writes an .ACK file to your Participant Inbox and deletes the zip file.
  1. MSATS detects the zip file is deleted so deletes the .ACK file it created in your Participant Inbox.

  1. If within 30 seconds you see the following message, MSATS is unable to delete the zip file. Follow the steps below to delete the ACK and zip files.

ACK deletion -API

Follow the steps for Batch or web portal ACK deletion.

ACK deletion - batch

To delete an the ACK from the Participant File Server:

  1. Login to the Participant File Server and highlight the ACK file.

  2. On your keyboard, press Delete.

  3. Check the Inbox and Outbox are empty.

ACK deletion - web portal

To delete an ACK from the web portal:

  1. Login to the web portal and navigate to Data Load Import > Participant Inbox.

  2. Select the check box next to the ACK file and click Delete Selected.

Zip deletion – API

Follow the steps for Batch or web portal zip deletion.

Zip deletion – batch

To delete the zip from the Participant File Server:

  1. Login to the Participant File Server.

  2. Select the zip file and on your keyboard press delete.

Zip deletion - web portal

Sometimes, the MSATS Web Portal cannot complete the Hokey-Pokey Protocol The MSATS FTP Protocol on your behalf (producing the ACK file and deleting your zip file) so the following message displays:

MSATS allows you to continue using the web portal for other tasks but the zip file is sitting in your Participant Inbox and there is no ACK file in your Participant Outbox.

When MSATS completes the Hokey-Pokey Protocol there is an ACK file in your Participant Outbox, so you must delete your zip file. You can do this in the web portal or Participant File Server.

Make sure you save the relevant information from the .ACK file before you delete the zip file because MSATS is not monitoring the processing for you.

When MSATS detects you deleted your zip file it deletes the ACK file in your Participant Outbox, so the file handling cycle is complete.

To delete the zip file from the web portal:

  • In Data Load Import > Participant Inbox, select the check box next to the File Name and click the Delete Selected.

MSATS zip file deletion


Before MSATS deletes a zip file, it is copied to your Participant Archive on the Participant File Server. The time a file stays in the archive is about six months to one year. Participant Users with access rights can view the Participant Archive in the following interfaces.

  • The web portal > Data Load Import. For help, see Guide to MSATS Web Portal.
  • Participant File Server > Archive.

The folder structure follows the model in Participant archive structure. For example, the full path and name of an archived file might be:

X:\PARTID\Archive\2003\4-Apr\07\15.22.03\catsm_abcde_123456zip

The folders underneath Day contain a maximum of 500 files. The names of these folders correspond to the time when the first file in the folder was archived and follow each other chronologically. The Last Modified date of each file in the directory is the original date the file was first created in your Participant Outbox and remains unchanged.

Figure 152 Participant archive structure

View outstanding messages and acks

You can view your outstanding Messages and ACKS in the MSATS Web Portal > Participants > Participant Schema interface.