Quick start

This section describes how to easily store data on the OpeNRJ platform.

Introduction

The OpeNRJ platform is an energy data storage platform, offering an Open Data API for communities wanting to freely share their data, all energy data received is stored as a time series. Energy data can come from various sources (electricity, water or gas meters, temperature or CO2 sensors, door opening or window detectors, etc).

To avoid misunderstanding, this platform uses certain concepts and a certain nomenclature.

The variables

We wanted to avoid creating a platform centered on sensors but rather on data time series, so instead of using the word " sensor " to categorize a time series, We will use the word " variable ". Thus, the same sensor will be able to generate several " variables " (example of a temperature sensor that will be able to move during its lifetime).

More clearly, let's detail together a case of use. Let's say you use a plug-type energy sensor (plug that plugs into an electrical outlet and connects other devices If you connect this sensor to the desktop A and plug a lamp on it, you can call this variable " office lamp_A ". You will therefore create a time series of energy data dedicated to the lamp Office A. If you want to use this socket to analyze the power consumption of the computer screen in the office B, you will unplug the socket from office A, plug it into office B, create a variable called & quot; , And you can create a time series dedicated to the desktop computer screen without conflicting with the first time series, and greatly simplifies computer processing in the event of sensor failure and/or replacement.

The sites

We also use the notion of " site ". A site can be either a university containing several buildings, a public institution, a house, an apartment, etc. Each site can contain several " Variables " and each " variable " can start or end its time series at any time. In our example you will be the manager of a site and you can declare as many " variable " You desire it.

Now that this introduction is complete, we will now describe how to create a site, create variables, store time series for these variables and read these time series.

Step 1 - Creating the Site and User IDs

To start using the OpeNRJ platform, you must go to the page https://sites.openrj.eu In order to create an account, once you have created your account, you will be drafted to the "dashboard"page where you can create a site by clicking the " Add site "button. To add a site, you must enter a unique identifier and a label.

Step 2 - Creating Site Variables

To create the variables of our site, we will first declare our variables in a CSV file and then we will send this file via a REST request.

Format of metadata

The metadata file must be written in a format that the server understands, so it should contain a file declaration header, the column names of the file, and the list of variables and their metadata The separator used is the tab (make sure that you use tabs and not spaces.) Here is an example of a well-formatted file :

                FORMAT_VERSION  1
                CREATION_DATE   2014-02-13T14:22:07Z
                ID_SITE test_site
                FEEDBACK_TO     me@worldcompany.com

                varname label   type    value_type      units   lower_bound     upper_bound     delta_min       delta_max
                energy_aircond_3        aircond_3       energy  N       kWh
                energy_aircond_1        aircond_1       energy  N       kWh
                energy_aircond_2        aircond_2       energy  N       kWh
                energy_plugs_2  plugs_2 energy  N       kWh
                energy_plugs_3  plugs_3 energy  N       kWh
                energy_lights_3 lights_3        energy  N       kWh
                energy_plugs_1  plugs_1 energy  N       kWh
                energy_lights_1 lights_1        energy  N       kWh
                energy_lights_2 lights_2        energy  N       kWh
                

In our previous example :

FORMAT_VERSION Version of the file format used. Currently we are using version 1.
CREATION_DATE File creation date.
ID_SITE Identifier of your site.
FEEDBACK_TO E-mail address of the person in charge of the site who will be notified of the creation of the data.
varname Unique identifier of the variable.
label Label used to name the variable in a text space, for example.
type Type of the variable (energy, temperature, ...).
value_type Type of data received (N for digital, L for logic, T for text).
units Unit of variable.
lower_bound (optional) Lower limit inclusive of allowable values.
upper_bound (optional) Upper limit inclusive of allowable values.
delta_min (optional) Minimum differential inclusive with previous value.
delta_max (optional) Maximum differential inclusive with previous value.

Queries

Now that our file is created, we need to use the following query to send it to the platform (replace SITE_ID with your site ID, username: password with your username and password and path / to / Your / metadata / file.csv by locating your file containing the metadata):

            $ curl -i -X POST 'https://api.openrj.eu/v1/users/current/sites/SITE_ID/variables' -u username:password -F metadata=@path/to/your/metadata/file.csv
            

An HTTP response with status 201 should inform you that the variables have been created. Otherwise, there was an error and you need to correct your file.

Step 3 - Creating the series for a site

Now that we have created our variables, we will be able to send time series.

Since the OpeNRj platform can accommodate a large volume of data, we wanted to minimize the impact on bandwidth, so we chose to write the series in a format Which can be compressed using standard tools and the processing of these files can take a long time, we wanted to dissociate the reception part and the storage part of the data. Asynchronous in order to release as soon as possible our server of reception of the data.

To limit the size we will send a ZIP archive containing all the time series of our site, here is the description of the format expected by the platform.

Format of data

Each time series must be stored in a different TSV file where each entity must be separated by tabs, so there will be a TSV file by Variable. The ZIP archive will be named SITE_ID-series.zip where SITE_ID is the site identifier Each TSV file inside the archive will be named VARIABLE_ID.YYMMDDHHmmss.csv where VARIABLE_ID is the identifier of the variable , And YYMMDDHHmmss is the creation date of the file in YYMMDDHHmmss format. For example, if we create the file on May 20, 2014 at 18:33:24 for variable plug_3, the file name will be plug_3.140520183324.csv. Let's continue our example and that our site is called TEST_PARIS, we will generate a ZIP archive which will have as name TEST_PARIS-series.zip and will contain a single file that will be called plug_3.140520183324.csv.

Here is the format of a time series of a variable (replace VARIABLE_ID with the identifier of your variable, SITE_ID with the identifier of your site and the " timestamps " Data) :

            FORMAT_VERSION  1
            CREATION_DATE   2014-05-15T16:04:34Z
            ID_SITE SITE_ID
            FEEDBACK_TO     me@worldcompany.com
            VARNAME VARIABLE_ID

            2013-10-24T00:00:00Z    11894
            2013-10-24T06:00:00Z    11894
            2013-10-24T12:00:00Z    11894
            2013-10-24T18:00:00Z    11895
            

Dans notre exemple précédent :api_quickstart_

FORMAT_VERSION Version of the file format used. Currently we are using version 1.
CREATION_DATE File creation date.
ID_SITE Identifier of your site./td>
FEEDBACK_TO E-mail address of the person in charge of the site who will be notified of the creation of the data./td>
VARNAME Unique identifier of the variable./td>
timestamp Each subsequent line must contain a valid UTC date in ISO8601 format./td>
value Each following line must contain a value that is consistent with the metadata definition of the variable

Queries

Once we have created our ZIP archive, we will be able to send it to the platform. Here is the query to create all the time series of all the variables of a site (replace SITE_ID with your Site, username: password by your username and password and path / to / your / series / archive.zip by locating your ZIP archive containing the time series) :

            $ curl -i -X POST 'https://api.openrj.eu/v1/users/current/sites/SITE_ID/variables/series' -u username:password -F zip=@path/to/your/series/archive.zip
            

This query will return either a 404 error to tell you that the site does not exist, or a 400 error to tell you that your archive is malformed, or a success message with a jobID. Asynchronous, each ZIP archive sending creates a time series processing job. To verify that the data has been stored successfully, you must make another request (replace SITE_ID with your site identifier, username: password by your Username and your password and JOB_ID by the identifier of the job that was sent to you in the answer of the previous request) :

            $ curl -i -X GET 'https://api.openrj.eu/v1/users/current/sites/SITE_ID/jobs/JOB_ID' -u username:password
            

If the job status answers " Ongoing ... " this means that the job is still processing your archive If the job returns " Completed " Processed and you can retrieve them with the query in the next section, otherwise there is a problem and the error message should help you to correct your archive.

We will now describe how to format your ZIP archive.

Step 4 - Reading the series

To retrieve the time series from the series stored on the server, there are two options :

        $ curl -i -X GET 'https://api.openrj.eu/v1/sites/SITE_ID/variables/series?from=2014-01-01T00:00:00Z&to=2014-12-31T23:59:59Z'
        
        $ curl -i -X GET 'https://api.openrj.eu/v1/sites/SITE_ID/variables/VARIABLE_ID/series?from=2014-01-01T00:00:00Z&to=2014-12-31T23:59:59Z'