Victoria's Blog | PVOutput integration v1

PVOutput integration v1

29/01/2012

As I mentioned back in November, I’ve been working on integrating with the PVOutput website so I don’t have to manually read the Sunnybeam output files and add stats to the website.

In order to be able to look at creating a solution, I need a set of requirements. These requirements are ones I have thought up:

Read new files off of the sunny beam into specified directory
I store the sunny beams output files on my computer as the device only keeps a rolling 90 days of output stats. If I ever wanted to look back at older stats, I would need them on my pc.
Separate the daily files from the monthly files
The sunny beam creates a daily file and at the end of the month, a month file. Monthly files are stored in a different directory to the daily files on my pc.
Upload new file data to pvoutput
I need to be able to upload the new files which have been transferred to my pc to pvoutput.
Upload weather data if available from csv file in root of file dir
For each day, an average picture of the weather can be set on the record created on pvoutput. As such I shall be creating a simple csv on my pc in the root directory where the outputs are stored which will contain a date and a weather value for that day.

There are also requirements from PVOutput:

Make use of the API provided
Call services providing system Id and API key in the header
Limit the number of calls per hour and minute according to the guidelines

From those four requirements I can break the work needed down into the following:
Create a csv sunny beam reader which will gather the statistics from the file.
Create a csv reader for the weather file
Read the latest output files from the sunny beam and copy them to the directory chosen on pc, separating monthly files from daily files.
Ask pvoutput for the last recording.
The getMissing service on PVOutput takes two parameters df and dt which should be dynamic. So as I’m using Mule – the url contains the expressions to pick out the properties from the MuleMessage header. The response from this service is a list of dates. The dates need to be converted to a Calendar object once received.
Upload to pvoutput from the last recording.

The solution I have so far created is only partially written. I still need to add the weather section and decide how the application will be started / run. If the application is run as a service there are benefits, particularly in not breaking the guideline limits on the service calls to pvoutput. Also as new csv files are created on the device they could be uploaded almost straight away.

There is also no error checking on the service calls to pvoutput. This is relatively easy to add in and I’ll get round to it in version 2.

The testing also needs to be beefed up. There are tests in the project, but more are needed. Especially around reading files off the sunny beam and copying them. Also around error handling on service calls.

The next section is about the most interesting aspects of the code so far

I took the rather lazy approach to using Mule :-). At the same time, I decided to try out something I haven’t had a chance to do before in main code (only test code previously) – calling services using the MuleClient.
Creating the MuleContext and MuleClient is potentially expensive, so there is a factory storing the MuleContext and MuleClient for further use.

MuleFactoryImpl

package uk.co.vsf.solar.mule.impl;

import org.mule.DefaultMuleMessage;
import org.mule.api.MuleContext;
import org.mule.api.MuleException;
import org.mule.api.MuleMessage;
import org.mule.api.config.ConfigurationException;
import org.mule.api.lifecycle.InitialisationException;
import org.mule.context.DefaultMuleContextFactory;
import org.mule.module.client.MuleClient;

import uk.co.vsf.solar.mule.GeneralMuleException;
import uk.co.vsf.solar.mule.MuleFactory;

public class MuleFactoryImpl implements MuleFactory {

    private static final String MULE_CONFIGS = "mule/pvoutput.xml,mule/pvoutput-getmissing.xml,mule/pvoutput-addoutput.xml,spring/applicationContext.xml";

    private MuleClient muleClient;
    private MuleContext muleContext;

    /**
     * {@inheritDoc}
     */
    public MuleClient getClient() {
        if (muleClient == null) {
            try {
                DefaultMuleContextFactory mcfactory = new DefaultMuleContextFactory();
                muleContext = mcfactory.createMuleContext(MULE_CONFIGS);

                muleContext.start();
                muleClient = new MuleClient(muleContext);
            } catch (InitialisationException e) {
                throw new GeneralMuleException();
            } catch (ConfigurationException e) {
                throw new GeneralMuleException();
            } catch (MuleException e) {
                throw new GeneralMuleException();
            }
        }

        return muleClient;
    }

    /**
     * {@inheritDoc}
     */
    public MuleMessage createMuleMessage(Object payload) {
        return new DefaultMuleMessage(payload, muleContext);
    }
}

The add output Mule config file calls the addoutput service on pvoutput indirectly through TcpTrace (What is TcpTrace). The service is post and the mime type has been set to represent form data. Without that you may find unusual data sent…
In the post request, I’ve used message property transformers to add in the pvoutput apikey and systemId. These will appear in the header when sent.
pvoutput-addoutput.xml

<mule xmlns="http://www.mulesoft.org/schema/mule/core" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xmlns:spring="http://www.springframework.org/schema/beans" xmlns:http="http://www.mulesoft.org/schema/mule/http"
	xmlns:vm="http://www.mulesoft.org/schema/mule/vm"
	xsi:schemaLocation="
       http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.0.xsd
       http://www.mulesoft.org/schema/mule/core http://www.mulesoft.org/schema/mule/core/3.2/mule.xsd
       http://www.mulesoft.org/schema/mule/http http://www.mulesoft.org/schema/mule/http/3.2/mule-http.xsd
       http://www.mulesoft.org/schema/mule/vm http://www.mulesoft.org/schema/mule/vm/3.2/mule-vm.xsd">

	<flow name="addOutputFlow">
		<vm:inbound-endpoint path="addoutput"
							 exchange-pattern="request-response"
							 connector-ref="pvoutputConnector" />
		<http:outbound-endpoint address="http://localhost:40000/service/r2/addoutput.jsp"
								method="POST"
								connector-ref="HttpConnector"
								exchange-pattern="request-response"
								followRedirects="false"
								mimeType="application/x-www-form-urlencoded">
			<message-properties-transformer>
				<add-message-property key="X-Pvoutput-Apikey" value="${pvoutput.api.key}" />
				<add-message-property key="X-Pvoutput-SystemId" value="${pvoutput.system.id}" />
			</message-properties-transformer>
		</http:outbound-endpoint>
	</flow>

</mule>

The same message transformation is applied to the get missing service request. Also for this service, the url has to change to get the dates of outputs not added from the system install date to today. For that I’ve used message property EL selection.
pvoutput-getmissing.xml


<mule xmlns="http://www.mulesoft.org/schema/mule/core" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xmlns:spring="http://www.springframework.org/schema/beans" xmlns:http="http://www.mulesoft.org/schema/mule/http"
	xmlns:vm="http://www.mulesoft.org/schema/mule/vm"
	xsi:schemaLocation="
       http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.0.xsd
       http://www.mulesoft.org/schema/mule/core http://www.mulesoft.org/schema/mule/core/3.2/mule.xsd
       http://www.mulesoft.org/schema/mule/http http://www.mulesoft.org/schema/mule/http/3.2/mule-http.xsd
       http://www.mulesoft.org/schema/mule/vm http://www.mulesoft.org/schema/mule/vm/3.2/mule-vm.xsd">

	<flow name="getMissingFlow">
		<vm:inbound-endpoint ref="GetMissingEndpoint" />
		<http:outbound-endpoint address="http://localhost:40000/service/r1/getmissing.jsp?df=#[header:INBOUND:df]&dt=#[header:INBOUND:dt]"
								method="GET"
								connector-ref="HttpConnector"
								exchange-pattern="request-response"
								followRedirects="false">
			<message-properties-transformer>
				<add-message-property key="X-Pvoutput-Apikey" value="${pvoutput.api.key}" />
				<add-message-property key="X-Pvoutput-SystemId" value="${pvoutput.system.id}" />
			</message-properties-transformer>
		</http:outbound-endpoint>
	</flow>

</mule>