Home Monitoring (home made) – Reborn

I’d previously shared on my blog the home made monitoring application I’d built to aggregate pv, meter and weather data – see: http://blog.v-s-f.co.uk/2014/03/home-monitoring-home-made-overview/.

Over the last few months I re-wrote the app to not only output the data to PVOutput every minute, but store that same data into an HSQLDB on my server. The new code base uses less Java than the previous version and moves a lot of the aggregation into Mule flows. I also took the opportunity to try out Git Hub, so the code is all available online here: https://github.com/vls29/aggregator

So far it’s been running very stable since the start of January and the refactoring enables new data sources to be added very quickly, e.g. Mains Voltage as can be seen below on the graph as a purple line starting above 3000W:

mains-voltage

(link to my system on PVOutput: http://pvoutput.org/intraday.jsp?id=4836&sid=4409)

Mule Enum in Flow – issue with enum of same name as variable

When creating a flow in Mule recently, I hit an issue that kept me scratching my head for a number of hours, so thought I’d share it…

<?xml version="1.0" encoding="UTF-8"?>

<mule xmlns:http="http://www.mulesoft.org/schema/mule/http" xmlns:servlet="http://www.mulesoft.org/schema/mule/servlet" xmlns:vm="http://www.mulesoft.org/schema/mule/vm" xmlns="http://www.mulesoft.org/schema/mule/core" xmlns:doc="http://www.mulesoft.org/schema/mule/documentation"
	xmlns:spring="http://www.springframework.org/schema/beans" version="CE-3.5.0"
	xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-current.xsd
http://www.mulesoft.org/schema/mule/core http://www.mulesoft.org/schema/mule/core/current/mule.xsd
http://www.mulesoft.org/schema/mule/vm http://www.mulesoft.org/schema/mule/vm/current/mule-vm.xsd
http://www.mulesoft.org/schema/mule/servlet http://www.mulesoft.org/schema/mule/servlet/current/mule-servlet.xsd
http://www.mulesoft.org/schema/mule/http http://www.mulesoft.org/schema/mule/http/current/mule-http.xsd">

    <flow name="input-data" doc:name="input-data">
        <composite-source doc:name="Composite Source">
            <vm:inbound-endpoint exchange-pattern="one-way" path="pvoutput-data" doc:name="vm-pvoutput-data"/>
            <servlet:inbound-endpoint path="/pvoutput-data" responseTimeout="10000" doc:name="servlet-pvoutput-data"/>
        </composite-source>
        <set-variable variableName="Received" value="#[new java.util.GregorianCalendar()]" doc:name="Received Date Time"/>
        <set-variable variableName="DataSource" value="#[DataSource.valueOf(message.inboundProperties['X-Data-Source'])]" doc:name="Source"/>
        <http:body-to-parameter-map-transformer doc:name="Body to Parameter Map"/>
        <logger message="#[message]" level="DEBUG" doc:name="Logger"/>
        <logger message="Received #[DataSource] Data: #[payload]" level="INFO" doc:name="Logger"/>
        <choice doc:name="Source Choice">
            <when expression="#[DataSource == DataSource.HOT_WATER]">
                <expression-transformer doc:name="Hot Water" expression="#[new HotWaterData(Received, payload);]" />
            </when>
            <when expression="#[DataSource == DataSource.WEATHER]">
                <expression-transformer doc:name="Weather" expression="#[new WeatherData(Received, payload);]" />
            </when>
            <when expression="#[DataSource == DataSource.INVERTER]">
                <expression-transformer doc:name="Inverter" expression="#[new GenerationData(Received, payload);]" />
            </when>
            <when expression="#[DataSource == DataSource.METER]">
                <expression-transformer doc:name="Meter" expression="#[new MeterData(Received, payload);]" />
            </when>
            <otherwise>
                <logger message="Unexpected Data Source Type: #[DataSource]" level="ERROR" doc:name="Logger"/>
            </otherwise>
        </choice>
        <vm:outbound-endpoint exchange-pattern="one-way" doc:name="combine-data" path="combine-data"/>
    </flow>
</mule>

(or in pictorial format)
aggregator

I’ve created a simple flow which receives a request on a servlet endpoint and then adds a variable depending on the source. The variable is named “DataSource”, but I also have an enum named DataSource which contains the following values:

package uk.co.vsf.aggregator.domain;

public enum DataSource {
	HOT_WATER,
	WEATHER,
	INVERTER,
	METER
	;
}

There’s also a Mule configuration block which has imports to a number of classes including the DataSource enum.

<configuration doc:name="Configuration">
		<expression-language autoResolveVariables="true">
			<import class="uk.co.vsf.aggregator.domain.DataSource"/>
			<import class="uk.co.vsf.aggregator.domain.HotWaterData"/>
			<import class="uk.co.vsf.aggregator.domain.WeatherData"/>
			<import class="uk.co.vsf.aggregator.domain.GenerationData"/>
			<import class="uk.co.vsf.aggregator.domain.MeterData"/>
			<import class="uk.co.vsf.aggregator.pvoutput.domain.AddStatusRequest"/>
			<import class="uk.co.vsf.aggregator.CalendarUtils"/>
			<import class="java.util.Calendar"/>
			<import class="java.util.GregorianCalendar"/>
		</expression-language>
	</configuration>

I’ve then written a simple test which calls the composite input vm with a set of values and a data source of Hot_Water. You’ll notice from the flow that after determining the data source, it logs out the message, so we can see the stored variable:

Message properties:
  INVOCATION scoped properties:
    DataSource=HOT_WATER
...

All ok at the moment…
Just below that log statement is another which prints out the DataSource, but this time we get:

Received class uk.co.vsf.aggregator.domain.DataSource Data: {t=40.56, i=10}

Not what I’d expected!
And after that log statement the data source is used to determine the route of the message being processed. I expect it to get transformed into a HotWaterData object, but instead the message goes to the else statement:

Unexpected Data Source Type: class uk.co.vsf.aggregator.domain.DataSource

Well… after much head scratching… it was as simple as renaming the variable to something other than the same name as the enum value! (DOH)

Now I get the results I’d expected:

Received HOT_WATER Data: {t=40.56, i=10}

And the returned object is an instance of HotWaterData.

Hope this helps someone else!

Email Price Checker – Part 3: Filtering From Addresses

In the part 2, the basic app and splitting into sub codes had been built. To keep this part short, I’ll just be adding fromAddress filtering.

When a price request is received via email, not only does the payload have to be of the correct type and contain a code, but the email needs to have originated from an email address in a “white list” (someone I know about and let use the feature). So the first step is to add a filter reference and a new filter which checks the inbound property “fromAddress” against a list of addresses in the properties file.

price-checker-16

The next step is to have a basic filter which can take a list of spring injected valid from addresses to check the inbound property against. Because the fromAddress is similar to the replyToAddresses in that looks like this: “Fred Bloggs ” I’ve split the email address cleaning out into a separate class which can be reused from the filter and in the ReplyToAddressTransformer.

These are the classes:

EmailCleaningComponentImpl.java

package uk.co.vsf.pricechecker.component.impl;

import uk.co.vsf.pricechecker.component.EmailCleaningComponent;

public class EmailCleaningComponentImpl implements EmailCleaningComponent {

	/**
	 * {@inheritDoc}
	 */
	@Override
	public String cleanEmail(String uncleanedEmail) {
		String[] parts = uncleanedEmail.split("<");
		if (parts.length > 1) {
			parts = parts[1].split(">");
			if (parts.length == 1) {
				return parts[0];
			}
		}

		throw new IllegalArgumentException("No valid email address in - " + uncleanedEmail);
	}
}

ReplyToAddressTransformer.java

package uk.co.vsf.pricechecker.transformer;

import org.mule.api.MuleMessage;
import org.mule.api.transformer.TransformerException;
import org.mule.transformer.AbstractMessageTransformer;

import uk.co.vsf.pricechecker.component.EmailCleaningComponent;

/**
 * Transforms the incoming message replyToAddresses into a clean email address.
 */
public class ReplyToAddressTransformer extends AbstractMessageTransformer {
	public static final String CLEANED_REPLY_TO_ADDRESS = "CleanedReplyToAddress";

	private EmailCleaningComponent emailCleaningComponent;

	@Override
	public Object transformMessage(MuleMessage message, String outputEncoding) throws TransformerException {
		String replyToAddresses = message.getInboundProperty("replyToAddresses");
		if (replyToAddresses == null) {
			throw new IllegalArgumentException("Need a replyToAddress in order to return email");
		}

		String cleanedEmail = emailCleaningComponent.cleanEmail(replyToAddresses);
		message.setOutboundProperty(CLEANED_REPLY_TO_ADDRESS, cleanedEmail);
		return message;
	}

	public void setEmailCleaningComponent(EmailCleaningComponent emailCleaningComponent) {
		this.emailCleaningComponent = emailCleaningComponent;
	}
}

And…

FromAddressFilter.java

package uk.co.vsf.pricechecker.filter;

import java.util.ArrayList;
import java.util.List;

import org.mule.api.MuleMessage;
import org.mule.api.routing.filter.*;

import uk.co.vsf.pricechecker.component.EmailCleaningComponent;

/**
 * Determines whether the from address is in the list of valid email addresses
 * and if so lets the message through.
 */
public class FromAddressFilter implements Filter {

	private EmailCleaningComponent emailCleaningComponent;
	private List<String> validFromAddresses = new ArrayList<String>();

	@Override
	public boolean accept(MuleMessage message) {
		String fromAddress = message.getInboundProperty("fromAddress", "");
		String cleanedFromAddress = emailCleaningComponent.cleanEmail(fromAddress);

		return validFromAddresses.contains(cleanedFromAddress.toUpperCase());
	}

	/**
	 * Complex setter which converts from the string array values spring injects
	 * to a list of upper case email addresses.
	 * 
	 * @param values
	 *            list of valid email addresses
	 */
	public void setValidFromAddresses(String[] values) {
		if (values != null) {
			for (String value : values) {
				validFromAddresses.add(value.toUpperCase());
			}
		}
	}

	public void setEmailCleaningComponent(EmailCleaningComponent emailCleaningComponent) {
		this.emailCleaningComponent = emailCleaningComponent;
	}
}

Add the EmailCleaningComponent and FromAddressFilter as a spring bean into the mule config file and tie them together as well as setting the filter reference to the spring bean. Additionally alter the ReplyToAddressTransformer to have a spring property of the EmailCleaningComponent.

<spring:beans>
        <spring:bean id="FromAddressFilter" name="FromAddressFilter" class="uk.co.vsf.pricechecker.filter.FromAddressFilter" scope="singleton">
            <spring:property name="emailCleaningComponent" ref="EmailCleaningComponent"/>
            <spring:property name="validFromAddresses" value="${valid.from.addresses}" />
        </spring:bean>
        <spring:bean id="EmailCleaningComponent" name="EmailCleaningComponent" class="uk.co.vsf.pricechecker.component.impl.EmailCleaningComponentImpl" scope="singleton"/>
    </spring:beans>
    ...
    <filter ref="FromAddressFilter" doc:name="Filter Reference"/>
    <custom-transformer class="uk.co.vsf.pricechecker.transformer.ReplyToAddressTransformer"
			doc:name="Java">
	<spring:property name="emailCleaningComponent" ref="EmailCleaningComponent"/>
    </custom-transformer>