Email Price Checker – Part 2: Code Verifying and Splitting

In part 1 we built the basics of the app – and by basics, it really was basic. In this part, code checking and sub code splitting will be added.

To start with we’ll add sub code splitting.

This is heavily dependent on the website code and for the website I’m price checking (CPC) it uses the following code structures. The first code is the code itself. E.g. LP04169
Then there are the code + a two digit subcode starting at 01 up to 99. E.g. LP0416901 to LP0416999

The first class needed here is a class to represent the Item being priced. The item being priced has a code, sub code (where appropriate), a flag to indicate whether it was available, and a price with and without VAT.

package uk.co.vsf.domain;

import java.math.BigDecimal;

import org.apache.commons.lang.builder.ToStringBuilder;

/**
 * Represents an item that's being priced checked.
 */
public class Item {

	private String code;
	private String subCode;
	private boolean available;
	private BigDecimal excVat;
	private BigDecimal incVat;

	public Item(String code, String subCode) {
		this.code = code;
		this.subCode = subCode;
	}

	public Item(String code) {
		this.code = code;
	}

	public String toString() {
		ToStringBuilder tsb = new ToStringBuilder(this);
		tsb.append(code);
		tsb.append(subCode);
		tsb.append(available);
		tsb.append(excVat);
		tsb.append(incVat);

		return tsb.toString();
	}

	public String getCode() {
		return code;
	}

	public String getSubCode() {
		return subCode;
	}

	public boolean isAvailable() {
		return available;
	}

	public void setAvailable(boolean available) {
		this.available = available;
	}

	public BigDecimal getExcVat() {
		return excVat;
	}

	public void setExcVat(BigDecimal excVat) {
		this.excVat = excVat;
	}

	public BigDecimal getIncVat() {
		return incVat;
	}

	public void setIncVat(BigDecimal incVat) {
		this.incVat = incVat;
	}
}

In order to create the list of items that will be priced, we need to transform the payload to create an item for the code and all sub codes. So, a custom transformer has been used.

package uk.co.vsf.pricechecker.transformer;

import java.util.ArrayList;
import java.util.List;

import org.mule.api.transformer.TransformerException;
import org.mule.transformer.AbstractTransformer;

import uk.co.vsf.domain.Item;

/**
 * Transforms the incoming code into X number of sub codes.
 */
public class SubCodeTransformer extends AbstractTransformer {

	private int subCodeQuantity;

	@Override
	protected Object doTransform(Object src, String enc)
			throws TransformerException {
		if (!(src instanceof String)) {
			return src;
		}

		List<Item> items = new ArrayList<Item>();

		addInitialItem((String) src, items);
		addSubCodeItems((String) src, items);

		return items;
	}

	private void addInitialItem(String code, List<Item> items) {
		Item item = new Item(code);
		items.add(item);
	}

	private void addSubCodeItems(String code, List<Item> items) {
		for (int i = 0; i < subCodeQuantity; i++) {
			String subCode = (i < 10 ? "0" : "") + i;
			Item item = new Item(code, subCode);
			items.add(item);
		}
	}

	public void setSubCodeQuantity(int subCodeQuantity) {
		this.subCodeQuantity = subCodeQuantity;
	}
}

The next step it to modify the flow to include the transformer and a property to specify how many sub codes.

price-checker-10

The payload after the transformer will be a list of Items which then needs to be split so that each can be processed individually.

price-checker-11

And at this point it’s worth testing what we’ve got! So a logger has been added to the end to print out the message payload.

price-checker-12

Having sent a basic test email to the app, this is the log result:

Payload after split: uk.co.vsf.domain.Item@3e8ad1[Abc1225,96,false,<null>,<null>]
Payload after split: uk.co.vsf.domain.Item@bb1ee[Abc1225,97,false,<null>,<null>]
Payload after split: uk.co.vsf.domain.Item@1ce835b[Abc1225,98,false,<null>,<null>]
Payload after split: uk.co.vsf.domain.Item@11201a1[Abc1225,99,false,<null>,<null>]

The other requirement was to filter received messages containing a valid code. In the case of the website I’m price checking, the codes are 2 characters, 5 numbers, which makes for a good regex \w\w\d\d\d\d\d (or another permutation which does the same). If the code is valid, the item will be price checked, otherwise the message could be terminated, but in this case, it’s more useful to send the sender a message to say that the code isn’t valid. So a choice element makes more sense. One route will go to the splitter and the other to an SMTP endpoint.

In order to make the flow easier to read once the choice element has been added, a sub flow has been added to contain the happy path after the filter logic.

price-checker-14

price-checker-15

The filter default SMTP endpoint needs the from address from the email in order to send a response back to the user. This was a little tricky to find, but by putting full trace on, you’ll be able to see the entire Email properties and pick the one that suits best. Alternatively use a simple transformer (etc) and debug the entry to see the message properties.

When I actually tried to send a filter exception response email I kept getting a 501 warning about a host being needed on the address. It turns out that the from address shouldn’t be encoded like the user account… (things like that annoy me)

The complete flow code now looks like:

<?xml version="1.0" encoding="UTF-8"?>

<mule xmlns:smtps="http://www.mulesoft.org/schema/mule/smtps" xmlns:tracking="http://www.mulesoft.org/schema/mule/ee/tracking"
	xmlns:core="http://www.mulesoft.org/schema/mule/core" xmlns:context="http://www.springframework.org/schema/context"
	xmlns:pop3s="http://www.mulesoft.org/schema/mule/pop3s" xmlns:smtp="http://www.mulesoft.org/schema/mule/smtp"
	xmlns:email="http://www.mulesoft.org/schema/mule/email" xmlns:pop3="http://www.mulesoft.org/schema/mule/pop3"
	xmlns="http://www.mulesoft.org/schema/mule/core" xmlns:doc="http://www.mulesoft.org/schema/mule/documentation"
	xmlns:spring="http://www.springframework.org/schema/beans" version="EE-3.3.2"
	xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="
http://www.mulesoft.org/schema/mule/pop3s http://www.mulesoft.org/schema/mule/pop3s/current/mule-pop3s.xsd 
http://www.mulesoft.org/schema/mule/smtps http://www.mulesoft.org/schema/mule/smtps/current/mule-smtps.xsd 
http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-current.xsd 
http://www.mulesoft.org/schema/mule/core http://www.mulesoft.org/schema/mule/core/current/mule.xsd 
http://www.mulesoft.org/schema/mule/ee/tracking http://www.mulesoft.org/schema/mule/ee/tracking/current/mule-tracking-ee.xsd 
http://www.mulesoft.org/schema/mule/email http://www.mulesoft.org/schema/mule/email/current/mule-email.xsd">

	<pop3s:connector name="Pop3sConnector" doc:name="POP3"
		checkFrequency="${email.check.frequency}">
		<reconnect-forever frequency="${email.reconnect.retry.frequency}" />
	</pop3s:connector>

	<context:property-placeholder location="pricechecker.properties" />
	
	<flow name="CPCPriceCheckerInbound" doc:name="CPCPriceCheckerInbound">
		<pop3s:inbound-endpoint host="${email.host}"
			port="${email.port.pop3}" user="${email.account}" password="${email.password}"
			responseTimeout="${email.response.timeout}" connector-ref="Pop3sConnector"
			doc:name="POP3" />
        <payload-type-filter expectedType="java.lang.String" doc:name="Payload"/>
        <custom-transformer
			class="uk.co.vsf.pricechecker.transformer.ReplyToAddressTransformer"
			doc:name="Java">
		</custom-transformer>
		<set-payload value="#[message.payload.trim()]" doc:name="Set Payload"/>
        <custom-transformer
			class="uk.co.vsf.pricechecker.transformer.CodeTransformer"
			doc:name="Java">
		</custom-transformer>
		<choice doc:name="Choice">
			<when expression="#[message.outboundProperties['CpcMatch']]">
				<processor-chain>
					<flow-ref name="TransformAndSplitLogic" doc:name="Flow Reference" />
				</processor-chain>
			</when>
			<otherwise>
				<processor-chain>
                    <set-payload value="Code: '#[message.payload.toString()]' is not a valid CPC code" doc:name="Set Payload"/>
					<smtps:outbound-endpoint host="${email.host}" port="${email.port.smtp}"
						responseTimeout="10000" doc:name="SMTP" 
						user="${email.account}" password="${email.password}"
						from="${email.from}" replyTo="${email.replyTo}" subject="${email.subject}"
						to="#[message.outboundProperties['CleanedReplyToAddress']]" />
				</processor-chain>
			</otherwise>
		</choice>
	</flow>
	<sub-flow name="TransformAndSplitLogic" doc:name="TransformAndSplitLogic">
		<custom-transformer
			class="uk.co.vsf.pricechecker.transformer.SubCodeTransformer"
			doc:name="Java">
			<spring:property name="subCodeQuantity" value="${number.of.sub.codes}" />
		</custom-transformer>
		<splitter expression="#[message.payload]" doc:name="Splitter" />
		<logger message="Payload after split: #[message.payload]"
			level="INFO" doc:name="Logger" />
	</sub-flow>
</mule>

There is a problem with the response subject if the filter hasn’t been met being a copy of the original request subject rather than the overridden properties value.

Two new transformers were also needed.

One for the code checking.

package uk.co.vsf.pricechecker.transformer;

import org.mule.api.MuleMessage;
import org.mule.api.transformer.TransformerException;
import org.mule.transformer.AbstractMessageTransformer;

/**
 * Determines the pricing website the code is applicable for.
 */
public class CodeTransformer extends AbstractMessageTransformer {

	private String cpcRegex = "\\w\\w\\d\\d\\d\\d\\d";

	@Override
	public Object transformMessage(MuleMessage message, String outputEncoding) throws TransformerException {
		if (!(message.getPayload() instanceof String)) {
			return message;
		}

		String payload = (String) message.getPayload();
		boolean cpcMatch = payload.matches(cpcRegex);

		message.setOutboundProperty("CpcMatch", cpcMatch);

		return message;
	}
}

And another for the toAddress to reply back to in the case of a failure / price response.

package uk.co.vsf.pricechecker.transformer;

import org.mule.api.MuleMessage;
import org.mule.api.transformer.TransformerException;
import org.mule.transformer.AbstractMessageTransformer;

/**
 * Transforms the incoming message replyToAddresses into a clean email address.
 */
public class ReplyToAddressTransformer extends AbstractMessageTransformer {
	public static final String CLEANED_REPLY_TO_ADDRESS = "CleanedReplyToAddress";

	@Override
	public Object transformMessage(MuleMessage message, String outputEncoding) throws TransformerException {
		String replyToAddresses = message.getInboundProperty("replyToAddresses");
		if (replyToAddresses == null) {
			throw new IllegalArgumentException("Need a replyToAddress in order to return email");
		}

		String[] parts = replyToAddresses.split("<");
		if (parts.length > 1) {
			parts = parts[1].split(">");
			if (parts.length == 1) {
				message.setOutboundProperty(CLEANED_REPLY_TO_ADDRESS, parts[0]);
				return message;
			}
		}

		throw new IllegalArgumentException("No valid replyToAddress in email - " + replyToAddresses);
	}
}

There are more improvements to be made if this were production ready code. For example, the ReplyToAddressTransformer might well come unstuck if there was an unusual to replyToAddresses value. Also the CodeTransformer needs the regex moving into properties.

In part 3 I’ll clean up the CodeTransformer and add in address filtering to prevent spam and unwanted email from going through the price checking application.