Spring properties – multiple values representing a list of objects

There are many cases in a recent project I’ve worked on where we need to configure the app using properties and the properties values represent a list of objects. The list can be a variable size and the objects might not have all the same fields defined. The solution to this that I’ve seen is less than ideal. It involves writing all the fields as one really long string separated by a certain character value. That string is then read in and split when needed. I had a look yesterday on the net for a better solution, but couldn’t find one (might be that I wasn’t sure what to search for). So here’s my simple solution to the problem.

Rather than having a single key in the properties file like:

key=A|B|C

I’d like a properties file to look like:

event.class=uk.co.vsf.utilities.bean.Event
event.size=3
event.0.name=Ball
event.0.date=06/30/2012
event.0.numberOfGuests=1200
event.0.description=Work Ball
event.1.name=Ball
event.1.date=07/14/2012
event.1.numberOfGuests=97
event.1.description=Own Ball
event.2.name=Ball
event.2.date=07/28/2012
event.2.description=Olympics Weekend

The first thing I wrote was the test class:

@ContextConfiguration(locations={"/spring-cfg.xml"})
public class EventsTest extends AbstractTestNGSpringContextTests {

    @Autowired
    private Events events;

    @Test
    public void getListOfEventsOk()
    {
        Event firstEvent = new Event("Ball", "06/30/2012", "1200", "Work Ball");

        List<Event> eventsFound = events.getEvents();

        assertEquals(eventsFound.size(), 3);
        assertEquals(eventsFound.get(0), firstEvent);
    }
}

It’s not particularly interesting test class. It simply checks that when the list of events is retrieved, there should be 3 in the list, the first object should equal the object defined above. It’ll also test the spring wiring is correct when running the test.

The next thing I needed was a bean to represent the data from the properties file

public class Event {
    public static final String NAME = "name";
    public static final String DATE = "date";
    public static final String NUMBER_OF_GUESTS = "numberOfGuests";
    public static final String DESCRIPTION = "description";

    private String name;
    private Date date;
    private int numberOfGuests;
    private String description;

    public Event() {

    }

    public Event(String name, String date, String numberOfGuests, String description) {
        this.name = name;
        this.date = new Date(date);
        if (numberOfGuests != null) {
            this.numberOfGuests = Integer.parseInt(numberOfGuests);
        }
        this.description = description;
    }

    @Override
    public int hashCode() {
        final int prime = 31;
        int result = 1;
        result = prime * result + ((date == null) ? 0 : date.hashCode());
        result = prime * result + ((description == null) ? 0 : description.hashCode());
        result = prime * result + ((name == null) ? 0 : name.hashCode());
        result = prime * result + numberOfGuests;
        return result;
    }

    @Override
    public boolean equals(Object obj) {
        if (this == obj)
            return true;
        if (obj == null)
            return false;
        if (getClass() != obj.getClass())
            return false;
        Event other = (Event) obj;
        if (date == null) {
            if (other.date != null)
                return false;
        } else if (!date.equals(other.date))
            return false;
        if (description == null) {
            if (other.description != null)
                return false;
        } else if (!description.equals(other.description))
            return false;
        if (name == null) {
            if (other.name != null)
                return false;
        } else if (!name.equals(other.name))
            return false;
        if (numberOfGuests != other.numberOfGuests)
            return false;
        return true;
    }

    public String getName() {
        return name;
    }

    public Date getDate() {
        return date;
    }

    public int getNumberOfGuests() {
        return numberOfGuests;
    }

    public String getDescription() {
        return description;
    }

    public void setName(String name) {
        this.name = name;
    }

    public void setDate(Date date) {
        this.date = date;
    }

    public void setNumberOfGuests(int numberOfGuests) {
        this.numberOfGuests = numberOfGuests;
    }

    public void setDescription(String description) {
        this.description = description;
    }
}

The beans fairly self explanatory. (The setters are for the second part)

I then needed a very simple interface for the Events “service”.

public interface Events {

    public List<Event> getEvents();
}

And a spring config file which is fairly basic, but loads the properties file into a spring class PropertiesFactoryBean and injects the properties file into EventsImpl.

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
	xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.1.xsd">

	<bean id="propertyFactoryBean"
		class="org.springframework.beans.factory.config.PropertiesFactoryBean">
		<property name="locations">
			<list>
				<value>classpath:default.properties</value>
			</list>
		</property>
	</bean>

	<bean id="events"
		class="uk.co.vsf.utilities.properties.EventsImpl"
		init-method="afterPropertiesSet">
		<property name="properties" ref="propertyFactoryBean" />
		<property name="propertyKey" value="event" />
	</bean>
</beans>

Last but not least is the EventsImpl class which does all the work.

public class EventsImpl implements Events {

    private Properties properties;
    private String propertyKey;
    private List<Event> events = new ArrayList<Event>();

    public void setProperties(Properties properties) {
        this.properties = properties;
    }

    public void setPropertyKey(String propertyKey) {
        this.propertyKey = propertyKey;
    }

    public void afterPropertiesSet() throws Exception {
        int countOfPropertiesForKey = Integer.parseInt(properties.getProperty(propertyKey + ".size"));

        for (int i = 0; i < countOfPropertiesForKey; i++) {
            String name = properties.getProperty(propertyKey + "." + i + "." + Event.NAME);
            String date = properties.getProperty(propertyKey + "." + i + "." + Event.DATE);
            String numberOfGuests = properties.getProperty(propertyKey + "." + i + "." + Event.NUMBER_OF_GUESTS);
            String description = properties.getProperty(propertyKey + "." + i + "." + Event.DESCRIPTION);
            Event event = new Event(name, date, numberOfGuests, description);
            events.add(event);
        }
    }

    @Override
    public List<Event> getEvents() {
        return events;
    }
}

On startup the afterPropertiesSet method will be called. First of all the size property is read from the properties file. With that count, a loop is created and the properties are read from the file. An Event object is then created and filled in with the values. The Event object is then added to the Event field list.

This solution works, but it’s not very generic as the programmer has to know the types of fields which will be present in the properties file and object and also create an instance of the specific type of object.

The alternative is generic and makes use of BeanUtils from apache commons and also creates a specific class based on the properties value:

public class EventsImpl implements Events {

    private Properties properties;
    private String propertyKey;
    private List<Object> events = new ArrayList<Object>();

    public void setProperties(Properties properties) {
        this.properties = properties;
    }

    public void setPropertyKey(String propertyKey) {
        this.propertyKey = propertyKey;
    }

    public void afterPropertiesSet() throws Exception {
        int countOfPropertiesForKey = Integer.parseInt(properties.getProperty(propertyKey + ".size"));
        String className = properties.getProperty(propertyKey + ".class");

        Object clazz = ClassUtils.getClass(className).newInstance();
        Map propertyMap = BeanUtils.describe(clazz);
        Set<String> fieldNames = propertyMap.keySet();

        for (int i = 0; i < countOfPropertiesForKey; i++) {
            Object instance = ClassUtils.getClass(className).newInstance();
            for(String fieldName: fieldNames)
            {
                String fieldValue = properties.getProperty(propertyKey + "." + i + "." + fieldName);
                BeanUtils.setProperty(instance, fieldName, fieldValue);
            }
            events.add(instance);
        }
    }

    @Override
    public List<Object> getEvents() {
        return events;
    }
}

PVOutput integration v1

As I mentioned back in November, I’ve been working on integrating with the PVOutput website so I don’t have to manually read the Sunnybeam output files and add stats to the website.

In order to be able to look at creating a solution, I need a set of requirements. These requirements are ones I have thought up:

  • Read new files off of the sunny beam into specified directory

    I store the sunny beams output files on my computer as the device only keeps a rolling 90 days of output stats. If I ever wanted to look back at older stats, I would need them on my pc.

  • Separate the daily files from the monthly files

    The sunny beam creates a daily file and at the end of the month, a month file. Monthly files are stored in a different directory to the daily files on my pc.

  • Upload new file data to pvoutput

    I need to be able to upload the new files which have been transferred to my pc to pvoutput.

  • Upload weather data if available from csv file in root of file dir

    For each day, an average picture of the weather can be set on the record created on pvoutput. As such I shall be creating a simple csv on my pc in the root directory where the outputs are stored which will contain a date and a weather value for that day.

There are also requirements from PVOutput:

  • Make use of the API provided
  • Call services providing system Id and API key in the header
  • Limit the number of calls per hour and minute according to the guidelines

From those four requirements I can break the work needed down into the following:
Create a csv sunny beam reader which will gather the statistics from the file.
Create a csv reader for the weather file
Read the latest output files from the sunny beam and copy them to the directory chosen on pc, separating monthly files from daily files.
Ask pvoutput for the last recording.
The getMissing service on PVOutput takes two parameters df and dt which should be dynamic. So as I’m using Mule – the url contains the expressions to pick out the properties from the MuleMessage header. The response from this service is a list of dates. The dates need to be converted to a Calendar object once received.
Upload to pvoutput from the last recording.

The solution I have so far created is only partially written. I still need to add the weather section and decide how the application will be started / run. If the application is run as a service there are benefits, particularly in not breaking the guideline limits on the service calls to pvoutput. Also as new csv files are created on the device they could be uploaded almost straight away.

There is also no error checking on the service calls to pvoutput. This is relatively easy to add in and I’ll get round to it in version 2.

The testing also needs to be beefed up. There are tests in the project, but more are needed. Especially around reading files off the sunny beam and copying them. Also around error handling on service calls.

The next section is about the most interesting aspects of the code so far

I took the rather lazy approach to using Mule :-). At the same time, I decided to try out something I haven’t had a chance to do before in main code (only test code previously) – calling services using the MuleClient.
Creating the MuleContext and MuleClient is potentially expensive, so there is a factory storing the MuleContext and MuleClient for further use.

MuleFactoryImpl

package uk.co.vsf.solar.mule.impl;

import org.mule.DefaultMuleMessage;
import org.mule.api.MuleContext;
import org.mule.api.MuleException;
import org.mule.api.MuleMessage;
import org.mule.api.config.ConfigurationException;
import org.mule.api.lifecycle.InitialisationException;
import org.mule.context.DefaultMuleContextFactory;
import org.mule.module.client.MuleClient;

import uk.co.vsf.solar.mule.GeneralMuleException;
import uk.co.vsf.solar.mule.MuleFactory;

public class MuleFactoryImpl implements MuleFactory {

    private static final String MULE_CONFIGS = "mule/pvoutput.xml,mule/pvoutput-getmissing.xml,mule/pvoutput-addoutput.xml,spring/applicationContext.xml";

    private MuleClient muleClient;
    private MuleContext muleContext;

    /**
     * {@inheritDoc}
     */
    public MuleClient getClient() {
        if (muleClient == null) {
            try {
                DefaultMuleContextFactory mcfactory = new DefaultMuleContextFactory();
                muleContext = mcfactory.createMuleContext(MULE_CONFIGS);

                muleContext.start();
                muleClient = new MuleClient(muleContext);
            } catch (InitialisationException e) {
                throw new GeneralMuleException();
            } catch (ConfigurationException e) {
                throw new GeneralMuleException();
            } catch (MuleException e) {
                throw new GeneralMuleException();
            }
        }

        return muleClient;
    }

    /**
     * {@inheritDoc}
     */
    public MuleMessage createMuleMessage(Object payload) {
        return new DefaultMuleMessage(payload, muleContext);
    }
}

The add output Mule config file calls the addoutput service on pvoutput indirectly through TcpTrace (What is TcpTrace). The service is post and the mime type has been set to represent form data. Without that you may find unusual data sent…
In the post request, I’ve used message property transformers to add in the pvoutput apikey and systemId. These will appear in the header when sent.
pvoutput-addoutput.xml

<mule xmlns="http://www.mulesoft.org/schema/mule/core" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xmlns:spring="http://www.springframework.org/schema/beans" xmlns:http="http://www.mulesoft.org/schema/mule/http"
	xmlns:vm="http://www.mulesoft.org/schema/mule/vm"
	xsi:schemaLocation="
       http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.0.xsd
       http://www.mulesoft.org/schema/mule/core http://www.mulesoft.org/schema/mule/core/3.2/mule.xsd
       http://www.mulesoft.org/schema/mule/http http://www.mulesoft.org/schema/mule/http/3.2/mule-http.xsd
       http://www.mulesoft.org/schema/mule/vm http://www.mulesoft.org/schema/mule/vm/3.2/mule-vm.xsd">
       
	<flow name="addOutputFlow">
		<vm:inbound-endpoint path="addoutput" 
							 exchange-pattern="request-response"
							 connector-ref="pvoutputConnector" />
		<http:outbound-endpoint address="http://localhost:40000/service/r2/addoutput.jsp"
								method="POST" 
								connector-ref="HttpConnector" 
								exchange-pattern="request-response"
								followRedirects="false"
								mimeType="application/x-www-form-urlencoded">
			<message-properties-transformer>
				<add-message-property key="X-Pvoutput-Apikey" value="${pvoutput.api.key}" />
				<add-message-property key="X-Pvoutput-SystemId" value="${pvoutput.system.id}" />
			</message-properties-transformer>
		</http:outbound-endpoint>
	</flow>
	
</mule>

The same message transformation is applied to the get missing service request. Also for this service, the url has to change to get the dates of outputs not added from the system install date to today. For that I’ve used message property EL selection.
pvoutput-getmissing.xml

<mule xmlns="http://www.mulesoft.org/schema/mule/core" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xmlns:spring="http://www.springframework.org/schema/beans" xmlns:http="http://www.mulesoft.org/schema/mule/http"
	xmlns:vm="http://www.mulesoft.org/schema/mule/vm"
	xsi:schemaLocation="
       http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.0.xsd
       http://www.mulesoft.org/schema/mule/core http://www.mulesoft.org/schema/mule/core/3.2/mule.xsd
       http://www.mulesoft.org/schema/mule/http http://www.mulesoft.org/schema/mule/http/3.2/mule-http.xsd
       http://www.mulesoft.org/schema/mule/vm http://www.mulesoft.org/schema/mule/vm/3.2/mule-vm.xsd">

	<flow name="getMissingFlow">
		<vm:inbound-endpoint ref="GetMissingEndpoint" />
		<http:outbound-endpoint address="http://localhost:40000/service/r1/getmissing.jsp?df=#[header:INBOUND:df]&amp;dt=#[header:INBOUND:dt]"
								method="GET" 
								connector-ref="HttpConnector" 
								exchange-pattern="request-response"
								followRedirects="false">
			<message-properties-transformer>
				<add-message-property key="X-Pvoutput-Apikey" value="${pvoutput.api.key}" />
				<add-message-property key="X-Pvoutput-SystemId" value="${pvoutput.system.id}" />
			</message-properties-transformer>
		</http:outbound-endpoint>
	</flow>
	
</mule>

Complete Code

Can be found here: http://blog.v-s-f.co.uk/2012/01/pvoutput-integration-v1-code/