Near Real Time Energy Consumption Data with Octopus Home Mini
Back in 2021, I got a smart meter, not to watch the little in-home display and panic about the watts being consumed (been there 6+ years before!!) but to benefit from some of the smart meter tariffs available. The little display has sat in it’s box* since the meter was installed as its not exactly in the same league as my home dashboard (in my opinion😁)
The smart meter has been fine** but one key difference between the non-smart and smart meter is that the non-smart meter stopped flashing the impulse light when there was excess electricity being exported to the grid. The new smart meter counts export by continuing to flash the impulse light! I use the impulse light flash to count consumption from the grid, which means on a sunny day when there’s lots of excess electricity, the stats are incorrect as export has been counted as import.
So how do I correctly report the figures? I could write some code to check if grid load is less than zero and count any import as export, but there are cases when grid load is -10W and we are still importing, so the figures might be incorrect again. Best option would be to query the energy providers API and pull the data from them.
Luckily, my energy supplier (Octopus Energy) has API endpoints for their customers to query information about their account and the API documentation has an endpoint to get exactly the information I’m after: https://developer.octopus.energy/docs/api/#consumption but after querying it, to my surprise the consumption data is not up to the minute. The nearest consumption data is for the day before but when you’re displaying real time data, it’s not exactly relevant…
After a bit of googling, I stumbled across a little device Octopus Energy are trialling called the Octopus home mini. The home mini talks to the smart meter and uploads near real time data straight to Octopus. I signed up to the waiting list in April (I think it was) and hoped I might get hold of one of these devices in late 2023. At the time I signed up, I wasn’t sure if the data collected by the device would be available via the API, but I thought there’d be a good chance.
Roll on late June and I got an email saying my home mini had been posted 🤩.
A few days later and the home mini had arrived. It arrived in a tiny box with a plug* and cable*, although I wish there was an option to not receive those as I have plenty of plugs and USB micro cables lying around doing nothing.
I was surprised at how small the device was – it’s smaller than the width of my phone and about the same depth.
On the back it tells you the case is made from ocean plastic
Before plugging it in, I took a note of the responses from the consumption endpoint just to compare pre and post plugging the device in. This screenshot was taken on the 30th of June, with the latest data available being the 28th, ignoring the two entries for the 29th.
(p.s. for anyone wondering where that Swagger UI is, I spent a weekend converting the Octopus Energy API documentation into OpenAPI spec ahead of getting the Octopus home mini. It can be found here: https://vls29.github.io/octopus-energy-openapi/)
It was then time to setup the home mini.
The first problem while trying to setup the device was that my Wi-Fi networks broadcast on the 2.4 + 5Ghz frequencies and my phone prefers 5Ghz. After spending a minute trying to tell my phone unsuccessfully to use 2.4Ghz only, I gave up and stopped the 5Ghz broadcast within the UniFi controller. The setup pages didn’t immediately see that change, so I closed the Octopus app and restarted the setup process. Although not shown in the pictures above, the setup failed twice before working on the third attempt. It said the password I’d entered for the Wi-Fi network was wrong but I’m sure I typed in the right password.
After it was setup, I found the home mini was listed in devices under my account but there aren’t any options other than to change the Wi-Fi network it’s using.
Live consumption appeared almost immediately on the Usage tab, and 20 minutes later, there were several imported Wh events.
For those wondering, the house was running solely from solar and battery at the time but there seems to be a small import even when self-powered. I have tried asking Octopus Energy in the past if this is to be expected by Powerwall users but never had a response. Given it’s under 1kWh** a day, it’s not the end of the world. It was far worse on the incompatible smart meter!!
At this point I got a bit curious as there were no options in the app and there was only one button that I could determine on the case, so I wondered how easy it was to open up the case 😬 Turns out it was surprisingly easy! I pushed one fingernail between the seam on a corner and the case split right open.
Interesting 🧐
The large chip in the bottom right is an Espressif ESP32-WROOM-32E and at the top in the centre is a Silicon Labs MGM210PA22JIA2. Given the four-pin debug, I had a feeling nothing would happen (no drivers fetched and device not detected in Windows) if the device was plugged in via USB to a computer and that guess was right. edit: It also looks to my untrained eye as if there’s only 5V and ground, wired up on the USB connector.
Given there was data appearing in the app, I tried the consumption API endpoint again but it kept returning data from two days ago. After a 24 hour wait, it was again returning data from two days ago. But having watched a YouTube video (https://www.youtube-nocookie.com/embed/SgepCAjcQyM) a month before, I knew someone had managed to query an Octopus Energy API to get near real time data, so it was a matter of working out how they had achieved that.
After a good few hours of reading the code and trial+error testing against the GraphQL endpoint, it turns out that the answers are all in https://github.com/BottlecapDave/HomeAssistant-OctopusEnergy/blob/develop/custom_components/octopus_energy/api_client.py. It seems the consumption endpoint only returns the published smart meter data and not the data from the home mini (waiting on confirmation from Octopus Energy query via email) but the GraphQL endpoint can return the data from the home mini.
To get the home mini data back from the GraphQL endpoint, the first thing to do is get a token using your API key:
mutation {
obtainKrakenToken(input: {APIKey: "<API_KEY>"}) {
token
}
}
Just replace
Or as a cURL request:
curl -v \
https://api.octopus.energy/v1/graphql/ \
--data '{ "query": "mutation {obtainKrakenToken(input: {APIKey:\"<API_KEY>\"}) {token}}"}' \
-H "Content-Type: application/json"
The response will look something like this:
Copy the token value as it’s needed for all other calls.
Then you need your Meter GUID, which is not the serial number used on the consumption endpoint (as I found out after an hour wasted). To get the Meter GUID, use the following query on the GraphQL endpoint (or you might be able to find the GUID written on the front of your meter!):
query MyQuery {
account(accountNumber: "<YOUR_ACCOUNT_NUMBER>") {
electricityAgreements(active: true) {
meterPoint {
meters(includeInactive: false) {
smartDevices {
deviceId
}
}
}
}
}
}
Or as a cURL request:
curl -v \
https://api.octopus.energy/v1/graphql/ \
--data '{ "query": "query MyQuery {account(accountNumber: \"<YOUR_ACCOUNT_NUMBER>\") {electricityAgreements(active: true){meterPoint {meters(includeInactive: false) {smartDevices {deviceId}}}}}}"}' \
-H "Content-Type: application/json" \
-H "Authorization: JWT <TOKEN_FROM_OBTAIN_TOKEN_RESPONSE>"
The response will look something like this:
Store the value of deviceId for the next call.
And with the token and [deviceId] GUID, we can now retrieve the near real time data the home mini has been collecting:
{
smartMeterTelemetry(
deviceId: "<YOUR_DEVICE_ID_FROM_PREVIOUS_CALL>"
grouping: TEN_SECONDS
start: "2023-07-01T15:38:00+01:00"
end: "2023-07-02T00:00:00+01:00"
) {
readAt
consumptionDelta
demand
costDelta
consumption
}
}
Or as a cURL request:
curl -v \
https://api.octopus.energy/v1/graphql/ \
--data '{ "query": "{smartMeterTelemetry(deviceId: \"<YOUR_DEVICE_ID_FROM_PREVIOUS_CALL>\" grouping: TEN_SECONDS start: \"2023-07-03T17:02:00+01:00\" end: \"2023-07-04T00:00:00+01:00\") {readAt consumptionDelta demand costDelta consumption}}"}' \
-H "Content-Type: application/json" \
-H "Authorization: JWT <TOKEN_FROM_OBTAIN_TOKEN_RESPONSE>"
The response will look something like this:
Cool 😁
Now I just need to spend some time writing a script to query for the data every 10 seconds and send it to Splunk for graphing.
For anyone interested in swapping to Octopus, with this code: https://share.octopus.energy/pearl-kiwi-951, you can get £50 credit, plus I will get £50 credit for referring you to Octopus 🙂
* E-waste… @OctopusEnergy is there a way to recycle unwanted devices?
** since the one that was incompatible with the Powerwall was taken out) but one thing that it
*** and yes, I appreciate that sounds a bit like I don’t care and can afford to waste 1kWh, money wise, a day. I do care, and I’d prefer it to say no import while self-powered, but perhaps there’s a valid reason as the Powerwall isn’t perfect and it looks like it over and under compensates with how much power it sends to the grid as devices are switched on/off.
Please enable the Disqus feature in order to add comments