Battery time blog post

content/blog/2022-09-30-solar-installed.gmi

@@ -0,0 +1,67 @@
+---
+title: "Solar Installed"
+---
+
+Well, almost. Yesterday they put the scaffolding up, delivered the panels, and got to putting them on the roof. Six are up there, but the remaining three are sheltering in the garage because they didn't bring enough of the railings that connect panels to roof.
+
+Of course they didn't.
+
+The scaffold is now up until at least Wednesday, which is the first day they can come back with more railing to complete the job.
+
+The six panels are, however, functional, and comprise 2.4kWp of generation capacity. Despite only being there from 2pm, they generated ~3.4kWh yesterday. Today was foggy and overcast, but they managed another ~1.4kWh. Tomorrow and Sunday are looking pretty sunny, but October is hardly peak generation time. Still, according to PVCalc, I can expect an average of 5.23kWh/day in October once the remaining panels are up - nearly half our average daily usage. This lines up with the generation projection the solar survey came up with too.
+
+=> https://joint-research-centre.ec.europa.eu/pvgis-photovoltaic-geographical-information-system/getting-started-pvgis/api-non-interactive-service_en PVCalc
+
+Using the generated energy is the challenge. Of that 3.4kWh, I exported 2kWh - over half. Since there are no batteries right now, I need electricity loads in the house to soak up the solar as it's being generated, or some means of storing it. Energy in the UK is bucketed into half-hour billing periods, so generation and use don't have to match exactly, but it's frustrating to see a big energy use peak right next to a bucket where 0kWh was imported.
+
+## Regrets
+
+One of the electricity loads I have is an "iBoost", which is meant to turn on the immersion heater, bypassing the heat pump entirely, if there's excess generation. Charging the thermal store (hot water cylinder) at a CoP of 1 is pretty rubbish, but - economically speaking - better than exporting to the grid if you're getting paid peanuts to do so. For me, the device is pointless, but it's installed. At least I can turn it off.
+
+It seems that there might be room for fifteen panels on the roof, but the installers are refusing to install capacity over the G98 limit of 3.68kWp, even if the export is inverter-limited to that amount. So the scaffold will probably be going back up next year, but adding six, rather than three, panels, makes it more worthwhile. In retrospect, there was time to get the G99 application in to have more installed in a single go - although permission is not guaranteed, so perhaps this was the correct route after all.
+
+It's also occurred to me that my shiny new heat pump is plumbed in wrongly for a house with solar PV. Leaving aside the fact that it would have been wise to get solar thermal as well, it would be better if the heat pump exclusively heated the hot water cylinder, and the cylinder was used to drive the central heating as well as domestic hot water. This would allow me to avoid switching the pump on at night, when there is no solar, but would probably have required the cylinder to be much larger than it is. It's all good learning for the next house, though.
+
+## Buffering
+
+The cylinder is a thermal store, but not a large one, and not plumbed for maximum utility. However, the "grid as battery" idea that I talked about in "Off-grid in Suburbia" is looking better and better. I'm switched over to Octopus, from E.on (who were giving 3p/kWh for export). Once all the paperwork is done, I can either choose a guaranteed 15p/kWh (just increased!), or use "Agile Outgoing", with prices ranging anywhere from nothing to > 60p/kWh, depending on demand and time of use.
+
+=> /blog/off-grid-in-suburbia/ Off-grid in Suburbia
+=> https://www.energy-stats.uk/octopus-agile-outgoing-export-yorkshire/ Octopus export tariff prices
+
+Since I can't shift my generation (via batteries) to the highest-priced times, the fixed rate is probably the right one for me. It's still 5x the price my previous supplier was offering, and is high enough to obsolete the iBoost. The heat pump has a CoP of ~3.5; when it's running on imported energy (~33p/kWh), this means each kWh of heat costs ~9.4p. The iBoost, running on generated solar with CoP=1, has a notional cost of 15p/kWh, so using it is the same as throwing away ~5.6p/kWh. 
+
+Batteries can be added next year, but definitely not this year. I'll probably attempt to DIY it on the DC side; the batteries can then discharge when there isn't any solar, or when I want to export, through the existing inverter, without DNO involvement. I need to start being more hands-on with this stuff. Only without getting electrocuted.
+
+Buffering is about moving the energy to time of use; taking the same problem from the opposite side, we can move the time of use instead, which leads to:
+
+## Reasonable Home Automation
+
+It feels like that house is far more connected to the elements now it has solar panels. Suddenly, it's beneficial to take local conditions into account when using energy.
+
+For example, the heat pump is currently programmed to heat the cylinder between 12-1pm each day. When running directly off of solar, we can divide the export price by the CoP to arrive at a notional cost: 4.2p/kWh - less than half the cost of running it off imported energy. If it's cloudy at midday but sunshine is forecast for 4pm, it makes sense to delay running the pump until then, so it can take advantage.
+
+Similarly, we have "set-back" crudely configured into the thermostat at the moment - the temperature is allowed to drop from 21'C to 18'C between 9pm and 7am. These are very arbitrary, though, and I often find the pump is running at 8:30pm because temperatures have dropped to 20.8'C or so. Having the thermostat take sunrise and sunset into account, and having it progressively ramp setback up and down rather than it being a cliff edge, makes sense. So does allowing the temperature to drift a little (above or below 21'C) depending on current and predicted solar availability.
+
+There are more conventional examples too, like running conventional appliances while it's sunny, but for 4-6 months of the year, the heat pump is the single biggest energy user, so I'm focusing on that. (The second-largest appears to be cooking, which I'm ignoring for now, but changing the time you cook dinner is entirely feasible without any automation).
+
+All this is solved with "smart home" gadgetry, but sadly, those gadgets are of the "upload everything to the cloud" variety. I'm dipping my toes into more-reasonable approaches, and have the bones of a plan:
+
+=> https://shop.electrolama.com/collections/usb-rf-sticks/products/zzh-multiprotocol-rf-stick ZigBee zhh coordinator module in the home server
+=> https://shimmeringbee.io/ ShimmeringBee coordinator software (ZigBee + MQTT) running on the home server
+=> https://www.develcoproducts.com/products/meter-interfaces/external-meter-interface/ ZigBee pulse counter on the generation meter
+=> https://shop.glowmarkt.com/products/display-and-cad-combined-for-smart-meter-customers Glow "CAD" add-on to get (local, not cloud) MQTT feeds for the import + export meter
+=> https://github.com/aerona-chofu-ashp/modbus Scrape the heat pump's Modbus interface and get it into ShimmeringBee somehow
+
+That should be enough to get me the data I need to calculate scenarios like the ones above. "All" I have to do then is enrich it against internet-available data (weather predictions, say), make decisions automatically, with the potential for some human input too, and have them be implemented by hardware.
+
+These things need coordinator software. ShimmeringBee is a bit of a find, being a tiny Go daemon without any kind of UI. Trivial to deploy, trivial to modify, I glommed onto it as soon as I found it.
+
+Lots of other options, but I'm going to stick with it to begin with, even though I'm going to have to write code to get most of the items above talking to it. At least the zzh coordinator works out of the box. It has an MQTT port open too, so hopefully the import/export meter will work fine. The decision code can just go straight into the daemon, I think.
+
+=> https://ubidots.com/blog/open-source-home-automation/ Other options
+
+I don't have hardware for implementing decisions planned out yet. So much of it is going to be around the heating system. A thermostat I can control from ShimmeringBee will be the first requirement, possibly along with ZigBee-connected TRVs, so rooms can be heated or not individually.
+
+It's a techy dream (nightmare?), but the underlying goal is to get the house using less energy. Of course, all this tech *uses* energy, but I'll be doing my best to keep that aspect under control, mostly through careful selection of low-power components and software. At least the glue is code I'm looking forward to writing!
+

content/blog/2022-10-10-battery-time.gmi

@@ -0,0 +1,81 @@
+---
+title: "Battery time"
+---
+
+In my last post I talked a little bit about the newly installed solar panels, and a lot about home automation. Since then, the solar install has been completed, although the scaffolding is still up (grumble grumble). I have nine panels on the roof, and today was a new generation record - 17.1kWh, with 4kWh of import from the grid!
+
+=> /blog/solar-installed/
+
+Of course, we didn't use 21.1kWh overall; instead, we exported a large amount of it. For free, since I don't yet have the paperwork required to sign up to an export tariff.
+
+## Reasonable Home Automation (cont)
+
+The sensors part of the home automation project is going pretty well; I have data from:
+
+* Air quality sensor via ZigBee zhh adapter
+* Glow CAD
+* Solax DC->AC solar inverter
+
+All of this is being fed into MQTT (the mosquitto server), then exported into victoria-metrics via mqtt2prometheus. The CAD speaks MQTT natively; the air quality sensor is currently being filtered through zigbee2mqtt on the laptop while ShimmeringBee gets support for that specific device; and the Solax inverter is being scraped via some custom code I put together. More on that shortly.
+
+With the time series in victoria metrics, I can generate pretty dashboards in grafana at will:
+
+=> /img/battery-time/power-dashboard-1.png Power Dashboard part 1
+=> /img/battery-time/power-dashboard-2.png Power Dashboard part 2
+=> /img/battery-time/sensors-dashboard.png Air Quality Sensor dashboard
+
+
+(The spike on that last one is me breathing into the sensor to prove it worked.)
+
+## Solax inverter deep dive
+
+Being able to scrape the inverter was a bit of a surprise - the plan was a Zigbee pulse counter on the generation meter. When the installers put the inverter in, however, they equipped it with a hideously insecure and hostile "PocketWIFI" module, which acts as both a WIFI AP and client. The AP is unsecured and cannot be turned off, but the client can at least do WPA2-PSK, and is connected to a WIFI Home Automation Network I've set up with no internet.
+
+Either way, once you've got an IP address for the adapter, you can query it over HTTP and get data about the inverter it's plugged into. It's also listening on port 502 (Modbus-TCP), although I've not managed to get it to respond to anything mbpoll can send. Still, it's slightly scary to think that anyone in the vicinity of my house can speak modbus directly to the inverter by connecting to the unsecured AP.
+
+If this is what they put in someone's house when they explicitly request Luddite mode, I can't imagine what "full cloud connectivity mode" looks like.
+
+The adapter also tries to connect to an MQTTS (TLS, port 8883) server in The Cloud. This is used for aforementioned cloud connectivity and offers exciting features like "push arbitrary firmware to the inverter/adapter", alongside "view your inverter data with a 5-minute delay". Of course, they're not verifying certificates, so the MQTTS connection is trivial to hijack for harvesting data - or pushing said arbitrary firmware.
+
+Programmes like "Leverage" present hackers as modern-day wizards, able to do anything with any computer, take over any system, and exert absolute control over network-connected devices. It seems far-fetched until you run into actual real-world systems like this...
+
+Anyway, the plan is to replace the "PocketWIFI" module - either with a direct Modbus-RTU connection, as with the heat pump, or with a "PocketLAN" device if that proves infeasible. Both are significantly more secure, since they require physical access to take over the device.
+
+Meanwhile, the MQTTS stream proved to be inconveniently formatted for use with matt2prometheus, since it pushes binary data rather than JSON. I removed the interceptor and knocked together a tiny bit of code that scrapes the HTTP endpoint instead.
+
+This gives a JSON response that contains a 100-element array of uint16 values, which is the sensor/status data. There's a lot of Solax projects and integrations already that had made headway on interpreting this in general, but none for my specific inverter/firmware combination, so I got to do some detective work, helped along enormously by the squishykid/solax GitHub project. The result is solax2mqtt, a small (ca. 250LOC) Rust program that does the HTTP scrape, transform, and MQTT push, all in one.
+
+=> https://github.com/squishykid/solax/issues/93 squishykid/solax issue 93
+=> https://code.ur.gs/lupine/solax2mqtt Solax->MQTT project
+
+It's been a while since I've written anything in Rust, and this was an easy project, but I enjoyed it greatly and managed to get into a tight loop with the compiler to build out the whole thing without once running it against the actual inverter. Once the compiler was happy, and all my TODOs had been implemented, I did that and it worked first time, with no bugs, which was a great feeling. I should do more in Rust.
+
+Despite all the data, I don't yet have a reliable export number. I do have live export in watts, but not accumulated kWh, despite both the inverter and glow CAD being able to provide it in theory. Perhaps it will start working once I'm signed up to an export tariff.
+
+## Batteries
+
+Anyway, the sensor data makes two things very clear to me:
+
+* I should install MVHR
+* I should install a battery system
+
+The first point will be the subject of another article. The second follows from the energy stats, but there are a few other considerations.
+
+First, sizing, efficiency, and pricing - I'm looking at a 3.3kWh system (DIY-expandable in 3.3kWh modules), which will have ~90-95% efficiency and cost around £3750, fully installed. This has to compete with the grid, which, per my previous post, has infinite capacity and is free, but only has ~50% efficiency when round-tripping through money at current unit prices.
+
+My original plan was to leave thinking about batteries until next year, and only to do it in conjunction with a larger solar array. Since then, the UK has started talking about 3-hour rolling blackouts over winter (perhaps a 10% chance of this happening), and paying people £10/day to avoid using energy from the grid 4pm-7pm (much higher chance of this happening). The former affects me far more than it should - I've spent enough time without electricity, thank you - but the latter is very helpful.
+
+The no-grid bribe from November - February would be £1,200 - and much easier and more pleasant to collect with a battery installed. If it extended into March as well, that would be £1,500 - 40% of the total cost of the battery, in just five months.
+
+The more traditional benefit is that overnight usage can come from the battery, reducing daily import. Assuming I avoid 3kWh/day of import, the saving is £0.50/day compared to the grid-battery option. It really puts the size of that 4pm-7pm bribe into perspective! Paying back the remaining 60% of the battery cost from that would take 12 years - and the battery is only rated to last for ten. Without the grid bribe? 28 years for full payback.
+
+The no-grid scenario exercises me greatly, though. Without the battery system, the solar panels are useless - the inverter has "anti-islanding protection", which prevents line workers from getting fried by shutting the panels down. The house is also much more dependent on electricity now that it's powered by a heat pump (although it's worth noting that gas boilers do need a small amount of energy to function, although it could conceivably be provided with a plug-in UPS).
+
+The batteries we're looking at are AC-coupled and provide an "EPS" circuit - basically a UPS. Most of the time, the circuit is powered by the mains, but the batteries can put 3kW of power into it if the grid goes down. The heat pump, home server + internet + HAN, lights, and a couple of sockets in the kitchen, can be powered from there.
+
+Without solar, 3kW of load would empty the batteries in 1 hour, but the batteries allow the solar to keep working, so there's up to 3kW of power flowing into the system. On a good day, it could run for long as the sun is shining, plus the capacity of the battery. Load is also likely to be less than peak - overnight, including the heat pump set to 18'C, we do about 100W at current temperatures. On a cold winter's day, with the heat pump maintaining 21'C, we might do 500W before taking cooking into account, so the battery covers us for six hours. I suppose we could always turn the heat pump off to extend it further, but I've also had enough of being cold.
+
+So, despite the poor payback calulation, we're still getting them. Being able to function without the grid is more important than having them pay for themselves, even if the grid has a 90% chance of making it through winter unscathed. I look forward to writing more rust to scrape this inverter, too! I'll place the order tomorrow; the plan is to get it all done before November. Since the scaffolding is still up, Switched On won't be getting this contract; instead, I'm giving Big Green Beard a go.
+
+=> https://www.biggreenbeard.co.uk Big Green Beard
+