Logging SolarEdge inverter data using MODBUS over TCP

justinknight · 26 December 2020 18:05

I’m currently trying to get data from our SolarEdge inverter into my local emoncms running on our emonpi. The main reason for doing this is to free up one of the two emonpi CT sensors to record the power being used by our heat pump.

First step was to update the firmware on our SolarEdge 3500 inverter, since older versions don’t support MODBUS over TCP.

I then enabled MODBUS support on the inverter, instructions available here: https://www.solaredge.com/sites/default/files/sunspec-implementation-technical-note.pdf

I’ve added the following to my emonhub config:

[[ModbusTCP]]
    # this interfacer retrieves register information from modbusTCP clients 
    # retrieve register information from modbus TCP documentation for your inverter.
    # Information here is designed for Fronius Symo 3 phase inverter.
    Type = EmonModbusTcpInterfacer
    [[[init_settings]]]
        modbus_IP = 192.168.178.36   # ip address of client to retrieve data from
        modbus_port = 502          # Portclient listens on
    [[[runtimesettings]]]
        # list of names of items being retrieved
#        rName = DC_Current_1,DC_Voltage_1,DC_Current_2,DC_Voltage_2,Power,AC_Current,Frequency,Temperature,Residual_Current,Total_Yield,Daily_Yield
        rName = I_AC_Current,I_AC_Current_SF,I_AC_Power,I_AC_Power_SF,I_AC_VA,I_AC_VA_SF,I_AC_VAR,I_AC_VAR_SF,I_AC_PF,I_AC_PF_SF,I_DC_Current,I_DC_Current_SF,I_DC_Voltage,I_DC_Voltage_SF,I_DC_Power,I_DC_Power_SF,I_Temp_Sink,I_Temp_Sink_SF
        # List of starting registers for items listed above
#        register = 30769,30771,30957,30959,30775,30795,30803,30953,31247,30513,30517
        register = 40072,40076,40084,40085,40088,40089,40090,40091,40092,40093,40097,40098,40099,40100,40101,40102,40104,40107
        # List of # of registers used for each item 
#        nReg = 1,1,1,1,1,1,1,1,1,1,1
        nReg = 1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1
        # Data type for each item above
#        rType = float32,float32,float32,float32,float32,unint32,unint32,float32,float32,uint64,uint64
        rType = uint16,int16,int16,int16,int16,int16,int16,int16,int16,int16,uint16,int16,uint16,int16,int16,int16,int16,int16
        # nodeid used to match with node definition in nodes section below. Can be set to any integer value not previously used.
        nodeId = 27
        # Channel to publish data to should leave as ToEmonCMS
        pubchannels = ToEmonCMS,
        # time in seconds between checks, This is in addition to emonhub_interfacer.run() sleep time of .01
        # use this value to set the frequency of data retrieval from modbus client
        interval = 1

and then a matching node section at the end of the file:

[[27]]
    nodename = inverter
    [[[rx]]]
        names = ac_total_current,ac_current_scale_factor,ac_power_value,ac_power_scale_factor,apparent_power,apparent_power_scale_factor,reactive_power,reactive_power_scale_factor,power_factor,power_factor_scale_factor,dc_current,dc_current_scale_factor,dc_voltage,dc_voltage_scale_factor,dc_power,dc_power_scale_factor,heat_sink_temperature,heat_sink_temperature_scale_factor
        datacodes = H,h,h,h,h,h,h,h,h,h,H,h,H,h,h,h,h,h
        scales = 1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1
        units = A,n,W,n,VA,n,VAR,n,%,n,A,n,V,n,W,n,C,n

I tried leaving units blank for scale factors, but it didn’t like that so I’ve just used ‘n’ for no unit.

Waiting for some sunshine tomorrow to see if I’m getting any sensible values through.

My understanding from this post: Solar PV generation, pulse counter & virtual feeds - #9 by xyleth is that I need to perform the following calculation to get generated power:
I_AC_Power register x 10^I_AC_Power_SF

I don’t see a ^ operator in the input processes, I guess I need to add a custom input process - can anyone could point me in the direction of how to do that?

Also need to work out how to delete the input I accidentally created with my typo (dc_volate_scale factor)

justinknight · 29 December 2020 13:33

All up and running now. I added an input process to apply the scale factor to the power being reported from the solaredge inverter and have moved the CT sensor to monitor the heat pump energy usage.

I now have a nice dashboard showing house and heat pump energy usage, as well as solar:

justinknight · 30 December 2020 10:36

Hit a slight snag where it would occasionally take the value and scale for the power from different times resulting in the calculated power value being out by an order of magnitude or two. Fixed by retrieving the two modbus registers together and unpacking them.

To do this added I_AC_Power_Combined to emonhub.conf, as a uint32 spanning 2 registers:

[[ModbusTCP]]

    # this interfacer retrieves register information from modbusTCP clients 

    # retrieve register information from modbus TCP documentation for your inverter.

    # Information here is designed for Fronius Symo 3 phase inverter.

    Type = EmonModbusTcpInterfacer

    [[[init_settings]]]

        modbus_IP = 192.168.178.36   # ip address of client to retrieve data from

        modbus_port = 502          # Portclient listens on

    [[[runtimesettings]]]

        # list of names of items being retrieved

#        rName = DC_Current_1,DC_Voltage_1,DC_Current_2,DC_Voltage_2,Power,AC_Current,Frequency,Temperature,Residual_Current,Total_Yield,Daily_Yield

        rName = I_AC_Current,I_AC_Current_SF,I_AC_Power,I_AC_Power_SF,I_AC_VA,I_AC_VA_SF,I_AC_VAR,I_AC_VAR_SF,I_AC_PF,I_AC_PF_SF,I_DC_Current,I_DC_Current_SF,I_DC_Voltage,I_DC_Voltage_SF,I_DC_Power,I_DC_Power_SF,I_Temp_Sink,I_Temp_Sink_SF,I_AC_Power_Combined

        # List of starting registers for items listed above

#        register = 30769,30771,30957,30959,30775,30795,30803,30953,31247,30513,30517

        register = 40072,40076,40084,40085,40088,40089,40090,40091,40092,40093,40097,40098,40099,40100,40101,40102,40104,40107,40084

        # List of # of registers used for each item 

#        nReg = 1,1,1,1,1,1,1,1,1,1,1

        nReg = 1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,2

        # Data type for each item above

#        rType = float32,float32,float32,float32,float32,unint32,unint32,float32,float32,uint64,uint64

        rType = uint16,int16,int16,int16,int16,int16,int16,int16,int16,int16,uint16,int16,uint16,int16,int16,int16,int16,int16,uint32

        # nodeid used to match with node definition in nodes section below. Can be set to any integer value not previously used.

        nodeId = 27

        # Channel to publish data to should leave as ToEmonCMS

        pubchannels = ToEmonCMS,

        # time in seconds between checks, This is in addition to emonhub_interfacer.run() sleep time of .01

        # use this value to set the frequency of data retrieval from modbus client

        interval = 1

Added the same to the inverter node with datacode i for 32 bit integer:

[[27]]
    nodename = inverter
    [[[rx]]]
        names = ac_total_current,ac_current_scale_factor,ac_power_value,ac_power_scale_factor,apparent_power,apparent_power_scale_factor,reactive_power,reactive_power_scale_factor,power_factor,power_factor_scale_factor,dc_current,dc_current_scale_factor,dc_voltage,dc_voltage_scale_factor,dc_power,dc_power_scale_factor,heat_sink_temperature,heat_sink_temperature_scale_factor,ac_power_combined
        datacodes = H,h,h,h,h,h,h,h,h,h,H,h,H,h,h,h,h,h,i
        scales = 1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1
        units = A,n,W,n,VA,n,VAR,n,%,n,A,n,V,n,W,n,C,n,n

Then in my Modules/process/process_processlist.php added an input process to unpack the value and scale factor and calculate the power in Watts:

           array(
            "id_num"=>61,
            "name"=>_("value and scale"),
            "short"=>"v+s",
            "argtype"=>ProcessArg::NONE,
            "function"=>"value_and_scale",
            "datafields"=>0,
            "datatype"=>DataType::UNDEFINED,
            "unit"=>"",
            "group"=>_("Calibration"),
            "description"=>_("<p>Unpack combined value and scale factor, applying the scale factor.</p>")
         ),

and the function:

    public function value_and_scale($arg, $time, $value) {
        $scale = $value & 0xFFFF; // Right 16 bits are the scale
        if($scale >= 0x8000) { // Decode twos complement if negative
          $scale = -(($scale ^ 0xFFFF)+1);		
        }
        
        $val = $value >> 16; // Left 16 vits are the value

        // Apply scale factor to the value
        return $val * pow(10, $scale);
    }

Now logging correctly without spikes caused by incorrect scale values being applied, not that there’s much sun to log recently!

alexandrecuer · 30 December 2020 14:52

nice solution

garym999 · 8 May 2021 14:36

Hi
I’m trying to do the same but rapidly getting out of depth. My coding ability is poor and maffs even worse.

My first issue was getting comms with the inverter. Although I had enabled modbusTCP comms on the inverter I did not appreciate that if no modbus comms are instigated within a 2 minute window the inverter will lock you out. So get your emonPi (in my case) polling before you power cycle the inverter. Then the data started to pour

However by collecting the registers one by one, I immediately fell foul of out of sync data vs scaling factors.

Then after finding where process_processlist.php lived (//var/www/emoncms/Modules/process) & using the method in the previous post it works for consecutive registers

Attached is a pdf of the data I’m trying to capture.

SolarEdge modbusTCP.pdf (490.9 KB)

If you look at the pdf you will see that some of the registers are not consecutive. What’s the best way to handle that?

TIA

justinknight · 8 May 2021 16:09

Hi Gary,
I see the problem - since the value and scale factor aren’t concurrent you can’t use the trick of combining them to send together. It works for most of them, but not I_AC_Current and I_AC_Current_SF since there are 3 other 16 bit integers between them.

Maybe someone who understands emonhub better could suggest another way to ensure values from one point at time aren’t combined with scale factors from another?

Only other suggestion I have is to implement some kind of processor that sits between solaredge and emonhub that combines the values then passes them on, possibly using something like this python ModbusTCP library: Welcome to pyModbusTCP’s documentation — pyModbusTCP 0.1.10 documentation

Sorry not to be more help, I’m still a bit of an emonhub noob.

Justin.

garym999 · 8 May 2021 16:33

Cheers Justin.
I’m also struggling with one of the values for which this trick should work, the AC frequency.
For testing I’m pulling the combined data and individual elements…

but when submitted as a feed its value is wrong even though other values such as power seem OK?

justinknight · 8 May 2021 18:46

Hi Gary,
The individual values for AC Frequency and the scale factor look right, 50064 x 10^-3 = 50.064 Hz

I’m not sure why the combined value isn’t being calculated correctly. I_AC_Frequency is a uint16 (unsigned 16 bit integer) rather than an int16 (signed 16 bit integer) but I don’t think that should effect the calculation.

The input value 3281059837 is correct - it’s 50064 left shifted 16 bits plus -3 in two’s complement

Here’s the value and scale function running with the frequency and scale from your example:
Teh Playground! which gives a correct output of 50.064. Weird that the AC_Frequency feed value is showing wrong, especially since the other combined values are calculated correctly. All I can suggest is double check your function is the same as mine, maybe try deleting and recreating the feed?

Justin.

Bill.Thomson · 8 May 2021 21:41

Weird that the AC_Frequency feed value is showing wrong,

The value is passed as an integer, vice a float. To do that and provide accuracy to 2 decimal places,
the value is multiplied by 100 at the source, thus it needs to be divided by 100 at the destination.

In this particular instance, the value has been increased by an additional 10-fold.
i.e. it needs to be divided by 1000 vice 100.

justinknight · 9 May 2021 10:39

Hi Bill,
the amount the value is multiplied at source varies depending on the I_AC_Frequency_SF parameter. So in the example the SF parameter is -3 so the value needs adjusting by 10^-3 (i.e. divided by 1,000). The problem we get is if we send the base value and the scale parameter as separate values sometimes the scale factor from one point in time gets applied to the value from another causing spikes in the data.
The workaround for this is to combine the 16bit value and 16bit scale factor into a 32bit value then unpack them and combine later. The weird thing with Gary’s I_AC_Frequency example is that he has the correct value for the packed value and scale factor but not getting the expected 50.064 in the feed.

Justin.

Bill.Thomson · 9 May 2021 11:15

Does the SF for the I_AC_Frequency value change on its own? i.e. without being commanded to do so?

Robert.Wall · 9 May 2021 11:16

I’ve tried to see what is needed to be done to convert 3277 into 3281059837, i.e. applying the algorithm and comparing the bit patterns - and I can’t see anything that looks remotely plausible.

Robert.Wall · 9 May 2021 11:23

I was looking at the numbers at binary bit level - and comparing the old-fashioned way on a piece of paper.

      3277 = 0000 1100 1100 1101
3281059837 = 1100 0011 1001 0000 1111 1111 1111 1101

The -3

1111 1111 1111 1101

added after the bit shift is clear.

Bill.Thomson · 9 May 2021 11:26

I’m thinking if the I_AC_Frequency SF doesn’t change unless told to do so, why send it at all. Ignore it.
Divide the measurement by the SF and it’s a done deal.

Robert.Wall · 9 May 2021 11:35

That’s probably true for frequency, but not some of the other values (otherwise, why do it that way?). I’d say it’s a straightforward programming bug - possibly a memory overflow or something like that, which has corrupted the value.

Bill.Thomson · 9 May 2021 11:37

I should’ve specified Freq.
Post edited.

After thinking about it for a bit, I’m wondering why would any of the the scale factors change on their own?

garym999 · 12 May 2021 15:51

Thanks for everyones comments.

The scale factors do bounce around because the numbers do eg 50Hz, 50.1Hz, 50.12Hz.

This happens quite a bit with the Power figures towards the end of the day as the power drops.

I’m not sure its the best way to deliver the data but thats what we have to work with from SolarEdge.

Knowing my own abilty, I’m sure there are some fat fingers involved in the code preventing the results I’m expecting. I’m just triming it down to the basics and will report back.

Robert.Wall · 12 May 2021 16:21

So does it send those as 50 × 10⁰, 501 × 10^-1, 5012 × 10^-2 Ye Gods.

Bill.Thomson · 12 May 2021 16:42

That was more or less my reaction too.

I took a look at the SE 3500 inverter docs. (specifically, the Sunspec tech note attached below.
The docs linked to in post 5 of this thread say the same thing)

The Modbus register map says register 40087 sets the scale factor value,
which should mean the SF won’t change unless the value in that register changes.

sunspec-implementation-technical-note.pdf (1.5 MB)

garym999 · 12 May 2021 17:15

Still dont get it. It looks as though the line
$val = $value >> 16; // Left 16 bits are the value
is not working because as others have pointed out on paper it works (3281977341 = 0xC39EFFFD >>16 = C39E = 50078)