Thank will try when I get home. Stuck in traffic now and will test.
I’m willing to learn new techniques, if emonhub is the way to go, please assist. In setting it up. Im not sure where to start.
I appreciate it.
It’s a fake api key, I’ve been changing the characters in it.
Still getting same Error @pb66,
/emoncms/input/post.json?apikey=43b84401f6dad8a43effxz0ec754716a1&node=10&csv=1, 40, 30, 50, 26214
Error: Format error: csv value is not numeric
/emoncms/input/post.json?apikey=43b84401f6dad8a1056gfdxec754716a1&node=10&csv=1, 40, 30, 50, 26214
Error: Format error: csv value is not numeric
^CTraceback (most recent call last):
File "read_register_egx.py", line 80, in <module>
time.sleep(3)
KeyboardInterrupt
emoncms@SKUbuntu:~/pyModbusTCP/examples$
I would like to checkout emonhub Paul, not much documentation is available, can you guide me in the right direction?
EDIT: Whats odd, is I copied and pasted the url python used to send the data
http://localhost/emoncms/input/post.json?apikey=43b84401f6dad8a1056gfdxec754716a1&node=10&csv=1, 40, 30, 50, 26214
and it worked, it posted the data, and works as expected, could it be the httplib in python? doesnt covert the spaces to %20? or not sure why it causes issue. I used firefox for this, works as expected. But i would like to persue emonhub instead.
I will take a look at sending to a socket for you tomorrow, there is a guide to installing emonhub within the emoncms install docs, but most of that will not apply to you as you are not using serial or RFM, and using sockets isn’t covered there.
Try searching the forums (incl the old forum) for “Socket Interfacer” or I will find some links for you tomorrow.
In the mean time if you want to try using “join” on the CSV you may be able to confirm your suspicions, as it should provide a space free CSV string, I was pretty confident emoncms wasn’t fussy about the spaces and your use of the url in a browser seems to confirm that.
#Send to emoncms
CSV = ",".join(regs)
URL = "/"+emoncmspath+"/input/post.json?apikey="+apikey+"&node="+str(nodeid)+"&csv="+CSV
print(URL)
conn.request("GET", URL)
response = conn.getresponse()
print response.read()
1 Like
Thx @pb66 I will look through forums on emonhub socket interfacer.
as for python another errror came up. I am using python 2.7, not sure if that makes a difference or not. I know py2 and py3 syntax for the most part the same, but very picky if you run py2 scripts in py3.
emoncms@SKUbuntu:~/pyModbusTCP/examples$ python read_register_egx.py
Traceback (most recent call last):
File "read_register_egx.py", line 67, in <module>
CSV = ",".join(regs)
TypeError: sequence item 0: expected string, int found
skopytin@SKUbuntu:~/pyModbusTCP/examples$
EDIT: fixed it with this.
CSV = ','.join(str(v) for v in regs)
and does update, and now my output is missing the spaces, inbetween values and is updating emoncms correctly.
I haven’t made the switch, i still use 2.7 by default and without thinking about it, had assumed your code to be 2.7 by the lack of brackets on the print statement.
That makes total sense, my apologies.
So it is actually the “httplib” module that creates an issue when there are spaces in the csv rather than emoncms, that too makes better sense, not saying it’s right or not, but I have spaced out csv for better readability in the past and hadn’t hit that issue with emoncms before.
Later today i will look at editing a copy of your script for use with an emonhub socket interfacer.
EDIT - If you do go ahead and install emonhub, I would install the original not the emonPi variant for this, and maybe be aware that on Ubuntu there maybe an issue with the emonhub init service under systemd (ref Emonhub Systemd Ubuntu 15.10 thread)
EDIT2 - I had found this PDF yesterday, I will post it here for ref (so I can close the browser page) it lists the full list of registers.
http://www.veris.com/docs/comms/mb_pmap/h663_h704_100a_pm_10101.pdf
EDIT3 - I forgot to ask something about the device, That PDF says “100 amp model”, is that the overall or per channel? Just I noticed the registers were “16bit Integers” and the measure is mA, giving a range of less than +/- 33 amps, not sure why it would report negative amps either, are they “16bit unsigned integers” ie 65amps. (Just thinking out loud to myself, I’m sure it’s documented somewhere)
Your problem is the spaces. Remove them and I think it will be fine.
Thx Paul, I’m running into another issue, and that is overwhelming the Modbus TCP/IP device with DDOS attack essentialy, reqeusting all 42 registers at one time, and python freaks out as well, and I can’t pull more than 15 registers at one time before the modbus device freaks out which is normal, on other automation systems that i have done, we poll a modbus register, only 1 register every 1 second.
Before I build timers and delays, etc. in the script, I’ll wait for your script for emon hub, thx for the help.
EDIT: I have installed emonhub, and I ran into the ubuntu issue, and i left to figure it out for today.
EDIT 3: the veris meters I have are 50 amp CT’s, and I used unsigned 16 bit to get it to work correctly in my previous integrations.
The board i’m sure is capable of aquiring 100 amps CT’s, infact that is the acquisition board I have but I only have 50amp CT’s, and not sure it goes about handling 100amp CT’s,
Yep, Firefox didnt give me an issue when I used it to send data, but something about httplib in python module, doesn’t handle spaces correctly. removing them solved the sending issue, but now I got another issue traffic managing the communications.
@42 registers that would take 7 mins to cycle through, that seems like a really slow update interval.
With 275 registers available that would take over 45mins to do a full read of every register, I’m a real novice when it comes to modbus, but that doesn’t sound right to me. I can see there are warnings and alarms in the register map, I wouldn’t expect a warning or alarm to be of much use if it could take up to 45mins to report.
It’s easy enough to poll one at a time I guess, but surely that adds to the devices workload? perhaps smaller batches would work, are there no guidelines or proper practice for this in the modbus protocol?
It’s dependant on the instrument, rather than the protocol itself. e.g. all of the registers in the Elkor WattsOn can be read in less than one second at 9600 bps.
The H663 install guide specs (page 8) show the current measurement input update rate to be 1.2 seconds. (The H704 specs read the same, i.e an update rate of 1.2 seconds.)
With such a slow update rate, it doesn’t make much sense querying the instrument more often that that, does it?
Paul, I fat fingered the poll time. It’s 1 register per 1 second, sorry for the confusion.
i would assume that to be each input is updated every 1.2s, rather than each input taking 1.2s in turn therefore 42x1.2s = 50.4secs before the same input is updated again. In which case ideally, all 42 registers could/should be read every 1.2s to read every update and avoid redundant updates.
Even at 1 register per 1 second, and ignoring all the warning and alarm registers, 42 registers = 42secs to cycle back to the first register, in which time it has been updated and changed 35 times @1.2s, so only 1 in 35 results for each input is ever seen, 34 are wasted, that’s over 97% of the data posted to those 42 registers is redundant.
I’m still not convinced that’s right, but you guys have way more experience than me with this.
I suspect the devices registers should be able to be read entirely (not just the 42 values) in around 1.2s other wise it would be pointless updating that frequently.
Maybe the issue is with the pyModbusTCP lib? From what little I know, I would expect a connection to be opened and the registers read sequentially, perhaps even staying open indefinitely for the next cycle and ongoing, for a “denial of service attack” to be perceived the lib or the way it is being used must be instigating individual polls, which I wouldn’t expect to be efficient or healthy for the server or the client.
This example from the pyModbusTCP documentation examples creates a separate thread for polling Modbus and updates every 1s, granted its only 10 regs but that is still 10x faster than 1 per 1s.
Yes, my point exactly.
Perhaps not. If, for example, the registers that need to be read are non-contiguous, i.e. not in sequential order, then individual polls are the only way to read them. That’s how I have do do it with the WattsOn power transducers I have.
I learned about the WattsOn’s ability to have all of its registers (about 60 of them, if memory serves) read in under one second when I read about setting the seral interface data rate. The factory default rate is 9600 bps, but it can be changed to 57600 bps if desired. The manual recommended leaving the data rate at 9600 because all of the registers could be scanned in under one second at 9600 bps.
Looks like what’s needed is more info about the instrument, i.e. what’s the max read rate for the registers? Would running the serial interface at a higher data rate help?
From what I remember about using Modbus from many years ago, the most economical way is to read as much as possible in one block, then pick out what you need from that and ignore the remainder. That method has a much lower overhead. I think I was reading about half a dozen values (what size I can’t remember) and graphing at well under 1 s intervals, with a Compaq 386 PC at I think 9600 baud.
Here is an example script that posts values 0-9 every 5 seconds, I’ll happily help you merge the two, but I’m not sure how you want to stage the data based on the discussion above, will you want to simply forward every reg as it is read “hot potato” style or will you collate the data into batches or build up a single complete frame.
The best way IMO would to be send one complete packet.
#!/usr/bin/env python
import time
import socket
host = "localhost"
port = 50012
nodeid = 13
interval = 5 # in seconds
# Send the frame of data via a socket
def send(frame):
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((host, port))
frame = frame + '\r\n'
s.send(frame)
s.close()
# Create a "lastsent" time to count interval from
# this is done by dividing unixtime by the interval
# so the first send may be delayed by upto 1 interval.
lastsent = time.time()//interval*interval
while True:
# Determine current interval period
t = time.time()//interval*interval
# Check for 1 interval period since lastsent
if t >= (lastsent + interval):
##### DATA STAGED HERE eg
list = [0,1,2,3,4,5,6,7,8,9]
# Create a space seperated frame, timestamp nodeid val1 val2 etc
frame = ' '.join(str(val) for val in [t, nodeid] + list)
# Update time lastsent
lastsent = t
# Print and/or send data
print(frame)
send(frame)
# Don't loop too fast
time.sleep(0.10)
At emonHub you only need one interfacer set up as follows, plus the emoncms reporter will need your apikey and url (the default RFM2Pi interfacer can be deleted)
[interfacers]
[[Veris]]
Type = EmonHubSocketInterfacer
[[[init_settings]]]
port_nb = 50012
[[[runtimesettings]]]
timestamped = true
I’ve lifted this from one of my own scripts and I have adopted a new way of synchronizing my data packets using unixtime divided by interval, hopefully it makes sense, I’ve added comments, but you can use something more conventional if you prefer.
1 Like
That’s what I had expected.
Thx Paul,
I have stumbled upon node-red, and does seem to do what i am looking for, I am running into an issue with node-red not being able to read float values, but that’s another issue not related to emoncms,
I will keep this thread on the back burner, thx again for your replies.
Hello.
I recently got some help from a couple of people at the pymodbus google group, to make a working script in python for posting to emoncms.org. I find this thread now!
But the script I’ve had help finding and modifying is capable of taking alot of modbus writes. It puts the data in a queue, and then the queue can be read out any time. So it’s designed for high speed.
Here’s the original page for the code.
https://pymodbus.readthedocs.io/en/v1.3.2/examples/callback-server.html
Here’s my implementation. I’ve skipped queuing, but the functionality could included.
It’s set up to post using http.
server with callbacks.zip (4.5 KB)
The only thing I think is important to know when writing to this server, is that each write ‘address’ is presented as a different ‘key’ in emoncms. So write to different modbus ‘addresses’ for each different datatype; voltage, current, Wh etc… Which is the way it’s done anyway?
Cheers
1 Like