Add Absolute Value Operation to Process List

I have the need of use absolute values of some operations of the inputs.

Sometimes I am receiving rubbish values and I need to filter it. I know that I can use Node-Red for extra pre-process or post-process, but I think that it is better add an extra operation to process list.

The flow would be the following:

If abs(input -feed) < limit {
	
	Log to feed
	
}else{

	Reset to Zero
	+feed
	Log to feed

}

This flow prevents the values outside the limit range and uses the last good value logged in the feed.

This flow with an abs operation is more efficent than generate a conditional structure using the available operations.

Somebody knows another way to do it?
Does it make sense what I say?
Does it make sense to include it in the processlist of emon?

Yes, I agree that emoncms should have more process/functions. There are some things I wanted to be able to do, and found that the standard set of functions available in Input processing were not quite flexible enough, or in one or two instances, a perfectly sensible calculation was impossible to perform.

I have a similar process function to yours, one of several I have written. It is called “Limit to n”, with n being a user-set value. It replaces the current value with “n”, if the value is higher. It was quite possible to do this with several standard functions, but boy, it was clumsy! A single one is more efficient.

Another one is “Now”, which forwards the current time to the next process.

A third is “Store to n”, which saves the current value to an array variable, the user providing the index “n”. There is a corresponding “Recall from n”, though I haven’t needed it yet.

I needed the Store to allow my “Log to Rfeed” to retrieve a duration from earlier in the processlist while still letting it receive the current value (Watts). It needs the duration because its speciality over the standard one is updating earlier datapoints with the newly calculated Watts value, and the duration tells it how far back in the feed to go. The “Log to Rfeed” expects to see the Stored duration in Store, using its own feed id as the index. This is designed for pulse-based data, where the value for earlier datapoints could not be calculated until the next pulse arrived. I use this in my gas processlist.

“Inverse” just replaces the value with 1/value, so I could then use the standard “x” with a constant C set by the user, to effectively divide that value by the one that Inverse had received. Ie, 1 / V * C is equivalent to C / V. Trying to do that with the standard functions was doing my head in. It could be done, but again it was going to rely on feeds that I would create for no other purpose than to permit a calculation using only the most recent value.

Another useful one is “Difference n”, which is a feedless version of “Rate of Change”. It seemed silly and wasteful, especially in a low-write system, to use a continually written-to feed just to compare to the previous value, and never any earlier value. Again “n” is a user-set index to another array variable, as long as it is unique in all of the processlists, you can use it as often as you like. In conjunction with “Now”, I use it to derive the duration between the current increment in pulsecount and its previous increment (in my system, I never see more than one pulse in 32 seconds, so any increment is always 1). Without “Now”, there is no way to derive a duration.

By the way, just to be clear I am not trying to badmouth Emoncms at all, I am solidly impressed with its capabilities, and with the depth of thinking that has clearly gone into it. But it isn’t perfect. The process functions can be made more flexible than they are, and means exist within emoncms to allow for new functions to be written and added, and that is itself testament to forward-thinking.

My functions are all working, but I am aware that some of them need some sanitation applied before they are ready for the emoncms world. They were all written over the last few weeks, and some are a bit raw. Maybe in a couple of days or so I can post some.

Below is a screenshot I took yesterday, trimmed and edited for clarity. It had some comments but they were essentialy the same as my post above.

Hope this image thing works OK, now.

My Arduino with emonTxShield provides two pulse-based data streams, the standard continually incrementing pulsecount, and a custom stream called pulsegap, which is the duration in seconds between the current pulse, and the previous one.

I had been concentrating my efforts on the pulsegap processes, and once they were working, I wanted to see if I could acheive the same results with pulsecount, now that I know more about emoncms than I did when I started. The GOTO jumps to step 9 in the pulsecount list. The “n” in “Store to n” is the feed id of Log to Rfeed. The "n"s for the “Difference n” are just two unrelated unique indexes for Store to use internally for its array variable.

Pulsegap is limited to 7200 seconds by the ISR in the Shield/Arduino, but I limit it to 10 minutes (600s) in the processlist. It is that limited duration that I want “Log to Rfeed” to use, rather than pulsegap itself.

I am totally agree with you. EmonCMS needs to expand the number of available process/functions.

One month ago I tried to push an aditional function called “Reset to One” process as a opposite of the already available “Reset to Zero” process.

See Pull Request #484 on GitHub

Your proposed functions are very useful!

Just found a need for another function to be available for process lists.

If the pulse increment found by “Difference n” is larger than 1, I need to multiply my Joules or Watts figure by that number, so basically I need to multiply a current value by a Stored value, which is not a fixed value. In a Low-write system, I don’t want to keep a feed just to hold intermediates in a calculation, so, since I already have a “Store to n” facility, I just Store the pulse increment in the Store’s array with a unique index, and have a new “Scale n” where n is the index to the array, instead of being the operand itself. The function just multiplies the current value by the recalled value, and passes the result forward.

Naturally, for flexibility, there ought to be similar functions for “Offset n”, “Divide n” and so on. In effect we are implementing the “Memory” functions from the early days of calculators. After all, the process list is essentially a program written using a series of selectable functions from a menu. Adding variables opens up a lot of possibilities.