New functionality for feed sanitation

Labs
1

Hi there,

As part of several recent developments for emonCMS, I have been working in feed sanitation functionality. I invite people to try it before we can say it is stable. You can find it in the data_checker branch from this repository.

The new things:

  • New input processes (in Limit category)
    • Max. value allowed: If value is greater than max value allowed then the value passed to following process will be the max value allowed
    • Min. value allowed: If value is lower than min value allowed then the value passed to following process will be the min value allowed
  • Feed data checker/fixer: giving starting and end dates, it will check a feed for:
    • Values greater a specific number.
    • Values lower a specific number
    • Missing points (for fixed interval feeds).
      Then you can choose if you want to fix those datapoints. The fix for missing datapoints is basic interpolation.

The feed checker/fixer functionality will work with every feed engine but has been optimized for PHPFINA and PHPTIMESERIES. You can run it clicking on the new icon-check in your feeds view.

Beware that the “fix” will change your feed so use it with care.

Cheers,

Carlos
2 Likes
2

Does this still exist?

Would you please direct me to exactly where it is?

3

If you click on the link, then select the

image

Caveat emptor

I have no idea how to integrate it into the existing code base.

I suggest you fork the project into your own GitHub account, then clone it locally from there.

4

I noticed there where quite a few merge conflicts on @cagabi’s branch when comparing to the latest master branch. I have copied his data_checker branch across to the main emoncms repository under a branch of the same name and fixed the conflicts. I haven’t yet tested the functionality. The data check is only availble in the older feed list interface, so it would be good to get that in feedlist_view_v2.php if this is to be merged.

https://github.com/emoncms/emoncms/compare/data_checker

1 Like