This is an important issue. I did commercial scale data systems for decades. I saw things go from mainframes, to minis, to desktops, and now back to mainframes (cloud servers). I’ve seen many a well designed versatile software system fall under it’s own weight by trying to be what it is not and be all things to all people. This may sound like a rant, but it’s time to stop and think about what this eMonCMS thing is.
The current design is elegant and flexible. It accepts data in real time, or a stream of historical data in chronological order, and stores that information for later retrieval. If I read the history correctly, the data structures have evolved to leverage rapid retrieval over a broad spectrum of the timeline.
I appreciate the problems related here with correcting inaccuracies in the historical data, but my sense is that comes with a cost to the integrity of the design, the performance of the system. It may seem like an oversight, a bug, but it isn’t. It’s the fundamental design that precludes using it as a database that can be updated historically. The only oversight was not documenting the perhaps unintended consequence as the storage engines evolved to their present emphasis on retrieval.
A few weeks ago I had an interaction with Trystan on basically this same topic. He told me historical update doesn’t work, and after thinking about that for awhile, I’m fine with that. I’ve got a storage engine on the IoTaWatt that tries to accomplish pretty much the same things as eMonCMS. Likewise, changing history is a huge can of worms.
If there is a source of more accurate data than what was originally posted, then the feed can be deleted and the data sent again, in chronological order, to rebuild the feed with the corrected information. In the end, a simple do-over may prove easier for all involved.
This isn’t normal usage and I question whether it needs to be accommodated.