HTML code no longer saving in a dashboard

mhamer · 2020-01-07T14:48:46.119Z

I have created many different formatted text labels in a dashboard saved 1 year ago under emoncms v9.8.31 using HTML code such as:

Ground

Although this displays the word Ground as before with a bold style on a yellow background when created in a dashboard in emoncms v10.1.3 , I am unable to save any dashboard that contains this HTML code. Hitting the Changed, press to save button, generates a popup window saying:

ERROR: Could not save Dashboard. Error: Invalid dashboard content, content not saved

Could anyone with more familiarity with HTML code please explain what I am doing wrong or have there been changes to the way emoncms handles HTML code? Moreover, when this error message occurs is it possible to narrow down the exact item in the dashboard that is the cause.

I am running the latest OCT19 SD card image on a Pi v4 and am using a Safari browser in MacOS Mojave

mhamer · 2020-01-09T10:43:52.920Z

After some experimentation, I have established the cause of the problem that I have experienced. The dashboard will not save text that contains some of the extended HTML special characters that exist, in particular, in my case, the non breaking space:  . Although they display correctly on the dashboard when editing, they cannot be saved. A very simple example is:

a)   Item 

This issue is unrelated to the version of emoncms and arose by accident related to the order in which my desired end result was reached. The result that I was trying to achieve was a word on a coloured background whose colour extended a single space beyond the letters of the word itself. When I created my original text labels in v9.8.31 I did not have the spaces surrounding the text, the word Ground in this case. The dashboard would save. I subsequently added a space at either end of the word to extend the background colour and the dashboard again saved. However, when the spaces are included before the first attempt to save the dashboard, the error window appears.

To muddy the issue further, when a second attempt is made to configure the above simple example, the   text no longer shows in the HTML field but the dashboard still refuses to save! An attempt to include the £ symbol in the text using the extended HTML special character: £ will save successfully so this issue does not apply to all of the special HTML characters. The only way that I have found to achieve the result that I desired was to use the <pre>& </pre>tags to preserve the formatting thus:

<pre> Ground </pre>

The weird and wonderful world of HTML!!

borpin · 2020-01-09T10:46:45.850Z

@emrys - one for you?

emrys · 2020-01-13T13:09:21.779Z

issue already in github:

github.com/emoncms/dashboard

href addresses getting altered in emoncms!

opened 01:42PM - 04 Oct 18 UTC

closed 04:36PM - 12 Mar 20 UTC

pb66

bug

I have many dashboards that contain images, they range from a simple single imag…e alone on a page to much more complex pages involving multiple images. Generally these are set up using a simple paragraph widget with some html like this (but they do differ) ``` <div style="text-align: center;"> <a href="/linked_address"><img src="/images/displayed_image.png" style="max-width: 100%; height: auto;" alt="FYI, image alt text is required"> </a></div> ``` this has worked well for a substantial time, recently I updated the sever that was running 9.8.18 and the dashboard module version of that time, to the very latest versions as of now. I have just tried to change an image and keep getting this error ![image](https://user-images.githubusercontent.com/5606042/46475745-450dce80-c7de-11e8-8757-ae8da20e81d7.png) all I have done to get that error is change the image url by 1 character (eg "displayed_image.png" to "displayed_image1.png") both images are present and correct, I have even tried copying "displayed_image.png" to "displayed_image1.png" so it is even the same file it is trying to load/locate. I have since searched every where I could think of and eventually found that all the href entries in the mysql dashboard table has "`
`" appended to the end of the url. eg src="/images/displayed_image.png
". Armed with this info I have looked at the dashboard configuration content editor (designed for cutting and pasting content between dashes) and found the "`
`" is also shown there, but when viewing the same href url via the paragraph widget configuration box in the dashboard editor, it is filtered and doesn't show the "`
`" even though it is present in the source. I have now proved this is the cause of the error I see by editing the href url in the content editor to remove the "`
`" it will save and work fine, obviously there doesn't appear to be any change when viewed via the widget editor as the extra "
" doesn't show beforehand. My original task was to change 1 image on a page of 32 images. I have managed to achieve this by manually editing every one of the 32 image urls (to remove the "`
`" and add the single "1" to the image url in question) as editing any one or any number less than ALL of the links resulted in the same error being raised when saving. I do not know how those extra chars were added (by emoncms) but I do know that I have dozens of accounts and many dozens of these pages that all need the "`
`" removed from the end of many href urls. Is there a safe automated way to do this on a live server? or could what ever the paragraph widget box does to filter the source be embraced to remove these unwanted extra's when saving a dash? It's really annoying that the paragraph widget displays correctly when the source is wrong and yet, it says there is "no change" when you try to re-save without adding further changes, if it would just save what it displays (forced even though no change detected) it might remove the "`
`" itself. In fact, this check for change and block saving if no changes _detected_ has really hindered debugging this issue. If it was possible to open a dashboard for editing and attempt a save without being told there are no changes, it would have raised the error about "invalid content" and that would have rang alarm bells as the dash content was unchanged, the fact it forces me to make changes and only then triggers an error, suggests there is an error with the changes not the original source. I have to say, having just had to re-edit all the "`
`" strings in this post as they display as "
" without adding the backticks like so ``` "`
`" ``` I do not cherish the idea of trying to do a mysql query to edit by regex!!! PS - All the images and dashboards seem to display ok, this is just an issue with resaving the dashes after any edits.

i’ll take a look at it

emrys · 2020-01-13T13:44:00.991Z

github.com/emoncms/dashboard

checked for html characters in model

emoncms:master ← emrysr:feature/issue-171-checking-for-nsbp

opened 01:21PM - 13 Jan 20 UTC

emrysr

+27 -16

added some js translations added check for nbsp and added it back after xss fil…ter using async function to save dashboard content fix #171 [screenshot of 2 tests causing errors - now don't cause errors] ![image](https://user-images.githubusercontent.com/1466013/72259079-7f803100-3607-11ea-8f57-f5bf2a6ef442.png)

@mhamer - could you try this pull request?

mhamer · 2020-01-13T16:31:44.298Z

Thanks for your proposed fix to the   problems that I encountered. The issue seems to be closely related to the href bug raised in Oct 2018 in that the code renders correctly in the dashboard editing window but will not save upon exit from this window. Then some of the HTML characters disappear from the text editing window when the dashboard is opened a 2nd time.

Unfortunately, I am unfamiliar with GitHub. A first impression of the subject just undertaken is of a confusing, impenetrable jungle of jargon that will take me some considerable time to grasp. I need to find a GitHub for Dummies before going further!

Your screen shots of the two HTML lines I used for illustration appear to have rendered and saved correctly but I have to leave it to others to try out this pull request to verify your fix.

I have come to a standstill having initialised a git repository in my pi/home area and have no desire to corrupt my working system by any making further stab-in-the-dark changes!

emrys · 2020-01-14T11:19:14.467Z

Hi @mhamer, apologies if this is repeating what you already know - also using this post to document the issue for others that might have this issue.

Yes @mhamer you are correct the main change here is to preserve the   HTML entity from being decoded as a (space) and displaying errors when saved. This was the cause of the error you saw - the decoded and sanitized version of the value would not be valid as they don’t match the original (the entity was decoded as a ’ ')

The HTML entities are saved in the database as entered, however they are decoded in the textarea/input box as the intended character.
eg. £ would be shown as £, but still saved in the database as £

entering this into the edit window:

image968×214 5.52 KB

would result in this being displayed the next time you edit:

image972×180 4.71 KB

As HTML is allowed to be entered in here we must be careful not to allow malicious code that would run on other user’s machines as JavaScript. The above example shows that the   is shown as the non-breaking space character. HTML characters should be rendered as appropriate glyph within the input box / textarea.

We use a “Anti XSS library” to achieve this filtering. The full list of checked HTML Entities can be found here: dashboard/AntiXSS.php at master · emoncms/dashboard · GitHub

The library used replaces the   code for it’s (space) character. This caused issues on this character only. All other Entitles should be successful checked without the need to manually replace them before the validation test.

mhamer · 2020-01-14T17:48:39.898Z

Hi @emrys, I found your explanatory post very helpful being unaware of the behind-the-scenes validation that is required to mitigate against malicious code. Looking at your 2nd reply, I guess that the next stage is to gain approval to get the fix into the master branch of the emoncms code. Presumably it can then be rolled out at the next periodic update to emoncms on the Admin page. Hopefully there will be no more HTML extended characters that will require this type of manual intervention.