I downloaded the latest emonSD-07Nov16 image, mounted the data partition inside a limux virtual machine, and made some changes to emonHub and EmonCMS. Changes are non-breaking, 100% sure.
Now when I launched the image on the rasberry-pi, I was unable to receive inputs from the RFM device to emoncms.
With some debugging, I found that the mqtt.service file was missing. How come this happened? Does the latest image not have all the required files, or did it get deleted on start-up?
I see in some other threads people are having some issues using the mqtt.service aswell, but surely booting the latest image should not have caused this?
Can you download and attach the update.log for us to take a peek at?
Is there any chance these changes are blocking the emoncms repo updating via a git pull?
The latest image doesn’t have the mqtt_input.service file, it was added since the original image was built. The file is included in the updated emoncms repo and copied across to the /etc/systemd/system folder by the updated emoncms update script, If the script has updated to be aware of the file but cannot find the file it would suggest the emoncms git pull has failed or is pointing at a different branch.
I have bumped your user level up a notch so you can attach the logs in-line. We try to avoid the use of links in place of log files as if the source disappears the link is broken and the thread incomplete, so I have copied the log to your post.
Any suggestions how to go about this? I want to make the image pre-compiled, so that I dont need to do any extra work after booting.
Maybe I can edit where those pull requests are coming from, and just point them to my own repo, and I then need to manually keep the 2 synced?
Where is this update script located, and from where is it called on start-up?
That should work but you need to keep the “other” repo bang up to date, using this same example if your repo didn’t contain the mqtt_input.service file then it would have still fallen over, so you need to ensure any upstream changes are merged before any incomplete update routines can be run, unless you duplicate all the repos to retain control.
Even with the “other” repo you would still be vulnerable to changes in the main emonpi update script over ruling your changes, for example the backup module pull has a checkout command, if a checkout gets added to the emoncms update command then the update would also switch branches and your local changes discarded until you manually checkout your “other” branch again.
The update routine is quite involved but specifically at first boot the firstbootupdate script is run by rc.local at boot and that triggers an update if there is no pre-exising update.log.
Then the main update is done by service-runner-update.sh and that in turn uses the emoncmsupdate emonhubupdate and emonpiupdate scripts
Another question, does the filesystem expand function run during first-boot or something? I mounted a original unmodified emonpi image to a mem-card, and the saved it back to pc, but now its almost 8gigs.
Anyway to preserve the “small” sizing?
The actual content of the image is about 3.8 gigs.
Problem is getting a saved image back to pc once its been burned to a memory card. The image then becomes the size of memory card, regardless if it booted or not.
Perhaps @TrystanLea or @glyn.hudson could comment how they build or modify prepared images?
The usual method is to use dd to copy a set size as, yes, a disk or image copy will be dictated by the size of the card. A quick and dirty method might be to use a 4gb sdcard to carry out your personalisations and then save that 4gb image for use on 8gb SDcards.
There are a couple of tricks to really squash the image for delivery;
Before you DD the image off the card;
Update the system and clean up
Don’t expand the SD card
apt-get clean all
Do all your mods
Use DD on the booted Pi, write a file full of zeros and then delete it - do this on every partition.
dd if=/dev/zero of=/file.out bs=1M count=4096
rm -rf /file.out
Shutdown cleanly and DD the card to an image file - this will be the same size as the whole card - this is normal… the reason you DONT expand the SD image on the Pi - is so that once we are done with the next step it will fit on 4G cards still…
Thanks for the solid reply Andy!
Whats the purpose of “write a file full of zeros and then delete it”? Can I do this from through the ssh on the booted device? How would I access the other partitions then?
I have not used DD before, please forgive if questions don’t make sense!
The reason for the dd to write zeros all over the CF card is quite simple and quite complex all at once.
Most filesystems on a computer, don’t actually delete a file when you tell them to, they remove the pointers so that the system doesn’t know that the file is there any more, they mark the blocks as available and thats it - the actual data is still there (this is why file recovery software works… well so long as the blocks have not been over written…).
So when you use dd to take an image of the card - it does so at the block level, including all the crap you have written before and deleted again (or changed jn the case of anything that ever changed ever…).
When you compress the image - you also retain and compress all of the garbage with it, but its toast - we don’t need that, and this is where writing zeros all over the card comes in…
so now we make a giant file, full of zeros to deliberately fill up the CF card (with zeros) and then delete it (leaving those zeros behind remember…).
now when you take the image of the card, those zero’s are still copied (because its a block level copy) BUT, when you come to zip (compress) the image ready for deployment, zeros compress to nothing - where miscellaneous binary crap - doesn’t.
I hope thats a good enough explination - if not please say so and I’ll see if I can help you understand whatever you didn’t quite get.