We’ve put considerable effort into controlling the amount of writing the emonPi does to an SD card to prolong SD card life. The results seem to be pretty good, we’ve had SD card running for a number of years without any issues. However, there has been occasional reports of SD cards failing prematurely. Especially after power cuts.
We usually use standard Kingston SDCS2/16GB micro SD cards. Frustratingly SD card datasheets don’t quote expected write limits.
I’m considering doing some testing with ‘industrial grade’ SD cards to see if they offer any improvement in durability. This is obviously a difficult test to do since way may not know the result for 5+yrs!
From my limited research I’ve established that most ‘consumer grade’ SD cards have MLC NAND flash, industrial SD cards generally have SLC NAND flash which is quoted to achieve 10x-30x more writes. Here’s a detailed comparison between the two types of NAND. Unfortunately SLC SD cards command a 10x price increase over standard MLC SD cards of comparable size.
The lowest cost SLCD micro SD cards I’ve managed to find are:
Do you have any evidence of this? I’m not too sure how this could happen since the Pi is powered by a SMPS which should be able to filter out voltage spikes. If no other equipment in the house is damaged it’s probably unlikely to be a voltage spike. During EMC and CE testing the Pi was subjected to some severe voltage spikes on the power line and Ethernet ports without any damage sustained.
No, no evidence, just a worthwhile precaution and one less variable and I’ve never had an SD Card failure due to a power cut. 2 failures, one a very old emoncms setup that was also recording weather data and a second Home Assistant, again a while ago. However, I have never run any of those systems, in anger, on an SD Card since .
The other variable to consider is if the SD Card is genuine or not. Many get bought on eBay - few are checked to see if they are genuine.
This is something we have been worried about as well. Still not convinced the SLC cards are worth it as there are many other factors affecting performance/durability. There have been a few other developments in this space which are good to be aware of:
A lot of the ‘industrial’ part of the industrial cards is about having wider environmental ranges rather than increased write durability and the designation doesnt generally mean they are better in this regard. Industrial SD cards are often also targeted at specific applications and its worth contacting the distributors to find out what they are usually used for (i.e. they may not be suitable for use as RPI system/boot disk).
There are consumer cards with wear levelling now like WD Purple. These may offer most of benefits of industrial SLC at much lower price points.
Manufacturers have ranges of ‘endurance’ cards aimed at video dash cams. What exactly is meant by ‘endurance’ is unclear and it appears to be largely a marketing term. But it is worth noting they are warrantied for either more years of use and/or higher amounts of write. Some may also have other features like wear levelling to help them achieve this (but there are other strategies manufacturers cam employ to increase durability).
There is a newer intermediate class of memory called pSLC which is cheaper than SLC but has better write durability than MLC. I dont think this has found its way to consumer cards as of yet but it is offered as part of industrial SD card ranges.
That’s Ben, that’s very useful. Part of the challenge is trying to decode all the flashy marketing around SD cards to figure out which have may have benefit to us.
Endurance cards aimed at video dashcams may be a good fit for our use case since they will no doubt have a much higher write limit.
Warranty does not mean much, the Kingstone cards we currently use have a lifetime warranty. However, it’s not the financial loss of the card which is the issue is the associated data loss / downtime and support which is more of an issue.
I’ve not heard of pSLC, sounds interesting. I will do some research, SLC cards are crazy expensive.
At this point I’m trying to establish how widespread or not issues with SD cards are, hence the poll.
By warranty i was referring more to being warrantied for a certain TBW or hours of use (but probably use in raspberry pi would be in breach of many of these warranties).
Ultimately we should expect convergence between SSD and SD in terms of durability and performance. Wear levelling and things like TRIM will become available on more consumer SD cards and SD 7 cards (SDUC) with support for PCI-E/NVME under the ‘express’ moniker will enter mainstream use.
The WD Purple QD312 are quite interesting … very high ‘rated’ write durability and they are public about it using wear levelling / ECC (although not in spec sheet). And marketed at ‘IoT’ applications.
The Western Digital Industrial IX range I think is basically the same cards but they actually have the good stuff in their spec sheets:
These are both MMC based but the wear levelling / ECC will compensate for this. They should offer much better value than SLC but with much better guarantees on durability than consumer MLC cards.
People also talk about over-provisioning consumer cards to support durability but this relies on the controllers supporting this so doesnt provide same guarantees.
The WD XI SLC card is rated 10x more endurance writes (TBW) than the WD XI MLC card.
Both SD cards are expensive compared to our current Kingston cards. However, since the SLC card is not extortionately more (still cheaper than a hard drive) it may be worthwhile going all-out to get the longest life possible for applications where failure would be a significant issue. What do you think?
Industrial SD cards will be an optional extra that probably most users won’t want to pay extra for, but for some applications it may be worthwhile.
But is prevention of one problem better than a cure for most problems?
Could backup be the answer?
Any solution will add cost and, as an example, My Memory have a 16 GB USB 3.0 10 MB/s WRITE flash drive for £8 (but not currently in stock).
There are existing emon scripts for backup export and import. Would it be difficult to get emon to recognise when a flash drive is plugged in, to mount it and then to do a backup at regular intervals? This would give the user Plug-n-Play choice.
I use this approach. My backups are 300 MB in size and the process takes 20 mins. I do it weekly.
Agree, SD card failure is still a significant issue even with regular backups.
Often units are installed in remote locations or customer sites that require significant time, effort, cost and associated downtime and data loss before a site visit can be arranged to perform a restore from a backup.
Without testing (of which there is surprisingly little out there considering the widespread use of these types of products!) its hard to say what represents best value but presence of wear levelling/ECC on spec sheet for the MLC version should offer guarantees which may be worth the premium. And yes you could judge the SLC to be worth it - I think SLC at this price warrants serious consideration depending on application.
A lateral thought …
And I do not know how SDHC’s and USB flash drives compare re wear.
I believe it is possible to run emon from a USB drive - just booting from the SDHC.
Or perhaps just run /var/opt/emoncms (/dev/mmcblk0p3) on a USB drive.
So were that USB drive to be in fact two USB flash drives in RAID 1 configuration then redundancy would be built in, early warnings of failure published, etc.
It has generally been true that due to historic typical use applications and manufacture typical USB drives were better than typical microSD for boot/system disks. But there is nothing inherently better about the memory in USB drives. If we are comparing to industrial SD cards above (whether MLC with wear levelling/ECC or SLC/pSLC type) these may be better than many consumer USB drives and even as good as some SATA/NVME SSD drives.
If you are having to buy another (USB flash) drive, why not spend the same amount of money to get a much better quality microSD and also not have to bother with the extra configuration involved? I think this warrants serious consideration now that SLC and more robust MLC/pSLC microSD are much cheaper than they used to be.
I’ve been told that with solid state drives, the controller writes to different areas to prevent specific areas from being written to too often. The result is that if the SSD size is doubled, for the same software, each segment is written to half as often. The expected life (or time before serious degrade) is therefore doubled (or at least increased).
Would the same effect be noticed on an SD card, and would it therefore be a strategy to oversize the SD card?
.