New emonSD Pi WiFi interface (looking for testers!)

[TrystanLea]

Happy new year all!

I’ve been working away here trying to improve on the emonPi / emonBase WiFi configuration process & interface. Especially given that the new Pi Zero 2W option for the emonPi2 does not have an Ethernet port to fall back on.

I came across an alternative approach to access point creation using systemd-networkd rather than the ‘classic’ approach that we were using before. The big step forward with this approach at least vs the implementation that we had previously is that it allows for an access point to be active at the same time as a WiFi client connection which makes things much easier! It’s possible to return the WiFi client IP-Address whilst still connected to the access point. All standard stuff that you would expect from WiFi network configuration of course.

I’ve rebuilt the Emoncms WiFi module and associated scripts to use this new approach and added the option to turn on and off the access point from the push button on the emonPi2 display. Im working through testing and polishing off a few things at the moment, I want to make sure this is as stable as possible before it is available to update via the standard update process.

If anyone wants to help test, the manual install guide for this is available here:

https://github.com/emoncms/network

A couple of screenshots of the updated emonPi user interface:

Initial Setup page (seen when connecting to the Pi for the first time)

1. Scanning for networks:

2. WiFi network list:

3. Enter network password:

4. Connecting to a selected network:

5. IP address of connected network:

Network Interface available when logged in:

emonPi2 LCD option for turning on/off the access point:

---

I will update this topic once this is available via the standard update process.

[Robert.Wall]

A question that’s appeared a few times, amid considerable confusion it seems, is: “How do I forget one wireless network and connect to another?” Which maybe begs the allied question: “How do I change my Wi-Fi password?” Will these things be obvious and easy?

[borpin (Brian Orpin)]

@TrystanLea

If anyone wants to help test, the manual install guide for this is available here:

We have had issues in the past with PiZeroW and having to disable the WiFi module to successfully build a working system with the scripts. Have you tried one?

If not, I’ll give it a bash next week as I need to rebuild a PiZeroW that the SD died on (something went wrong last time I tried and I didn’t have time to go back and investigate).

Is it a specific branch on the WiFi module to load? Is there a change in the emonscripts?

[TrystanLea]

Thanks @Robert.Wall @borpin

@Robert.Wall

How do I change my Wi-Fi password?” Will these things be obvious and easy?

Yes making this easier is definetly a goal. As it is currently working a emonPi2 user with an LCD and push button can click on the push button to enable the access point to regain access to the emonPi2 in the case that they have either changed the WiFi password or changed router. Then login to update the WiFi details.

I’ve done away with the option to configure multiple WiFi clients from the user interface. That seemed like an advanced use case that complicated the older interface for everyone else. There’s no need to click a checkbox to enable a particular wifi network, which makes it a bit more straightforward on that front.
It’s still possible to configure multiple networks by editing wpa_supplicant.conf directly if someone really wants to do that - or a hidden network for that matter.

At the moment the access point is automatically enabled only if the Pi does not have any other active network connections and there is no network configuration present.

For emonPi2 users with the LCD and push button enabling the access point manually is straight forward but for emonBase users that manual recovery option does not work… which begs the question should there be a mechanism to automatically start the access point whenever whenever the wifi client fails. I’m reluctant to do that as it is a security risk, e.g a neighbour could access the Pi via the access point, but there may be a way to make it a limited window perhaps…?

@borpin

We have had issues in the past with PiZeroW and having to disable the WiFi module to successfully build a working system with the scripts. Have you tried one?

No Im not targeting that version but it would be interesting to hear if it does work.

@borpin

Is it a specific branch on the WiFi module to load? Is there a change in the emonscripts?

Its a new module called network that is installed in /opt/emoncms/modules as it collates a number of scripts that were previously located in multiple repos. I’ve detailed the install steps over at GitHub - emoncms/network: Emoncms networking module: wifi access point, client & ethernet there are a few other modules to update and change branches

[Robert.Wall]

@TrystanLea

which begs the question should there be a mechanism to automatically start the access point whenever whenever the wifi client fails. I’m reluctant to do that as it is a security risk, e.g a neighbour could access the Pi via the access point, but there may be a way to make it a limited window perhaps…?

You’ve really got to have the equivalent of the WPS button on your router - something physical, initiated by the user which opens it at a known time and for a limited duration. It’s wide open if it happens predictably – say on startup, which would inevitably follow when recovering from a mains failure.

In the absence of a convenient button, what could the Pi sense being plugged in that most users might have handy - a USB mouse for example? - which could put it into AP mode?

[TrystanLea]

@Robert.Wall

In the absence of a convenient button, what could the Pi sense being plugged in that most users might have handy - a USB mouse for example? - which could put it into AP mode?

Interesting idea… that would be a nice trick!

[borpin (Brian Orpin)]

@TrystanLea

No Im not targeting that version but it would be interesting to hear if it does work.

In that case, you will need to explicitly state what models of Pi are now supported as we will have moved from the universal position; perhaps create an option in the scripts for old_pis that disable the appropriate modules?

@TrystanLea

which begs the question should there be a mechanism to automatically start the access point whenever whenever the wifi client fails.

Most devices that create an AP, do so on boot if the configured AP connection fails, not when a working config stops.

@TrystanLea

I’m reluctant to do that as it is a security risk, e.g a neighbour could access the Pi via the access point,

Doing it on boot largely mitigates this risk. You can also make it such that the AP needs a password (even if it is a common password) and also make it such that it is only available for a defined period (30 mins).

Did you look at using systemd to check for the network connection? Something like networking - monitor network status for systemd script - Ask Ubuntu (but this was my first hit).

The NetworkManager-dispatcher will do the checking for you and in that sample above, you could then possibly call a script to restart as necessary. Take care not to be too quick off the mark though as transient drops are possible, but you could write to a log file when there is a change in status (might help with debugging if the network is continually dropping briefly).

Another thought re logging - DebuggingNetworkManager - Ubuntu Wiki, I’m not sure how much traffic there is, but perhaps increasing the log level might be worthwhile. There is also a connectivity check command available - not sure what that achieves though.

Generally, it would be better to use builtin connectivity checks than create your own I’d suggest.

[borpin (Brian Orpin)]

Just in general for debugging, I think a means of breaking out the display port to the case edge would be of benefit.

[TrystanLea]

Another option apart for an automatic access point might be to have the option to change the passkey of the access point e.g have it match the home network passkey and then only automatically enable if the passkey is different from the default. The access point could enable if the wifi client drops for more than 60 seconds and then automatically disable if the wifi client has reconnected for more than say 10 minutes…?

@borpin

you will need to explicitly state what models of Pi are now supported as we will have moved from the universal position;

There’s a good chance it might work, noting this guide for the same systemd approach that mentions the Pi Zero Instructions | Raspberry Pi networking the systemd way | Hackaday.io. If you could test it that would be amazing!

@borpin

Did you look at using systemd to check for the network connection?

Yes the implementation checks if the wpa_supplicant systemd service is running as part of the status check. There are two services, one for wlan0 and another for ap0.

Enabling and disabling the access point (on the backend) is just a matter of running:

sudo systemctl enable --now [email protected]
sudo systemctl disable --now [email protected]

Im using enable/disable here to persist the settings at reboot.

[TrystanLea]

I’ve just pushed a few more updates to the emonpi LCD script (wifi3 branch).

The other change I’ve made as part of this larger piece of work is to improve the responsiveness and feedback when using the emonPi2 push button. When you press and hold it now counts up to 5 giving a clear indication that a long press action is in process.

[TrystanLea]

I’ve deployed this update on about 4 different devices over the last day, it’s so much better! So far so good on stability but it’s a bit early to tell as much as I’d like to roll this out sooner

[Bill.Thomson]

@TrystanLea

but it’s a bit early to tell as much as I’d like to roll this out sooner

In the 10+ years I’ve been associated with OEM, I’ve seen this line of “thinking”
bite you in the backside on more than a couple of occasions.

I’d suggest testing it for at least 30 days. (90 would be better)

[TrystanLea]

I opened myself up for that comment!

[Robert.Wall]

And how long were emonLibCM and emonLibDB on test before I released those? - a good bit longer than 30 days.

[TrystanLea]

It turns out that the latest RaspberryPi OS (bookworm) now uses NetworkManager to mange wifi networks. It’s easy to disable NetworkManager and continue to use the implementation I’ve built above that uses wpa_supplicant, it’s just a matter of:

sudo systemctl disable --now NetworkManager.service

(The wpa_supplicant packages are all present and ready to be used directly on the newer images if needed - I think NetworkManager may just be a wrapper on top of wpa_supplicant).

However this brings up the question as to whether I should go back to the drawing board and re-implement this whole implementation using NetworkManager.

It is possible to achieve the same results in principle, I familiarised myself with the basic command line to configure NetworkManager yesterday and configure a simultaneous Access Point and client connection. I couldn’t find this documented that clearly elsewhere so here are the basic commands for anyone else interested:

1. Create a virtual interface for the access point:

sudo iw dev wlan0 interface add ap0 type __ap

2. Give the access point a name and password:

sudo nmcli dev wifi hotspot ifname ap0 ssid emonPi password emonpi2016

3. Connect to client WiFi (e.g home network):

sudo nmcli dev wifi connect MY_SSID password MY_PASSKEY ifname wlan0

4. Scan for WiFi networks on access point interface:

sudo nmcli device wifi list ifname ap0

easier to process format:

nmcli --get-values ssid,mode,signal,security device wifi list ifname ap0

5. Re-scan for WiFi networks on access point interface:

sudo nmcli device wifi rescan ifname ap0

6. Connect and disconnect from an interface:

sudo nmcli dev connect wlan0

sudo nmcli dev disconnect wlan0

Do I go back to the drawing board and explore the use of NetworkManager?

• Advantages: Minimal package changes on bookworm+ RaspberryPi OS images.
• Disadvantages: back to the drawing board, requires NetworkManager package installation on older images, dont know if it will install cleanly and work on older images? There are examples of installation on images as far back as 2015…

I’ve already spent a lot of development time on this before and over Christmas and there’s lots of other things I should attend to so as ever its also a question of how best to allocated my development time… Are there other reasons I should perhaps stick to the direct wpa_supplicant approach implemented so far above?

[TrystanLea]

ModemManager also produces one journalctl error on bookworm but does not materially affect operation so may be ok to leave running and may well be needed by 2G/3G/4G dongles. I’m not sure. Disabling is easy e.g:

sudo systemctl disable --now ModemManager.service

[Bill.Thomson]

@TrystanLea

sudo systemctl disable --now NetworkManager.service

Although that command prevents NM from running, it still gets loaded.
No sense in loading a service that’s not going to see any use.

To prevent it from being loaded, use mask after issuing the disable command.

e.g. sudo systemctl mask NetworkManager.service

Reboot to clear NM from memory.

If you want to re-enable NM, issue the unmask command first:

sudo systemctl unmask NetworkManager.service

then enable it with:

sudo systemctl ensable NetworkManager.service

[borpin (Brian Orpin)]

@TrystanLea

However this brings up the question as to whether I should go back to the drawing board and re-implement this whole implementation using NetworkManager.

Yes. at some point the old method may well be deprecated.

@TrystanLea

Do I go back to the drawing board and explore the use of NetworkManager?

• Advantages: Minimal package changes on bookworm+ RaspberryPi OS images.
• Disadvantages: back to the drawing board, requires NetworkManager package installation on older images, dont know if it will install cleanly and work on older images? There are examples of installation on images as far back as 2015…

Alternatively, leave the old method as is for older installs (don’t include in the update script) and just use the new module for Bookworm installs.

I’d also check what you are trying to achieve, is it that when there is no network connection, create the AP?

@Bill.Thomson

Although that command prevents NM from running, it still gets loaded.

Actually, disable stops the unit being automatically loaded/run at boot. If any other service has it as a required service, that will force it to start. As you say, the only way to prevent it ever being loaded is to mask it.

[MyForest (David Bowen)]

I’m not saying you should go back to the start, but when I’ve needed to do things with networking I’ve found nmcli quite capable and reliable.