New emonSD Pi WiFi interface (looking for testers!)

emonSD
11

I’ve deployed this update on about 4 different devices over the last day, it’s so much better! So far so good on stability but it’s a bit early to tell as much as I’d like to roll this out sooner :slight_smile:

12

In the 10+ years I’ve been associated with OEM, I’ve seen this line of “thinking”
bite you in the backside on more than a couple of occasions.

I’d suggest testing it for at least 30 days. (90 would be better)

13

:joy: I opened myself up for that comment!

2 Likes
14

And how long were emonLibCM and emonLibDB on test before I released those? - a good bit longer than 30 days.

1 Like
16

It turns out that the latest RaspberryPi OS (bookworm) now uses NetworkManager to mange wifi networks. It’s easy to disable NetworkManager and continue to use the implementation I’ve built above that uses wpa_supplicant, it’s just a matter of:

sudo systemctl disable --now NetworkManager.service

(The wpa_supplicant packages are all present and ready to be used directly on the newer images if needed - I think NetworkManager may just be a wrapper on top of wpa_supplicant).

However this brings up the question as to whether I should go back to the drawing board and re-implement this whole implementation using NetworkManager.

It is possible to achieve the same results in principle, I familiarised myself with the basic command line to configure NetworkManager yesterday and configure a simultaneous Access Point and client connection. I couldn’t find this documented that clearly elsewhere so here are the basic commands for anyone else interested:

1. Create a virtual interface for the access point:

sudo iw dev wlan0 interface add ap0 type __ap

2. Give the access point a name and password:

sudo nmcli dev wifi hotspot ifname ap0 ssid emonPi password emonpi2016

3. Connect to client WiFi (e.g home network):

sudo nmcli dev wifi connect MY_SSID password MY_PASSKEY ifname wlan0

4. Scan for WiFi networks on access point interface:

sudo nmcli device wifi list ifname ap0

easier to process format:

nmcli --get-values ssid,mode,signal,security device wifi list ifname ap0

5. Re-scan for WiFi networks on access point interface:

sudo nmcli device wifi rescan ifname ap0

6. Connect and disconnect from an interface:

sudo nmcli dev connect wlan0

sudo nmcli dev disconnect wlan0

Do I go back to the drawing board and explore the use of NetworkManager?

  • Advantages: Minimal package changes on bookworm+ RaspberryPi OS images.
  • Disadvantages: back to the drawing board, requires NetworkManager package installation on older images, dont know if it will install cleanly and work on older images? There are examples of installation on images as far back as 2015…

I’ve already spent a lot of development time on this before and over Christmas and there’s lots of other things I should attend to so as ever its also a question of how best to allocated my development time… Are there other reasons I should perhaps stick to the direct wpa_supplicant approach implemented so far above?

17

ModemManager also produces one journalctl error on bookworm but does not materially affect operation so may be ok to leave running and may well be needed by 2G/3G/4G dongles. I’m not sure. Disabling is easy e.g:

sudo systemctl disable --now ModemManager.service
18

Although that command prevents NM from running, it still gets loaded.
No sense in loading a service that’s not going to see any use.

To prevent it from being loaded, use mask after issuing the disable command.

e.g. sudo systemctl mask NetworkManager.service

Reboot to clear NM from memory.

If you want to re-enable NM, issue the unmask command first:

sudo systemctl unmask NetworkManager.service

then enable it with:

sudo systemctl ensable NetworkManager.service

2 Likes
19

Yes. at some point the old method may well be deprecated.

Alternatively, leave the old method as is for older installs (don’t include in the update script) and just use the new module for Bookworm installs.

I’d also check what you are trying to achieve, is it that when there is no network connection, create the AP?

Actually, disable stops the unit being automatically loaded/run at boot. If any other service has it as a required service, that will force it to start. As you say, the only way to prevent it ever being loaded is to mask it.

21

I’m not saying you should go back to the start, but when I’ve needed to do things with networking I’ve found nmcli quite capable and reliable.

1 Like
22

Thanks all, good to hear your experience @MyForest.

I encountered a few other issues building the latest image with bookworm that I will investigate and I think at least prototype with the above nmcli commands to get a clearer picture of level of work involved.

The key thing I am trying to achieve is to fix the less than satisfactory existing WiFi setup process.

  1. At the moment we create an access point on an un-configured system at startup. The user can then enter their WiFi client details, but if the password is entered incorrectly the system does not provide a fall back to allow correct entry. You either need to connect the Pi via Ethernet or edit wpa_supplicant on the SD card to fix.

  2. When a user changes their internet router or WiFi client passkey there’s no easy way of changing the WiFi details on the Pi. Again this needs to be done via Ethernet or via an SD card reader to edit wpa_supplicant.

On the emonPi2 the push button and display provides the opportunity to enable the access point in order to change WiFi details as in case 2.

Simultaneous support for an active client connection and an access point helps with case 1. As you can keep connected to the access point throughout the WiFi client configuration. If the connection fails due to incorrect passkey, you can enter it again without moving away from the WiFi configuration user interface.

For emonBase users without a push button and LCD to manually enable an access point ideally some way of enabling the access point when the WiFi client connection fails would be great / is technically a lot easier with the above implementation. Apart from @Robert.Wall’s idea of plugging in a USB mouse to trigger this another option could be to allow automatic access point creation if the default access point password has been changed? Perhaps the access point password could be set to the same password as the home WiFi password? This would need to be made clear to the user of course!

1 Like
23

The other issue with the existing implementation is that the system needs to reboot to switch between access point and wifi client (I never got this to work in a more seamless way - though Im sure that’s my fault not the capability of the underlying software). This makes the setup process quite slow. The above implementation is needless to say much faster and works as you would expect it to work.

24

I would not want an AP to be continually active, personally (if I read what you are suggesting correctly).

Is the issue then the detection of no WLAN and the actions to take if that happens?

Does that link I suggested earlier that uses the dispatcher work in that if you use an incorrect password so no network connection can be established, does an ifdown event occur? You’d need to do some checking to ensure it really is down of course and not just a transient issue.

25

An update on progress:

First I have decided to go with the nmcli approach, installation is very slightly more straightforward on newer RaspbianOS images.

My plan is not to offer this as an update to existing systems just yet. While it is possible to implement an update process there is a risk for systems already configured on WiFi that something could go wrong on the transfer process from the wpa_supplicant config over to the nmcli config, I don’t want to run that risk just yet and so think it’s safer for the EmonScripts update process to not update this particular part of the system - which incidentally it would not do anyway.

The plan would be to make this available on new images to start with at least and provide a manual update process for those particularly keen to switch it over on existing images.

In that direction:

  1. The EmonScripts install scripts have been updated to install the new network module and a stable release has been created. I know not many build from scratch using this script but if anyone does get a network related issue having built from scratch in the next days/weeks this update could be at fault - at least it’s something to watch out for. Im testing here at the moment so I expect that any remaining bugs will get ironed out shortly.

  2. In order to simplify backwards compatibility when updating older images I’ve split out the new emonPiLCD script that has the option to enable/disable the access point out from the emonpi repository into a new seperate repository called emonPiLCD https://github.com/openenergymonitor/emonPiLCD. This repository gets installed on new images but is not yet installed as part of the update process. The new image builds no longer require the original emonpi repository which is a plus as that repository became a large mix of different things and we had been looking for a way to move away from needing that repository on all builds. Existing systems will still use emonPiLCD script from the original emonpi repository for now at least.

  3. Existing systems/images will continue to use the old WiFi module rather than install the new Network module for now so as not to risk disturbing existing configuration.

I will be building a new image shortly with the new Network module and nmcli implementation and will also update this thread with a manual update script for those keen to upgrade existing systems to the new networking implementation.

26

That seems the best way. Sorry, not had time to test :frowning:

27

@borpin I’ve been through the EmonScripts build process now quite a few times and ironed out a few issues, all seems to be building fine now. /boot has moved to /boot/firmware with Bookworm and /vat/tmp needs about 128mb rather than the 1MB I had allocated in fstab. I’ve added in and fixed a number of other manual steps.

@MyForest @jeremypoulter I’m mostly happy with the nmcli based implementation now. Im just stuck on two last points that I wonder if either of you might have any insight into:

  1. Devices connected to the ap0 hotspot can access internet routing from wlan0. I thought this was not the case but it does seem to be doing that. I tried ipv4.method manual and setting related settings but that just seemed to break the ap0, devices could not get a DHCP address. The hotspot nmcli configuration code can be found here if either of you can see any obvious pointers? network/scripts/startAP.sh at master · emoncms/network · GitHub

  2. I can get captive portal to work for Android but not for iOS. Im redirecting the following addresses:

address=/connectivitycheck.gstatic.com/192.168.42.1
address=/clients3.google.com/192.168.42.1
address=/captive.apple.com/192.168.42.1
address=/apple.com/192.168.42.1
address=/www.msftncsi.com/192.168.42.1

and then trying to catch these in emoncms with a redirect to the login page, works fine for Android but for some reason not iOS…

any pointers would be gladly received :slight_smile:

1 Like
28

So it turns out that the captive portal I had working so nicely on Android only works when one of the other interfaces are also active e.g eth0 or wlan0. When the Pi starts up without ethernet or wlan0 something breaks with DNS when accessing over ap0. The dnsmasq settings are not adhered to https://github.com/emoncms/network/blob/master/scripts/startAP.sh#L35C1-L35C52 I’m at a bit of a loss tbh.

I think I should be using ipv4.method = manual but I cant seem to get that to provide DHCP and so clients are not able to establish a connection…

29

Did you mean this?

30

Is that a typo? i.e. should it be /var/tmp ?

31

Typo :+1:

1 Like