New emonSD Pi WiFi interface (looking for testers!)

emonSD
22

Thanks all, good to hear your experience @MyForest.

I encountered a few other issues building the latest image with bookworm that I will investigate and I think at least prototype with the above nmcli commands to get a clearer picture of level of work involved.

The key thing I am trying to achieve is to fix the less than satisfactory existing WiFi setup process.

  1. At the moment we create an access point on an un-configured system at startup. The user can then enter their WiFi client details, but if the password is entered incorrectly the system does not provide a fall back to allow correct entry. You either need to connect the Pi via Ethernet or edit wpa_supplicant on the SD card to fix.

  2. When a user changes their internet router or WiFi client passkey there’s no easy way of changing the WiFi details on the Pi. Again this needs to be done via Ethernet or via an SD card reader to edit wpa_supplicant.

On the emonPi2 the push button and display provides the opportunity to enable the access point in order to change WiFi details as in case 2.

Simultaneous support for an active client connection and an access point helps with case 1. As you can keep connected to the access point throughout the WiFi client configuration. If the connection fails due to incorrect passkey, you can enter it again without moving away from the WiFi configuration user interface.

For emonBase users without a push button and LCD to manually enable an access point ideally some way of enabling the access point when the WiFi client connection fails would be great / is technically a lot easier with the above implementation. Apart from @Robert.Wall’s idea of plugging in a USB mouse to trigger this another option could be to allow automatic access point creation if the default access point password has been changed? Perhaps the access point password could be set to the same password as the home WiFi password? This would need to be made clear to the user of course!

1 Like
23

The other issue with the existing implementation is that the system needs to reboot to switch between access point and wifi client (I never got this to work in a more seamless way - though Im sure that’s my fault not the capability of the underlying software). This makes the setup process quite slow. The above implementation is needless to say much faster and works as you would expect it to work.

24

I would not want an AP to be continually active, personally (if I read what you are suggesting correctly).

Is the issue then the detection of no WLAN and the actions to take if that happens?

Does that link I suggested earlier that uses the dispatcher work in that if you use an incorrect password so no network connection can be established, does an ifdown event occur? You’d need to do some checking to ensure it really is down of course and not just a transient issue.

25

An update on progress:

First I have decided to go with the nmcli approach, installation is very slightly more straightforward on newer RaspbianOS images.

My plan is not to offer this as an update to existing systems just yet. While it is possible to implement an update process there is a risk for systems already configured on WiFi that something could go wrong on the transfer process from the wpa_supplicant config over to the nmcli config, I don’t want to run that risk just yet and so think it’s safer for the EmonScripts update process to not update this particular part of the system - which incidentally it would not do anyway.

The plan would be to make this available on new images to start with at least and provide a manual update process for those particularly keen to switch it over on existing images.

In that direction:

  1. The EmonScripts install scripts have been updated to install the new network module and a stable release has been created. I know not many build from scratch using this script but if anyone does get a network related issue having built from scratch in the next days/weeks this update could be at fault - at least it’s something to watch out for. Im testing here at the moment so I expect that any remaining bugs will get ironed out shortly.

  2. In order to simplify backwards compatibility when updating older images I’ve split out the new emonPiLCD script that has the option to enable/disable the access point out from the emonpi repository into a new seperate repository called emonPiLCD https://github.com/openenergymonitor/emonPiLCD. This repository gets installed on new images but is not yet installed as part of the update process. The new image builds no longer require the original emonpi repository which is a plus as that repository became a large mix of different things and we had been looking for a way to move away from needing that repository on all builds. Existing systems will still use emonPiLCD script from the original emonpi repository for now at least.

  3. Existing systems/images will continue to use the old WiFi module rather than install the new Network module for now so as not to risk disturbing existing configuration.

I will be building a new image shortly with the new Network module and nmcli implementation and will also update this thread with a manual update script for those keen to upgrade existing systems to the new networking implementation.

26

That seems the best way. Sorry, not had time to test :frowning:

27

@borpin I’ve been through the EmonScripts build process now quite a few times and ironed out a few issues, all seems to be building fine now. /boot has moved to /boot/firmware with Bookworm and /vat/tmp needs about 128mb rather than the 1MB I had allocated in fstab. I’ve added in and fixed a number of other manual steps.

@MyForest @jeremypoulter I’m mostly happy with the nmcli based implementation now. Im just stuck on two last points that I wonder if either of you might have any insight into:

  1. Devices connected to the ap0 hotspot can access internet routing from wlan0. I thought this was not the case but it does seem to be doing that. I tried ipv4.method manual and setting related settings but that just seemed to break the ap0, devices could not get a DHCP address. The hotspot nmcli configuration code can be found here if either of you can see any obvious pointers? network/scripts/startAP.sh at master · emoncms/network · GitHub

  2. I can get captive portal to work for Android but not for iOS. Im redirecting the following addresses:

address=/connectivitycheck.gstatic.com/192.168.42.1
address=/clients3.google.com/192.168.42.1
address=/captive.apple.com/192.168.42.1
address=/apple.com/192.168.42.1
address=/www.msftncsi.com/192.168.42.1

and then trying to catch these in emoncms with a redirect to the login page, works fine for Android but for some reason not iOS…

any pointers would be gladly received :slight_smile:

1 Like
28

So it turns out that the captive portal I had working so nicely on Android only works when one of the other interfaces are also active e.g eth0 or wlan0. When the Pi starts up without ethernet or wlan0 something breaks with DNS when accessing over ap0. The dnsmasq settings are not adhered to https://github.com/emoncms/network/blob/master/scripts/startAP.sh#L35C1-L35C52 I’m at a bit of a loss tbh.

I think I should be using ipv4.method = manual but I cant seem to get that to provide DHCP and so clients are not able to establish a connection…

29

Did you mean this?

30

Is that a typo? i.e. should it be /var/tmp ?

31

Typo :+1:

1 Like
32

Did you mean this?

33

I feel like on this particular issue I am 99% of the way there, I just need some input from someone more knowledgeable on Linux networking and NetworkManager specifically.

Perhaps I should ask over on the RaspberryPi forums… thought I’ve been trawling through a lot of forums on this and there are quite a few unresolved nmcli threads…

It’s a strange behaviour:

  • If I start the pi with ethernet connected and then turn on the access point, captive portal or the underlying dnsmasq redirect settings work just fine.

  • If I then disconnect ethernet, the captive portal continues to work, even if I disable and re-enable the access point.

  • If I start the pi without ethernet, captive portal and the dnsmasq redirect settings do not work.

What I think it happening is that the ethernet connection or wlan0 connection gets nmcli into the right state/configuration to handle the DNS redirects. If neither of these connections load first then the pi does not get into the right configuration and the DNS redirect settings are ineffectual.

I think this has something to do with ip4v.method shared in that shared means that the ap loads some of it’s configuration from the other interfaces. Hence why I’ve also tried method manual but then that fails to provide an IP address to the connecting device (others have reported that as well on the RPi fourms but with the resolution being to switch to method shared)…

34

I did mean to post that on the staff thread that you posted @borpin but it might as well be here I guess :slight_smile:

2 Likes
35

Progress!

1. manually adding the following to /etc/resolv.conf

nameserver 127.0.0.1
options edns0 trust-ad

during AP startup and then running systemctl reload NetworkManager fixed this issue.

2. and it’s possible to set this via nmcli with:

nmcli con modify hotspot ipv4.dns-options "edns0 trust-ad"

which seems to also set the nameserver… testing to be sure Im not getting some caching issue

36

Looking into:

nmcli con modify hotspot ipv4.dns-options "edns0 trust-ad"

wierdly I think these settings may be incidental and that it’s just the setting of a dns setting that triggers nmcli to reset or load dnsmasq and add nameserver 127.0.0.1 to /etc/resolve.conf…

it works…

1 Like
37

I have the standard behaviour at the moment for automatic starting and stopping of the WiFi access point as:

1. On boot the WiFi access point starts up regardless of other interface connectivity status.

2. After 10 minutes of being enabled the WiFi access point disables automatically.

This means that the WiFi access point will enable for 10 mins after a:

  1. Manual power cycle (useful for emonBase configuration where there is no push button to enable the access point).

  2. Power cut - but will disable after 10 mins to limit window of potential access.

  3. If the Access point is enabled either via emonPi2 button press or via the emoncms network interface UI.

It’s possible to configure the access point settings from a config.ini file located in the network module directory /opt/emoncms/modules/network/config.ini:

[ap0]
ssid = "emonPi"
psk = "emonpi2016"
start_on_boot = 1
stop_mins = 10

Setting start_on_boot = 0 disables access point creation on power cycle. The passkey can also be changed to increase security.

1 Like
38

Another interesting problem, this is specific to the Pi Zero 2W:

  • When connected on wlan0 (client) OR ap0 (access point) to a Pi Zero 2W I can put a delay in the server response e.g network/scan of 1s and the client side on my phone works fine.

  • If both wlan0 (client) AND ap0 (access point) are running simultaneously on the Pi Zero 2W and I try to access the access point from my Android phone (an old pixel) AJAX API responses time out if the server side takes any longer than ~45ms. Tested by adding a delay to the server side php code.

This issue is not present on the RaspberryPi4 which has no trouble accessing via both wlan0 (client) AND ap0 (access point) regardless of the server side delay.

The other peculiarity is that this issue only affects the Pi Zero 2W when being accessed over the ap0 (access point) from an Android phone and not if I connect to the same ap0 (access point) with my laptop…

The issue also occurs on both Chrome and Firefox on my Android but works in both Chrome and Firefox on my laptop connected to the same access point…

39

Are the Wi-Fi chips the same? I’m guessing not and that may be the issue.

Not keen on the AP being created every power cycle if I’m honest. I think it should only appear either on very first boot where there is no Wi-Fi configured, or if there is no connectivity for a period.

Seems you are making progress though :slight_smile:

40

Different chips as I understand and I think that must be a key reason for the different performance. I’ve seen a number of comments elsewhere suggesting that the linux virtual interface implementation / state machine is not as stable as it could be. I cant find the post now.

I’ve decided that in the absence of a deep understanding of the internals of linux networking and perhaps accepting that there’s either a hardware or software limitation here that cant give perfect results, my best interim step is try and make the software I’m writing more resilient to timeout issues and AP connection drop outs.

In that direction:

  1. The latest version of the networking module processes network status and scan requests in the background triggered by an initial http request to these respective API end points, a second request following the first request then fetches the result keeping processing time on the server side php to below 44ms which reduces timeout issues significantly on the PI Zero 2W.

  2. Another issue in addition to these short timeouts is that maybe 8 times out of 10 the PI Zero 2W looses connection to the AP completely after the WiFi client connection is initiated. The network interface UI provides feedback to say disconnect and reconnect to the emonPi AP to finish the process and see the newly connected WiFi IP address - and while not as satisfying as holding the connection all the way through it does reliably provide a way to finish the process.

Performance is generally much better all the way through on the Pi 4, which doesn’t appear to suffer from these timeout issues or the access point dropping out as it established the client WiFi connection and overall while it may not sound perfect the implementation is a significant step forward over what we’ve had in place up to this point.

The latest image build with the latest Network module can be downloaded here for anyone keen to test:
http://openenergymonitor.org/files/emonSD-01Feb24.zip

I will share screenshots and hopefully a screencast of what WiFi setup looks like in the next post.

2 Likes
41

Pi Zero 2W WiFi setup using hotspot / access point to connect to WiFi client:
Example includes reconnecting to the access point to complete and see the device IP address