I just purchased StellarMateOS (1.4.2), wrote the image onto a 16G sd card and run the system on a RPi3. Everything looks good so far when using my (Debian-based) laptop -- I can connect to the stellarmate hotspot, I can SSH into the OS, I can access the desktop using VNC and I can conjour the INDI web manager.
Alas, the Android app (beta 1.9.11, Bundel v23) won't find the device, although my phone (Galaxy S9 running LineageOS) successfully connected to the stellarmate hotspot and I successfully logged into my stellarmate account (I think -- does the app give any feedback about login state? Is there any [Log out] button?)
Hmm... The stellarmate "Initial setup" page at www.stellarmate.com/support/first-time-setup.html says: "When using the App for the first time, you are required to sign in to your stellarmate.com account. This is required in order for the app to download your licence and serial numbers necessary for the device registration. This is only performed once and requires active internet connection." Somewhat further down on the same page it is stated that "After scan is complete, the left side panel should contain a list of detected StellarMate. Click on the detected StellarMate controller. [...] Before the app can be used with the device, it (<- the device, I suppose -- not the app?) must be registered first. The registration information should be automatically filled in" So if I understand correctly, a device must be found *before* the registration key comes into play.
Regarding KStars, I don't remember exactly if I run a device scan before I closed KStars through VNC. But I'm pretty sure that I run some more scans after starting INDI through the web manager. Since KStars -- at least to my knowledge -- doesn't provide any network service by itself, I suppose INDI is what counts. (But I may be wrong here -- I'll try again scanning for my RPi with KStars running.)
Did you ever enter your Stellarmate OS license key in the android app? Step 3.2 on the page you linked mentions it. When I first set up the app on my phone a few weeks ago, it prompted me for the license key along with my username and password. I believe the app needs to be associated with a purchased device (or in your case, the OS.) Maybe this step has changed since I first did it, but without the app being able to verify that it has a valid device, it probably won't connect. Otherwise, people could use the app to connect to any kstars/INDI implementation.
If you did enter the license key and it's still not working, I'd open a support ticket on the SM site.
The attached screenshot is the "registration" screen in the android app. I had to fill this out the first time to register the app to the device. If you don't see the register button, I'm not sure what's going on.
I really must have a different version of the app -- there is no [Register] button anywhere to be found! I also tried re-setting all app data again, but no such button appeared (not immediately and not after re-starting the app).
Well, I submitted a ticket at stellarmate.com -- let's see what they have to say about this.