[FuzzyJack]

Afternoon,

I tried to use the latest OP-Config with the new board from Torro and found that even though the latest version had the High DPI fix in it I still got the weird old formatting issue, which was strange.   So I had a bit of a tinker and I have upgraded the whole application to Qt 6.8.1 (Using Qt Creator 15).

I know on previous blogs there may have been some other reasons Luke hasn't previously upgraded the software level but I'm not totally sure what those reasons are.

I had to make a couple of small changes to the code to handle some deprecated functionality (QRegExp needed replacing and some of the currentIndexChanged functions no longer support QString) and I corrected a couple of items in the help files as well.

There was one initilization routine that was setting a QByteArray to values of '\0' that crashed the whole app.   I couldn't see why this was actually needed and commenting it out has made things work so that might need a look into later.

I had previously looked at using the QHelpEngine API for the documentation to get around using the Qt Assistant but it turned out it might be quite a lot of work to figure out how to open a new window with that in it so instead I worked our from the old InnoSetup post and a few other docs how to build the application and a include Assistant.exe and the required libraries as part of the installer.   I've scripted it and included all of the extra files needed to be able to produce the setup.exe automatically.

It's actually a bash shell script i run in gitbash on windows rather than a bat file but that was just because I could more easily do stuff like use which to find files on the path without having to figure out how to do that in Windoze.

I've branched from the master branch into my own repo and generated a pull request of my changes here:  https://github.com/OpenPanzerProject/OP-Config/pull/1

I need to do a little more testing with it I think but the core functionality works.  There are a couple of things niggling at the minute that I need to look at.

1) Connecting to the TCB directly is flakey and doesn't work very often at all.  There were a couple of small changes due to deprecation of functions in the SerialPort library so it might be related to that.   However, using snoop first and then connecting seems to work pretty much every time so that is the already known work around.

2) Downloading the latest hex for the TCB claims to work but "can't save" the file.   I need to look into that one.

I don't know what the numbering strategy is for version control at the moment and I figured a major upgrade of the software is probably big enough to increment to 0.94.0 which I have done for now but obviously that can be changed.

Hope that all makes some sense anyway and if anyone has chance to test it and find any issues let me know.

I have this working with the new TCB and need to fit it in to my Sherman but I may also be looking again at using some separate components and different sound options again.  Currently using the Taigen card and the engine running sounds are a bit more limited than I thought they might be.   Better than sounding like a Henglong Tiger 1 though!

[FuzzyJack]

Of course....   Blooming Windows Security.

By default the Flash function tries to download the HEX file into the /firmware folder of the current application.   Which when it has been installed using the OPConfig_Setup.exe a) doesn't exist and b) wouldn't allow a non system admin to write to it within elevating permissions.

It can be workaround by manually creating the firmware directory and granting write permissions to All Users.

Might be a way to get the InnoSetup to do this by default or set the firmware directory to a folder that does allow the download?

[LukeZ]

Hi Chris, I saw your pull request earlier and have been browsing the changes. Thanks for taking on yourself such a major challenge!

I myself probably have forgotten by now why I did or didn't do things in the past, but if I recall, some of the reasons I never updated previously was because of the loss of QtAssistant, and also because the one time I made a serious attempt, there were found to be some bugs in what was then the next version of Qt that caused other problems equally as bad as the screen scaling one. So it went nowhere, but the screen scaling issue has remained a pain for those users with high resolution monitors, and it if could be fixed it would certainly be a very worthwhile improvement. Nevertheless it has taken the energy of a motivated person such as yourself to move this forward. Indeed, yours is the first pull request received on any component of the entire Open Panzer project since its inception a decade ago, so I congratulate and thank you once more!

By the way, I see you posted a thread about QtAssistant nearly a year ago, which I somehow never noticed! My sincere apologies.

Anyway, I am going to install the newer version of Qt and see if I can compile your branch and do some testing. If I need to make changes I believe it should be possible for me to push to your pull request (or at any rate make a pull request to your fork), so we can continue development there until such a time as it's ready for merging.

It might take me a while to remember all the things I've forgotten which were necessary to even get Qt to run, but I'll keep you posted.

[FuzzyJack]

Hi Luke, yeah the high Dpi setting is deprecated in qt6 as it is supported by default so I removed that bit completely.

You will notice I removed a few other older bits like the doc creation scripts for the older versions.  I tend to find that leaving old versions around for a while is helpful but you rarely remove them again and they just hang around. Source control should handle that stuff for you so you can get them back if you need to.

I’m hoping you like the build script as well.  It didn’t take long to knock up and could be further improved potentially to also regenerate the help files and compile the code as well. 

I did have qt being able to build the release properly at one point but then it stopped working and I haven’t figured out why yet so I just used the debug version for now.

[FuzzyJack]

For the upgrade I just used the Qt Maintenance tool to install the 6.8.1 version and specifically adding the QSerialPort extra.

[LukeZ]

Hi Chris, I've wept and gnashed my teeth today and made some progress, all the ins-and-outs of which I won't bore anyone with. But I can compile the program, create a installation package, install it, and it seems to work.

I only have an Arduino Mega for testing, I don't have a TCB or even an RC radio. So for testing I am only able to confirm that I can connect to the processor (as you say, rather flaky this), load firmware, and read/write settings.

The problem I have yet to overcome and which I've given up on for today, is related to what you described as Blooming Windows Security, though in a different place.

The new version of Assistant (I am taking the assistant.exe from Qt\6.8.1\mingw_64\bin\, I assume this is what you did?) doesn't access the .qhc file directly that we so conveniently provide in the "help_files" folder, but rather as described on this page creates a copy of it in a "cache directory" which is a subfolder in "help_files," created when the Assistant is first called at runtime. Of course, "help_files" is a subdirectory of our program installation folder, and Windows does not permit the creation or modification of files within the Program Files directory, so this fails.

That link I provided tells us that we can modify the cache directory if we want, in the XML .qhcp file. However, that doesn't help us, it just changes the name of the cached subfolder, which will always be in the same folder as our .qhc file, it doesn't refer to an absolute path somewhere. Even if it did, there is no way I can see to point it to something like the Windows environment variable %LocalAppData% because this is an XML file and not a script, so that variable is not going to be parsed (I tried).

You'd think this would be a problem for the whole world but Google has not shown me other users who've struggled with this same issue. Maybe no one is using Assistant.

It seems strange you would not have experienced this same thing? Assistant works fine when I'm running from the debug/release folder created by QT Creator, because there we don't have file write permissions, but once installed in Program Files it doesn't.

The good news is it will not be a problem to divert our downloaded Hex files into the user appdata folder, though I haven't yet made that change.

I committed some changes to your fork (strangely I seem to have the permissions to do so), these are just fixing some fonts and formatting that seem to have been changed by the new Qt Creator.

[FuzzyJack]

Hmm yes it is strange I don't seem to have the problem but then I also have the Qt full installation on that machine.

If I get a minute I might launch a VM and see what happens with a clean build without Qt or Op-config already installed.  I did use the uninstall a couple of times during testing to remove the old one but a clean build might help.

Maybe I have done something different in the way I built the installation package using my little script that avoids that particular problem?   My install includes an extra "help" folder that your original install didn't have when I create it using windeployqt.

The only file in that directory is a helpplugin.dll file so I can't see how that would make a difference.

I have uploaded my setup.exe to a OneDrive folder and will provide a link so you could possibly try with that and see if it makes any difference?  Maybe I built it differnently somehow?  Maybe it's just windows security being a pain.

As you say, downloading the hex file to a more accessible folder (perhaps even the users download folder or provide them with a field to supply a location ought to be easy enough to work around that one.

[FuzzyJack]

https://1drv.ms/u/c/c6afd11e9584c0c7/EQYc4q5Zkn5FvL3u7hPV_w0BIa_wYkzV2tHIXz6RZCfojw?e=aeWQUU

[FuzzyJack]

Damn...  Maybe I didn't try the help file from the proper install or something as I just reinstalled and tried it again and I get a failure too.  I could have sworn I checked the help after running the install but maybe I did it without running an uninstall first or something and already had permissions or an old copy of the folder?   Not sure.

Attached the error I got and if I grant write permissions to All Users on the help_files folder and restart then it can create the required files and works ok.

I must have missed it first time.

[FuzzyJack]

According to the docs you can specify the cache folder location in the configuration for the collection and QtAssistant is supposed to create the cache in the location supplied by %QDesktopServices::AppDataLocation%.

https://doc.qt.io/qt-6/assistant-custom-help-viewer.html

At the moment the cache is set to:

        <title>OP Config Help</title>
        <applicationIcon>images/helpicon_32.png</applicationIcon>
        <cacheDirectory base="collection">OpenPanzer/OPConfig</cacheDirectory>

But it is definitely not trying to create it in the appdatalocation unless something is overriding it?

Obviously another workaround for this issue (and the hex download) is to run op-config as Administrator it can then do what it needs to to create the cache files and the firmware folder without needing to manually grant permissions.

There must be a way to get the Qt App to work in a more security friendly way though.

[LukeZ]

The problem with %QDesktopServices::AppDataLocation% is that it appears to be deprecated so it's hard to test what exactly it thinks that should be. No doubt it's using the new QStandardPaths collection but the documentation hasn't been updated so that adds to the confusion.

But I had some ideas after sleeping on it. I tried a few things and got it working, and only after I got it working, did I find in the documentation where it would have told me how to get it working. 

If we empty the value in the XML cache directory element like this:  <cacheDirectory base="collection"></cacheDirectory>, or if we just remove it entirely (the documentation implies it's optional), then regenerate our .qch and .qhc files using the createqhcp.bat script of course, amazingly now Assistant works. But there is no subdirectory created in Program Files, I had to search high and low for where it had put the cache file, which turns out to be:
C:\Users\*username*\AppData\Local\QtProject\Assistant\

That's definitely a better place to put it, though not very intuitive.

Then I got to wondering why I had that strange attribute base="collection" in the XML tag. So I removed that attribute but left the rest, like this:
<cacheDirectory>OpenPanzer/OPConfig</cacheDirectory>
...and then regenerated and tried again. Guess what, now it sticks the cache file here:
C:\Users\*username*\AppData\Local\OpenPanzer\OPConfig\

Exactly what we want. I still didn't understand why that fixed it, but knowing what the problem was, I was able to find the answer. If we go to the same link I provided before, and scroll down even further, there is actually information about this base attribute. If it is specified as "collection", that tells it to create the cache file in a directory relative to where the original collection resides, which in our case is program files. If set to "default" or nothing, it goes to AppDataLocation. Not sure why I had used the first option, but that was ten years ago, what did I know then!

So that is cleared up. I committed a change to your repo, I would be grateful if you could test to verify that it really works!

[FuzzyJack]

Good find Luke,

Yes I can confirm that it works fine for me and creates the cache files in the AppData folder structure for my user as it should.

The firmware download still tries to use Program files of course but we should be able to use the QStandardPaths to find something like the downloads folder to do that as well and problem solved!

e.g.

QStandardPaths::DownloadLocation   14   Returns a directory for user's downloaded files. This is a generic value. If no directory specific for downloads exists, a sensible fallback for storing user documents is returned.

I've tweaked the SaveWebHexToLocal function to use the "writableLocation" of the Downloads folder and it works fine :-)

It still creates the firmware directory if it doesn't exist and deletes an existing copy of the file before saving it.   Obviously once it is in the downloads and used it is the responsbility of the user to delete it afterwards if they want to.

Which does beg the question of whether an uninstall should leave the AppData/Local/OpenPanzer folder hanging around or not so  I might see if the InnoSetup uninistaller would be able to remove that as part of the uninstall.  Probably leave the downloads alone though.

[LukeZ]

Thanks for confirming, and also for fixing the Hex downloader. Since we're putting them now in the user's Downloads folder, it seems that we should put them in a folder with a better name than just "firmware," so I've changed that to OpenPanzer.

That was a good idea about InnoSetup removing those other files, and indeed that wasn't hard to do. I've gone ahead and had it remove both the Assistant cache as well as the hex downloads. I guess one could go either way on the second one, but if it were me I'd rather not have junk files laying around. Every hex can easily be downloaded from the website so it's not like the user can't recover them, even without OP Config.

I've committed these changes to your fork and a few other minor things.

If you make any other changes go ahead and also commit them to your fork rather than the source OP Config repo, and when it's stable I'll pull in the entire request.

Right now my setup is really crazy as I've kept the old version of Qt and everything else alongside the new, for dev purposes. But I'll clean it all up when we do the final over-write. 

Are there other issues that we should address? I know there is the flakiness of the serial connection, that has always been an issue and I can revisit it but I don't think it is affected one way or the other by the upgrade to Qt6.

I should also say that I don't actually have a high-resolution monitor, so I can't really see what the program looks like in regards to the problem we were originally trying to fix. I assume it does look better?

[LukeZ]

Hi Chris, I made a very slight change to the serial connection process which seems on my end at least to improve the reliability of connections to the board. I've committed them to your repo, if you want to test it out.

[FuzzyJack]

As far as I can tell all you have slightly increased the Blitz Timer and remove the specific call to SendInit().

Previously it was a bit strange that it setup the timers and then called SendInit() and the first thing the Blitz timer did was call it agian and I wasn't totally sure why it needed to do that.   Now without calling SendInit() delliberately it seems to connect perfectly every time now which is soo much nicer!

It also connects ok from the snoop still as well so nothing changed there.   It seems to work brilliantly so that's a good improvement.

As for the High DPI and the screens themselves I am using an old MacBook Pro (2015) with a Retina display running at 2880 x 1800 and with the last few tweaks you have made the fields look pretty good.

Obviously with the older version and the non High DPI setting the screen was all over the place with fields overlapping and text fields being output in all sorts of font sizes and it was mostly unusable like that.  With the settings now in Qt6 it looks really good.

I've attached shots of each screen just for reference.  They all look really good to me.

I need to build a small conversion board from my current Henglong Sherman wiring now to interface with the TCB (I have a 360 degree rotation slip ring to the turret connections so I don't want to directly rewire the Airsoft and will do it with a small PCB and some XH/PH connnectors instead so that it is reversible.   Once I have that sorted and wired in I will do some more testing
and see if there is anything else I can think of.

Chris.