Add ARM64 support

[Mattiwatti]

This PR adds ARM64 support to Open-Shell. See also #553 where this feature was requested.

The changes are not spectacularly interesting and mostly consist of adding a third supported platform to the two existing ones. The only part I'm not quite sure about is the FNV hashes of the MSIs that go into the setup; while it seems to work (the installer works on all three platforms for me) it probably wouldn't hurt if someone double-checked this.

Note that WiX has only added support for ARM64 in v3.14.0.3910. This means I've had to raise the required version used in the build script and documentation from 3.7 to 3.14. Edit: and I see the AppVeyor build failed because of this.

[AppVeyorBot]

❌ Build Open-Shell-Menu 4.4.163-thbuuqbs failed (commit c15d6ff756 by @Mattiwatti)

[ge0rdi]

Note that WiX has only added support for ARM64 in v3.14.0.3910

Unfortunately it seems that AppVeyor images contain 3.11 only :(
Maybe there is some way to ask AppVeyor to update it?

[ge0rdi]

Maybe there is some way to ask AppVeyor to update it?

Actually WiX 3.14 is just development build, not stable one.
So I don't think AppVeyor will add such.

So I guess for now we cannot merge this to master.

I'd suggest to not build ARM64 by default. Just add some possibility if one would like to try it.
Once WiX ARM64 support will get stable and into AppVeyor images we can enable ARM64 builds by default.

[ge0rdi]

Looks good to me 👍
Pity that WiX ARM64 support is not in stable yet :(

[AppVeyorBot]

❌ Build Open-Shell-Menu 4.4.164-eurgbsui failed (commit 026dc8198a by @Mattiwatti)

[Mattiwatti]

Re: the AppVeyor build: I agree that it's undesirable to have a build batch file that doesn't work on AppVeyor by default. However I see some problems with the 'optional parameter' approach: the build of Setup.sln will fail if the ARM64 MSI is not built, because a required file will be missing to compile the installer resources.

An option could be to make a separate 'Release with ARM64' configuration for the Setup project, so that the default Release configuration includes only the x86 and x64 MSIs, and only the new configuration would include all three MSIs. But this would require some additional changes to the Setup project.

I see another alternative that may work better: while AppVeyor does not support WiX 3.14, it does provide access to curl which can be used to download files. We use this at ScyllaHide to download dependencies that we cannot include on the repository for legal reasons. I've pushed a new commit to test this. Assuming this works, let me know which approach is preferable.

[AppVeyorBot]

✅ Build Open-Shell-Menu 4.4.164-afoefmgi completed (commit 4a9f50d3d1 by @Mattiwatti)

[ge0rdi]

Assuming this works, let me know which approach is preferable.

Well, I personally don't like idea of using devel version of software just to support very small amount of users.
I also don't like the idea of adding ARM64 version to common installer by default. As it almost doubles its size (from 7MB to 12MB) and vast majority of users will have to download unnecessarily bigger installer (in fact I don't even like the idea of combined x86/x64 installer, but that's another story).

These are my personal views, maybe @coddec, @XenHat have different views on this.

But to be constructive, what about this:

We can keep installer as it was (x86 + x64) and produce separate ARM64 installer (basically just MSI should be enough).
We can push ARM64 as separate artifact to AppVeyor so ARM64 users will be able to test with it.

Only ARM64 build will then use devel version of WiX (that can be downloaded when needed as you proposed).

What do you think?

[coddec]

Assuming this works, let me know which approach is preferable.

Well, I personally don't like idea of using devel version of software just to support very small amount of users.
I also don't like the idea of adding ARM64 version to common installer by default. As it almost doubles its size (from 7MB to 12MB) and vast majority of users will have to download unnecessarily bigger installer (in fact I don't even like the idea of combined x86/x64 installer, but that's another story).

These are my personal views, maybe @coddec, @XenHat have different views on this.

But to be constructive, what about this:

We can keep installer as it was (x86 + x64) and produce separate ARM64 installer (basically just MSI should be enough).
We can push ARM64 as separate artifact to AppVeyor so ARM64 users will be able to test with it.

Only ARM64 build will then use devel version of WiX (that can be downloaded when needed as you proposed).

What do you think?

I agree with you @ge0rdi
In fact, It's not rare to see people distribute installers for different architecture, e.g. installerxx_x86.exe installerxx_x86.msi installerxx_x64.exe installerxx_arm.exe etc. You get the idea. So I'll support the idea of separating installers for different architectures if that won't introduce any trouble/huge work load etc.
It saves bandwidth, storage etc. not sure if there's any negative sides.
For ARM, suggestion from @ge0rdi seems fine to me, what do you think? Idea of separating the installers for different architectures and ARM? @XenHat @Ibuprophen

[Mattiwatti]

Thanks for the feedback. Summarizing, so far it seems like the way forward for ARM64 is to make a separate installer (and maybe even split x86 and x64 into separate installers as well? I'll leave this up to you guys). This is fine by me, and as has been pointed out it will reduce the download size for both binaries.

What should be the preferred way to build the ARM64 setup? Should there be (a) new batch file(s) to create it (think e.g. __MakeFinalARM64.bat and __MakeFinalAllLanguagesARM64.bat), or should there be some argument that needs to be passed to the existing batch files that determines which version to build? If the latter, I'm assuming the 'no arguments' case should still build the x86 and x64 combined setup, correct? The reason I'm asking is because presumably it is desirable to make AppVeyor produce automated builds of ARM64 as well as x86/x64.

If the answer to the above is 'don't worry about this, someone else will take care of the installer', or 'do this in another PR', that's fine by me too. I mostly wanted to make this PR to add ARM64 support to Open-Shell, not necessarily Open-Shell's installer :) Though I'm OK with doing it if this is desirable.

[ge0rdi]

@Mattiwatti
First of all thanks for understanding and I personally appreciate the effort done (as there is not much people contributing to this project).

Should there be (a) new batch file(s) to create it (think e.g. __MakeFinalARM64.bat and __MakeFinalAllLanguagesARM64.bat), or should there be some argument that needs to be passed to the existing batch files that determines which version to build?

I think separate BAT may be easier to do.
But I guess combined script where one can select what should be built will make more sense.

Basically from my POV both ways make sense.

If the latter, I'm assuming the 'no arguments' case should still build the x86 and x64 combined setup, correct?

Yup. That would make sense, so it will be backward compatible.

Though in case when some argument will be present it probably should mean list of platforms to build.
So supplying ARM64 will build ARM64 only and supplying x86 x64 ARM64 will build all platforms.
What do you think?

The reason I'm asking is because presumably it is desirable to make AppVeyor produce automated builds of ARM64 as well as x86/x64.

Right.

Depending on the way the ARM64 build will be integrated we can modify appveyor.yml to build all 3 platforms.

[Mattiwatti]

I've pushed a new commit that separates the x86/x64 installer from the ARM64 one instead of the previous iteration where the setup was combined. The new ARM64 setup can be built with __MakeFinalARM64.bat or __MakeFinalAllLanguagesARM64.bat, which is equivalent to __MakeFinal.bat ARM64 or __MakeFinalAllLanguages.bat ARM64. So in principle the new batch files are not strictly necessary, since supplying "ARM64" as the optional argument will build the ARM64 setup. But I thought this way would be more obvious to newcomers who are wondering how to compile for a different architecture.
__MakeFinal[AllLanguages].bat without arguments will build the combined x86/x64 installer as before.

Though in case when some argument will be present it probably should mean list of platforms to build. So supplying ARM64 will build ARM64 only and supplying x86 x64 ARM64 will build all platforms.

I agree this would be preferable, but I'll leave this as a "nice to have" TODO for now. I spent some time trying to make this work for generic lists of architecture names (e.g. "x86 x64 ARM64 foo"), but with all the added batch file boilerplate and error checking the batch files (plural, because there are multiple batch files that all need to know which architecture(s) is/are being built) quickly became very unwieldy with most of the lines "wasted" on simply parsing the architecture(s) to build. So I reverted to the simpler method above for now.

[AppVeyorBot]

❌ Build Open-Shell-Menu 4.4.164-rmrbgmsh failed (commit 4d3e1e68f3 by @Mattiwatti)

[AppVeyorBot]

✅ Build Open-Shell-Menu 4.4.164-papqqypf completed (commit 60fa7e3a64 by @Mattiwatti)

[XenHat]

Late to the party here. I agree with the sentiment that architectures should be split. What's the status on this? Waiting for Wix to be stable?

[malxau]

I just bought an ARM64 device and built from Mattiwatti's branch to install. Is there anything I can do to help move this forward? Getting it installed from source worked in the end but was a rough experience (I've never built this code before.)

The only thing I encountered is Utility.rc is trying to include the AMD64 binary (so the x86 binary includes the AMD64 one?) In this build sequence though, BuildBinaries.bat only builds ARM64, not AMD64, so the later x86 build fails because it can't find the AMD64 binary.

[Ibuprophen]

I'm neutral about having separate installers as long as it's necessary to have...

I'll go with the consensus of the team @ge0rdi, @coddec, @XenHat...

~Ibuprophen

[blackcrack]

@coddec , could Ibuprophen have the "ownerstate" because he is active here and he is everytime here, have a good character and know what he does..
best

[AppVeyorBot]

✅ Build Open-Shell-Menu 4.4.196-erscjexb completed (commit 625484d307 by @ge0rdi)

[ge0rdi]

Oki, so I have merged current master to this PR and resolved conflicts.
I have also changed the build process to build ARM64 binaries. Also ARM64 binaries are now part of standard installer (it added ~2.2MB to the size).

Everyone interested in testing on ARM64 devices, please feel free to use installer from:
https://ci.appveyor.com/project/passionate-coder/open-shell-menu/builds/51987348/artifacts

And again one big thanks to @Mattiwatti who started this and did all the hard work.

[AppVeyorBot]

✅ Build Open-Shell-Menu 4.4.196-ufhncqhu completed (commit 82cf68f944 by @ge0rdi)

[AppVeyorBot]

✅ Build Open-Shell-Menu 4.4.196-rikfexxu completed (commit 3cce8324e7 by @ge0rdi)

[ge0rdi]

There is another build here:
https://ci.appveyor.com/project/passionate-coder/open-shell-menu/builds/51989416/artifacts

Please, use this one for testing.

[ftc2]

There is another build here: https://ci.appveyor.com/project/passionate-coder/open-shell-menu/builds/51989416/artifacts

Please, use this one for testing.

OpenShellSetup_4_4_196-rikfexxu.exe does not run for me.

Edition	Windows 11 Pro
Version	24H2
OS build	26120.961
Experience	Windows Feature Experience Pack 1000.26100.8.0
Processor	Apple Silicon   3.20 GHz  (6 processors)
Installed RAM	48.0 GB
System type	64-bit operating system, ARM-based processor

[ge0rdi]

@ftc2 Could you try to run installer with extractARM64 parameter from command line?
That should extract ARM64 installer to actual folder.
Does that work?
And does installing such extracted MSI work?

(sorry, I have no ARM64 machine at hand atm)

[malxau]

For what it’s worth, the installer works correctly for me on 23H2/22631. The installer stub is x86 but it correctly found and installed the arm64 MSI. I’m quite surprised by that error – I think it’s from pcasvc indicating a known appcompat problem, not an error indicating a corrupt binary or unsupported architecture. But if that’s true it should affect any Wix 3.14 installer on 26100 arm64.

[ge0rdi]

@ftc2 You also seem to be on beta Windows channel, but rather old build. Maybe try to update to latest build to see it that helps?

[ftc2]

i was amused to see that the 'Learn more' button opens https://go.microsoft.com/fwlink/?LinkId=2268343 which 302s to https://open-shell.github.io/Open-Shell-Menu/.

anyway, i opened up terminal running powershell and tried these:

.\OpenShellSetup_4_4_196-rikfexxu.exe --extractARM64
.\OpenShellSetup_4_4_196-rikfexxu.exe /extractARM64

is that how you run windows software with params?

neither worked. both gave the same error as in my last post, and neither extracted anything to the working dir or to the exe's dir.

i tried to extract the msi manually but couldn't figure it out. i unpacked the exe:

.rsrc
├── GROUP_ICON
│   └── 101
├── ICON
│   ├── 1.ico
│   ├── 2.ico
│   ├── 3.ico
│   ├── 4.ico
│   ├── 5
│   ├── 6
│   ├── 7
│   └── 8
├── MANIFEST
│   └── 1
├── MSI_FILE
│   ├── 132
│   ├── 164
│   ├── 165
│   └── 167
├── string.txt
└── version.txt

renaming those files in MSI_FILE to foo.msi didn't work. do they need to be catted together or something?

You also seem to be on beta Windows channel, but rather old build. Maybe try to update to latest build to see it that helps?

good eye. it would not surprise me at all if that's the issue. the build's not that old – it was fully up to date a little less than a year ago when i installed it and then ran windows update. it's an airgapped vm (lol) and has to be that way, sorry. that's all i have for now.

i have not had trouble installing any other software on this thing, and while i feel it should be supported, i'm ok with just assuming it would work on a slightly more standard installation of windows if you are. maybe revisit the issue if someone else complains? i'll just rock @malxau's build on this vm.

if you really want, i can make a new vm with the latest win11 arm64 image test it again, though. lmk.

[ge0rdi]

is that how you run windows software with params?

Just use the parameter as I wrote without anything else:
.\OpenShellSetup_4_4_196-rikfexxu.exe extractARM64

the build's not that old

Hehe, funny how relative that is. To me a year old Windows is very old ;)

if you really want, i can make a new vm with the latest win11 arm64 image test it again, though. lmk.

Well, it is up to you how much do you want to test this and spend your time with it.
I'm glad for any help here.

[ftc2]

ok, i'll make a new vm in the next day or so :)

[ftc2]

Just use the parameter as I wrote without anything else:
.\OpenShellSetup_4_4_196-rikfexxu.exe extractARM64

same result as my other attempts (didn't work)

[ge0rdi]

I got access to ARM64 machine with Win11 24H2 and getting the same PCA compatibility issue dialog.
So I can investigate now (hopefully).

[ge0rdi]

Windows (24H2 so far?) seems to have some compatibility block for OpenShellSetup*.exe (and hopefully some other conditions).
Most likely because our installer didn't work on ARM64.

It was enough to rename the installer (OpenShell_4_4_196.exe) and then it was allowed to run.

Trying to figure out how to deal with this (without changing the name of installer obviously).

[ge0rdi]

I guess we will have to change name of installer (probably no big deal).
Not sure how to contact MS to lift such hard block (for future versions).

[ftc2]

haha
so my version of windows was actually too new? ;)

[malxau]

Yes. The block was added April 2024, so I don’t have it. It looks like StartMenuHelper64.exe (Amd64) was being injected into Explorer on Arm64 systems, causing Explorer to crash, hence the block. I’ve started an email thread to find the best process to alter or remove it.

[ge0rdi]

I guess we will have to change name of installer (probably no big deal).

Hmm, this will actually break built-in updater. As it looks for OpenShellSetup* asset on releases page.
So for now I will keep the name as it is.
And rather add some notice/warning on releases page that on ARM64 it may be necessary to rename installer before starting it.

I’ve started an email thread to find the best process to alter or remove it.

Fingers crossed.

[AppVeyorBot]

✅ Build Open-Shell-Menu 4.4.196-bsgluqcx completed (commit 1788aedcbc by @ge0rdi)

[malxau]

I think the advice is to release an ARM64-capable version and let them know the version number so anything above that can be excluded from the check. As you’ve seen the current block does not have any version restriction, so this still sounds like it will require a Windows Update. On the other hand, note the ARM64 experience doesn’t change – currently users have a block, once an ARM64 build is released they’ll have a block (with a workaround) and it’ll probably take a couple months for the block to be relaxed.