Presence of libjnidispatch.jnilib triggers Mac notarization failure

[rschmunk]

I am trying to finish packaging up a copy of Panoply for macOS using netCDF-Java 5.3 SNAPSHOT. Everything is great right up until almost the end when I have to get the disk image DMG notarized by Apple. That fails because of the presence of libjnidispatch.jnilib in the NJ jar, i.e., netcdfAll-5.3.0-SNAPSHOT.jar/com/sun/jna/darwin/libjnidispatch.jnilib. In short, the notarization process is complaining that the libjnidispatch.jnilib binary is not properly signed.

Note that this problem did not occur last Friday (January 31) when I was making a Mac package, but I have a dim recollection that Apple was going to be closing some loophole for notarizing Java based apps effective February 1. Or perhaps that was apps built on an an old Java, such as the Java 8 that NJ is based on.

I realize that dealing with this is pretty much outside Unidata's purview, but it's something that could completely break my ability to distribute a netCDF-Java based app to macOS users with an up-to-date operating system. But I am wondering what libjnidispatch.jnilib is used for in NJ, and whether it's really necessary?

ETA: I see that JNI is necessary in order to use the library to write NC4 files. A quick and dirty test suggests that removing the offending jnilib from the NJ jar does not cause any breakage when opening and reading datasets, but I will need to test that more throughly before relying on it.

[lesserwhirls]

Greetings @rschmunk! Yes, the libjnidispatch.jnilib is coming in from the netcdf4 module, which writes netCDF-4 files via the C library, and can also read netCDF files through the C library for debugging purposes. It's actually JNA that is pulling in libjnidispatch.jnilib. That code should never be touched unless the netCDF-Java is explicitly told that netCDF-C exists and should be used, so it should be safe to remove.

It looks like Apple flipped the switch on their relaxed notarization prerequisites, and that's causing lots of headaches. I'm not sure if you ran across this stackoverflow post, but it describes what they had to do to sign native libraries bundled within a jar.

[rschmunk]

@lesserwhirls, So if .unidata/nj22Config.xml has been read in so that the user can use libnetcdf, would that jnilib be needed? At some point a few years ago due to a user request, I modified Panoply to look for that XML file and if found, then feed it to RuntimeConfigParser.

[lesserwhirls]

In that case yes, libjnidispatch.jnilib would be needed as it allows the JNA library to make calls to the netCDF-C library.

[rschmunk]

Hmmmmmm..... Well, I only know of the one user who might have specifically taken advantage of this feature in Panoply, so it might have to go away until NJ uses a newer JNA. Or perhaps I'll bundle a version of NJ missing JNA in the Mac version only.

[lesserwhirls]

I get the feeling that the second option is what will need to be done, at least for now. The JNA developer does not seem interested in signing the Jars for Mac (or Windows, for that matter), so it looks to be falling on the individual developers utilizing JNA to sign as needed. (which invloves expanding the jar, signing, rejarring, etc.). The IDV group at Unidata does some signing, but I'm not familiar with how it works myself, and I believe they rely on Install4J to manage all of that. I'd be happy to put you in touch with one of them to see if they have any tips/tricks.

[rschmunk]

Thx, @lesserwhirls. I'll figure something out. Simplest option for me is to just disable JNA, but as long as I can script whatever I implement I'll be satisfied.

I'm an odd outlier in that I use Xcode as my IDE and ant for my build scripting. (Hey, I only started using ant maybe 7-8 years ago!) The Mac notarization process has messed that up as my process for building, signing and uploading was for several years a single ant command-line call. Now that has been replaced by two CL commands with a quick coffee break in between, assuming no notarization failure.

[rschmunk]

I discovered an issue in the java-native-access/jna#1154 repo about problems submitting apps using Java and JNA to the Mac app store beginning late last year, which was being triggered by the JNA code for Darwin being fat (32- and 64-bit) and the app store no longer accepting any submission that included 32-bit code.

One commenter on that issue showed how to strip the 32-bit code out of the Darwin JNA libjnidispatch.jnilib. So after un-jarring netcdfAll.jar, stripping the 32-bit code out of that jnilib, and re-jarring, then packing up Panoply and submitting for notarization, I got... successful notarization!

That said, I have no idea whether use of the nj22Config.xml is actually working on a Mac using this modified jnilib, as I do not have libnetcdf installed and am hesitant to get into that. There is no complaint from NJ's RuntimeConfigParser when I feed it a nj22Config.xml file, not even a whine that no library exists at the path in my dummy nj22Config.xml.

[lesserwhirls]

Nice! I wonder if we could do some jar shading magic in our build once the two mac platforms are separated out, and make the new darwin-x86 platform stuff optional somehow (could we call it netCDFAll.jar if it didn't include, uh, all though?).

[rschmunk]

Has any further thought been given to producing a netcdfAll.jar that does not include the 32-bit Darwin native binary code?

This is not a priority for me now, as I have set up a script that builds netcdfAll, then unjars it, strips out the offending binary, and re-jars everything back up. Nevertheless, I'm curious, and so wanted to see if it might be in the queue somewhere.

Also wondering to what extent are there still Darwin boxes out there that are 32-bit only?

[lesserwhirls]

It looks like jna still ships the universal binary for darwin, so we would not be able to shade out out the 32-bit specific parts on our end just yet. Once jna splits the universal binary into a 32-bit and a 64-bit version, then we'd be able to work some gradle magic. I have no idea as to the extent of 32-bit darwin systems, or the use of 32-bit java on 64-bit darwin systems (which would still require the use of a 32-bit lib).

[rschmunk]

@lesserwhirls, I can dig it. The only negative in this is that it means I can forget about downloading a netcdfAll.jar release binary and plugging it into my app. But as the majority of my releases the past year have been using SNAPSHOTs, that's not really a problem at the moment.

[julienchastang]

@rschmunk speaking of notarization, we are starting to look into it for the IDV. Do you, perchance, have any advice or suggestions for us as we embark on this effort? I'm speaking more broadly about the notarization of a Java client application, not specifically about this issue. Did you ever successfully notarize Panoply? Thank you. (ping @yuanho).

[rschmunk]

@julienchastang and @yuanho, Panoply releases for macOS since IIRC late 2019 have been all been notarized, as have a couple of my other Java apps. There have been a few times, such as the case described in this issue, when I had to tweak the process because of Apple tightening the screws a little more.

Aside from the occasional update, the only problem I have with notarization now is that it used to be that I could trigger a compile, build, codesign, and upload of an app in a single command. Now it's a two-command process with a 5-10 minute coffee break in between while I wait for notarization to process. (Well, three steps at the moment due to WFH and dealing with the office VPN, but two steps whenever I'm allowed back in the office.)

I have a few notes on the notarization process for a Java app, mostly reminders to self rather than a detailed step-by-step menu. I could probably flesh that out. You'll note my workplace in my profile; you can e-mail me at the appropriate address.