Make UI module independent sub-project

[malliaridis]

See discussion https://lists.apache.org/thread/onc0zyyp8tjoj3fdg3bmpnh9cqsgjhy6

Description

We have noticed slow build times and problems related to the Admin UI module that requires developers to disable the module entirely, which led to other dependency resolution issues.

This PR attempts to separate the module from the rest of the project by making it an independent sub-project.

Solution

The approach that was picked is a completely independent gradle project. Some important changes:

• Dependency resolution for the UI module is happening separately from the rest of the project, so dependencies are not aligned
• Many build configurations that apply to most solr modules are removed, allowing the UI module to use simpler builds
• The exclusion of many plugins and configurations means there are no default checks included

We should evaluate before merging which of the configurations or routines we want to keep and apply to this module as well. We may also keep it as is since the Admin UI is still experimental.

The module is no longer included as a configuration or task dependency as implemented before. Instead, the webapp now has to fetch the UI artifacts from https://nightlies.apache.org/solr/ui/wasm/[commit-hash]/.

Developers can call ./gradlew dev -Psolr.ui.buildFromSource=true to build the UI module locally and include the artifacts from the current state, or alternatively define a commit (and optionally a checksum for validation) in the gradle/solr-ui.properties to download the UI bundle from the nightlies server.

A guard has been added for specific release tasks that prevents using a nightly build artifact, and forces the UI to be built from source.

The jenkins job https://ci-builds.apache.org/job/Solr/job/solr-ui-wasm-assemble/ is building and deploying a version of the project. Right now it is executed manually on this branch, but it can be updated once this is merged to run from main.

I chose running the jenkins job manually so that we can pin down a commit hash and avoid spamming the nightlies with bundles. But we can switch to daily builds too if we prefer that. THe zip size is about 5MB large.

Important Note: This PR introduces a new flag for building the UI module from source, replacing the previously configured production flag in gradle.properties. The flag defaults to false requiring the user to either set it to true or provide a commit hash for download wasmJs. The PR uses the hash from the PR branch, which will no longer match a commit hash on main once this PR is squashed and merged.

Tests

The changes can be tested by adding these lines to your gradle.properties file:

solr.ui.buildFromSource=false
solr.ui.commit=473a55ef64a9a1605e00eb6c1c386887d4cb7dac
solr.ui.wasmSha256=654494303e4af3b8bfb26a83d250603f19033820295591d88db0597991ce8ad2

And run ./gradlew dev. This should download the zip file from https://nightlies.apache.org/solr/ui/wasm/473a55ef64a9a1605e00eb6c1c386887d4cb7dac/ and build the webapp with a pre-built version of the new UI, while verifying the download with the provided checksum.

Checklist

Please review the following and check all that apply:

• I have reviewed the guidelines for How to Contribute and my code conforms to the standards described there to the best of my ability.
• I have created a Jira issue and added the issue ID to my pull request title.
• I have given Solr maintainers access to contribute to my PR branch. (optional but recommended, not available for branches on forks living under an organisation)
• I have developed this patch against the main branch.
• I have run ./gradlew check.
• I have added tests for my changes.
• I have added documentation for the Reference Guide
• I have added a changelog entry for my change

[dsmiley]

I think we should use the same gradle wrapper, at least.

It's not clear how new master Solr build acquires the UI artifacts to serve.

[malliaridis]

It's not clear how new master Solr build acquires the UI artifacts to serve.

It is not clear because it is not solved yet. 🫤

I haven't figured it out yet how to do that the proper way (e.g. where can we publish artifacts, how to build it in advance for a release, how to retrieve artifacts?). I wanted to look into the solution we use for lucene and see if it can be reliably used for the UI in releases as well.

Another thing that I am unsure is all the licensing stuff. We would have to treat it completely different. I believe the best option we have is to display open-source licenses in the UI somewhere, but for that we would have to retrieve them first.

All the project configurations are removed, which is both a blessing and a curse as you can see.

If you have any input on how to solve specific issues, feel free to leave a comment or commit to this PR. Happy to colaborate on this one. :)

[janhoy]

Does this need to be a new top-level folder? The old prometheus-exporter was a separate build but lived under solr/. So we could let the ui live under solr/ui/ but still disconnect it from the main build. Less top-level clutter. The UI is still a part of Solr and it is the solr ui, so nothing funny with solr/ui as location?

[janhoy]

I like the convenience solr.ui.buildFromSource=true for development.

During a release the UI will need to be built before the main tarball and then included in the main tarball. But do we need to release the UI in downloads.apache.org as well? I don't think so, as long as it is evident that the embedded wasm UI was built from the same commit as the main server.

Having a nightly CI job a pre-built UI assembly to nightlies for the common branches provides a nice default fallback for those who just want to play around with local build. If you do cutting edge UI dev or if you want to build a PR branch or an older commit than latest nightly, you may need to re-build UI locally anyway.

[malliaridis]

But do we need to release the UI in downloads.apache.org as well?

I believe for the jvm standalone client it would make sense to publish it to downloads.apache.org. For the wasmJs that is included in the Solr distribution I would add a guard where the release manager is forced to build from source / same commit. This will avoid the need of managing just another release voting for UI publications, right?

If you do cutting edge UI dev or if you want to build a PR branch or an older commit than latest nightly, you may need to re-build UI locally anyway.

Yeah, devs working on UI or people testing newest changes can always build from source. The rest can pick a commit that has been published on nightlies and bypass the build hassle.

I haven't finished nor tested the changes yet, as I have to update the jenkins nightly build to see if the fetch works as expected. I will also update the directory if it is preferred to be placed under solr/. Thanks for the early feedback!

[malliaridis]

In another PR I will enable spotless checks on the UI module and address the indentation to change it to 2 spaces as it is configured project-wide.

[dsmiley]

Can we publish a normal durable released version and reference that? The nighties hash references are going to need to be updated often maybe and will probably get purged.