GH-35460: Introduce simdjson and migrate ObjectParser

[Reranko05]

Rationale for this change

This PR is the first in a series of PRs for GH-35460 to migrate Arrow's JSON implementation from RapidJSON to simdjson.

As the foundation for the migration, this PR introduces simdjson into Arrow's build system, updates the bundled simdjson version to v4.6.4, and migrates the self-contained arrow::json::internal::ObjectParser.

What changes are included in this PR?

This PR:

• Introduces simdjson as a third-party dependency.
• Adds CMake support for discovering and building simdjson.
• Updates the bundled simdjson dependency to v4.6.4.
• Adds system package and vcpkg integration required for simdjson across supported build environments.
• Links simdjson into the Arrow JSON library.
• Migrates arrow::json::internal::ObjectParser from RapidJSON to the simdjson DOM API.
• Adds dedicated unit tests covering:
  • String retrieval
  • Boolean retrieval
  • String map extraction
  • Invalid JSON
  • Missing keys
  • Incorrect types
  • Non-object root documents
  • Empty objects

This establishes the build infrastructure needed for the remaining JSON migration, which will be completed incrementally in follow-up PRs.

Are these changes tested?

Yes.

• Added dedicated unit tests for ObjectParser.
• Verified the ObjectParser JSON test suite locally.
• Updated the build system and validated the dependency integration across supported CI configurations (currently 41 CI checks passing; remaining failures are limited to two Windows-specific workflows under investigation).

Related #35460

• GitHub Issue: [C++] can we use simdjson to replace rapidjson #35460

[github-actions]

⚠️ GitHub issue #35460 has been automatically assigned in GitHub to PR creator.

[Reranko05]

@kou As we discussed, I've started working on the RapidJSON → simdjson migration and will try to split it into small, reviewable PRs. This PR is the first step, introducing the dependency and migrating ObjectParser.

I'm currently investigating the remaining CI failures. The implementation builds locally, the JSON test suite passes, and several CI workflows also pass. The remaining failures all seem to fail during CMake configure because simdjson::simdjson isn't available in some configurations, while other workflows successfully fall back to building simdjson from source.

I've narrowed it down to the dependency integration (FindsimdjsonAlt.cmake / CMake target creation). If there's an existing Arrow pattern you'd recommend for handling this kind of third-party dependency integration, I'd really appreciate your guidance.

[Reranko05]

Hi @kou, I've updated the bundled simdjson to v4.6.4, added the required system packages for the SYSTEM dependency path, and added simdjson to the vcpkg manifest. At this point, 41 CI checks are passing and only two Windows-specific workflows are still failing:

• AMD64 Windows MSVC GLib: link failure with
  unresolved external symbol simdjson::internal::error_codes
• AMD64 Windows C++ RTools 40 ucrt64: failure while building the bundled simdjson dependency with
  fatal error: had to relocate PCH

I investigated the MSVC failure by looking into the vcpkg integration (cpp/vcpkg.json, FindsimdjsonAlt.cmake, ThirdpartyToolchain.cmake) and also tried a couple of Windows-specific compile definition experiments, but I haven't been able to resolve it yet.

Do you have any suggestions on where I should look next for these remaining Windows-specific issues?

[github-actions]

⚠️ GitHub issue #35460 has been automatically assigned in GitHub to PR creator.

[kou]

• AMD64 Windows MSVC GLib: link failure with
  unresolved external symbol simdjson::internal::error_codes

FetchContent will fix it.

• AMD64 Windows C++ RTools 40 ucrt64: failure while building the bundled simdjson dependency with
  fatal error: had to relocate PCH

We may need to disable PCH (pre-compiled headers) with old MinGW.

Could you try this?

diff --git a/ci/scripts/PKGBUILD b/ci/scripts/PKGBUILD
index 6a66d86867..4737a7b1d8 100644
--- a/ci/scripts/PKGBUILD
+++ b/ci/scripts/PKGBUILD
@@ -137,6 +137,7 @@ build() {
     -DARROW_CXXFLAGS="${CPPFLAGS}" \
     -DAWSSDK_SOURCE=BUNDLED \
     -DCMAKE_BUILD_TYPE="release" \
+    -DCMAKE_DISABLE_PRECOMPILE_HEADERS=ON \
     -DCMAKE_INSTALL_PREFIX=${MINGW_PREFIX} \
     -DCMAKE_UNITY_BUILD=OFF \
     -DCMAKE_VERBOSE_MAKEFILE=ON \

See also: https://cmake.org/cmake/help/latest/variable/CMAKE_DISABLE_PRECOMPILE_HEADERS.html

[Reranko05]

Hi @kou, I ran a quick microbenchmark comparing the current RapidJSON implementation and the simdjson On-Demand implementation using the same ObjectParser API (Parse, GetString, GetBool) over 10 million iterations in a Release build.

RapidJSON: 18.33 s average (1.83 µs/iteration)
simdjson On-Demand: 2.91 s average (0.29 µs/iteration)

That is approximately a 6.3× speedup (about 84% lower execution time) for this workload.

[Reranko05]

I updated the simdjson integration to use the FetchContent target as well, but the AMD64 Windows MSVC GLib job still fails with the same unresolved external symbol simdjson::internal::error_codes during linking.

I'm not yet sure why the MSVC configuration is the only one exhibiting this linker issue. Do you have any suggestions?

[Reranko05]

I investigated the MSVC failure further.

The failing job is using the vcpkg-provided simdjson (4.6.4), and simdjson::simdjson is resolved as an imported static library:

TYPE = STATIC_LIBRARY
IMPORTED = TRUE
include path = .../vcpkg_installed/x64-windows/include
library = .../vcpkg_installed/x64-windows/debug/lib/simdjson.lib

However, the build still fails with unresolved references to:

simdjson::internal::error_codes
simdjson::internal::structural_or_whitespace_negated

I tried investigating whether this is related to how the vcpkg package is built or linked, but I haven't been able to isolate the cause.

Do you have any suggestions on where I should look next, or is there something specific about the MSVC/vcpkg configuration that I'm missing?