Add $(python,...) built-in

This introduces a portable $(python,...) preprocessor function that
evaluates a Python code string in-process via exec(), returning "y" on
success and "n" on exception. 'exec' namespace exposes os, sys, shutil,
platform, and a shell-free run(*argv) helper for subprocess checks. Each
call receives a fresh globals copy for namespace isolation.

Rewrite all six toolchain functions (cc-option, ld-option, as-instr,
as-option, cc-option-bit, rustc-option) to use subprocess.Popen with
argument lists (shell=False) and os.devnull, eliminating /dev/null,
2>/dev/null, and printf-pipe constructs.  This closes the shell
injection vector from environment variables and makes the functions
work on Windows.

Add _run_argv() helper with timeout support and proper process cleanup.
Remove dead _run_cmd_in_tmpdir(); replaced by TemporaryDirectory
context managers in individual functions.

Add quote tracking to the macro expander so that commas and parentheses
inside single, double, and triple-quoted strings are not treated as
argument separators (required for $(python,assert "a,b" == "a,b")).

Close #41

Author: jserv

.github/workflows/test.yml

@@ -88,12 +88,17 @@ jobs:
         path: ${{ matrix.target.headless-only && '.' || 'Kconfiglib' }}
 
     - name: Run pytest selftests
-      # Skip on Windows: several tests depend on Unix shell ($(shell,...)),
-      # gcc, and forward-slash paths that are unavailable on Windows CI.
-      if: ${{ matrix.target.headless-only != true }}
       working-directory: ${{ matrix.target.headless-only && '.' || 'Kconfiglib' }}
       run: |
-        python -m pytest tests/ -v --tb=short --ignore=tests/test_conformance.py
+        if [ "${{ matrix.target.headless-only }}" == "true" ]; then
+          # Windows: portable subset -- excludes tests that need gcc or
+          # Unix shell commands (Kpreprocess uses $(shell,...) with
+          # bash-specific syntax).
+          python -m pytest tests/test_preprocess.py -v --tb=short \
+            -k "test_user_defined or test_success_failure or test_python_fn or test_kconfig_warn"
+        else
+          python -m pytest tests/ -v --tb=short --ignore=tests/test_conformance.py
+        fi
 
     - name: Apply Linux Kconfig Makefile patch
       # Skip for Windows (headless-only mode)

KBUILD.md

@@ -1,6 +1,7 @@
 # Kbuild Toolchain Functions
 
-Kconfiglib implements Kbuild toolchain detection functions used by the Linux kernel since version 4.18.
+Kconfiglib implements Kbuild toolchain detection functions used by the Linux kernel since version 4.18,
+plus the portable `$(python,...)` built-in for cross-platform boolean checks.
 These preprocessor functions enable runtime detection of compiler, assembler, and linker capabilities,
 allowing kernel configurations to adapt to different toolchain versions.
 
@@ -26,6 +27,39 @@ For comprehensive Kconfig syntax documentation, see the
 `$(failure,command)`
 : Returns `n` if command succeeds, `y` otherwise. Inverse of `success`.
 
+### Portable In-Process Checks
+
+`$(python,code)`
+: Evaluates a Python code string in-process via `exec()`. Returns `y` if
+execution succeeds without exception, `n` otherwise. No subprocess, no shell,
+no PATH dependency. Use `assert` for boolean checks.
+
+The exec namespace provides pre-imported modules and a shell-free subprocess helper:
+
+| Name       | Type     | Purpose                                    |
+|------------|----------|--------------------------------------------|
+| `os`       | module   | env vars, paths, file ops                  |
+| `sys`      | module   | platform, version, `sys.executable` path   |
+| `shutil`   | module   | `which()` for tool detection               |
+| `platform` | module   | machine/system/architecture                |
+| `run`      | function | shell-free subprocess, returns bool        |
+
+`run(*argv)` executes a command as an argument list (`shell=False`) and returns
+`True` if it exits with code 0, `False` otherwise.
+
+Each `exec()` call receives a fresh copy of the namespace so assignments in one
+`$(python,...)` invocation do not leak into subsequent calls.
+
+`SystemExit` is handled specially: `SystemExit(0)` and `SystemExit(None)` map
+to `y`; non-zero/non-empty codes map to `n`. `AssertionError` maps to `n`
+silently (expected for boolean checks via `assert`). All other exceptions
+(`NameError`, `SyntaxError`, etc.) map to `n` and emit a Kconfig warning
+with the exception type and message, aiding diagnosis of typos in code strings.
+
+Trust model: `$(python,...)` has the same trust level as `$(shell,...)`.
+Kconfig files are trusted code (like Makefiles). The restricted globals provide
+scope isolation (no parser internals visible), not security sandboxing.
+
 ### Compiler Detection
 
 `$(cc-option,flag[,fallback])`
@@ -99,6 +133,48 @@ config HAS_32BIT
     def_bool "$(m32-flag)" != ""
 ```
 
+### Portable Checks with $(python,...)
+
+```
+# Boolean checks (in-process, no subprocess)
+config PYTHON_AVAILABLE
+    def_bool $(python,)
+
+config HAS_CC
+    def_bool $(python,assert os.environ.get('CC'))
+
+config IS_LINUX
+    def_bool $(python,assert sys.platform == 'linux')
+
+config IS_X86_64
+    def_bool $(python,assert platform.machine() == 'x86_64')
+
+config HAS_GCC
+    def_bool $(python,assert shutil.which('gcc'))
+
+# Shell-free subprocess checks
+config CC_IS_CLANG
+    def_bool $(python,assert run(sys.executable, 'scripts/detect-compiler.py', '--is', 'Clang'))
+
+config HAVE_SDL2
+    def_bool $(python,assert run('pkg-config', '--exists', 'sdl2'))
+```
+
+Commas inside `run(...)` are safe: the Kconfig preprocessor tracks parenthesis
+depth and only splits on commas at the top level of the function call.
+
+Quoted strings are also safe: the preprocessor tracks single (`'`), double (`"`),
+triple-single (`'''`), and triple-double (`"""`) quoted regions. Commas and
+parentheses inside quotes are treated as literal characters, not argument
+separators or nesting markers. Backslash escapes (`\"`, `\'`) inside quoted
+regions are handled correctly.
+
+Use semicolons instead of commas for multi-statement code:
+`$(python,import os; assert os.path.isfile('Makefile'))`.
+
+For string-valued results (e.g., getting the compiler type name), `$(shell,...)`
+remains the right tool. `$(python,...)` only returns `y` or `n`.
+
 ## Implementation
 
 ### Design
@@ -110,6 +186,28 @@ Functions are implemented in `kconfiglib.py` following these principles:
 - Python 3.6+ using standard library only
 - Graceful error handling (missing tools return `n`)
 
+### Shell-Free Toolchain Functions
+
+Toolchain functions (`cc-option`, `ld-option`, `as-instr`, `as-option`,
+`cc-option-bit`, `rustc-option`) use `subprocess.Popen` with argument lists
+(`shell=False`) and `os.devnull` instead of Unix shell syntax. This
+eliminates shell injection from environment variables and Kconfig-supplied
+options, and makes the functions portable to Windows.
+
+Internal helpers:
+
+`_run_argv(argv, stdin_data=None)`
+: Runs a command as an argument list. Returns `True` if exit code is 0.
+Used by all toolchain functions.
+
+`_run_cmd(command)`
+: Runs a command via shell (`shell=True`). Used by `success`, `failure`,
+and `if-success`, which accept user-supplied shell commands by design.
+
+`_run_helper(*argv)`
+: Shell-free subprocess for `$(python,...)` code strings. Exposed as `run()`
+in the exec namespace.
+
 ### Environment Variables
 
 Functions respect standard build variables:
@@ -119,33 +217,34 @@ Functions respect standard build variables:
 
 ### Performance
 
-Functions execute shell commands during Kconfig parsing, which can be slow.
-For applications that parse configurations repeatedly, consider implementing
-caching or using `allow_empty_macros=True` to skip toolchain detection.
+Toolchain functions spawn subprocesses during Kconfig parsing, which can be
+slow. `$(python,...)` checks that don't call `run()` execute in-process with
+no subprocess overhead. For applications that parse configurations repeatedly,
+consider implementing caching or using `allow_empty_macros=True` to skip
+toolchain detection.
 
 ## Testing
 
-Four test suites validate the implementation:
+Tests live in `tests/test_preprocess.py` (part of the pytest suite):
 
-`test_issue111.py`
-: Validates basic toolchain function parsing.
+`test_kbuild_functions`
+: Verifies toolchain functions (`cc-option`, `as-instr`, etc.) and
+`$(python,...)` via Kconfig parsing. Exercises the full preprocessor path.
 
-`test_issue109.py`
-: Tests nested function calls and complex expressions.
+`test_success_failure_fns`
+: Tests `success`, `failure`, and `if-success` directly in Python using
+`sys.executable` as a portable true/false replacement.
 
-`test_kbuild_complete.py`
-: Comprehensive suite with 35+ test cases covering all functions, edge cases, and error conditions.
+`test_python_fn_isolation`
+: Verifies that variable assignments in one `$(python,...)` call do not
+leak into subsequent calls.
 
-`test_kernel_compat.py`
-: Real-world kernel Kconfig snippets from init/Kconfig, arch/x86/Kconfig, etc.
+`test_python_fn_system_exit`
+: Verifies `SystemExit` handling: `exit(0)` maps to `y`, non-zero to `n`.
 
-Run all tests:
+Run:
 ```bash
-python3 test_basic_parsing.py && \
-python3 test_issue111.py && \
-python3 test_issue109.py && \
-python3 test_kbuild_complete.py && \
-python3 test_kernel_compat.py
+python3 -m pytest tests/test_preprocess.py -v
 ```
 
 ## Compatibility
@@ -165,6 +264,30 @@ Tested with:
 - binutils 2.31+
 - rustc 1.60+ (optional)
 
+## Portability
+
+### Unix vs Windows
+
+| Unix shell idiom | Portable replacement |
+|---|---|
+| `$(shell,cmd 2>/dev/null && echo y \|\| echo n)` | `$(python,assert run('cmd', 'arg'))` |
+| `$(shell,test -n "$CC" && echo y \|\| echo n)` | `$(python,assert os.environ.get('CC'))` |
+| `$(shell,scripts/foo.py --flag ...)` | `$(python,assert run(sys.executable, 'scripts/foo.py', '--flag'))` |
+| `$(shell,pkg-config --exists lib && echo y \|\| echo n)` | `$(python,assert run('pkg-config', '--exists', 'lib'))` |
+| `$(success,true)` | `$(python,)` |
+| `$(failure,false)` | `$(python,assert False)` |
+
+`$(shell,...)` remains necessary for string-valued output (e.g., compiler
+type name, version strings). For boolean checks, prefer `$(python,...)`
+on cross-platform projects.
+
+### Toolchain functions
+
+`cc-option`, `ld-option`, `as-instr`, `as-option`, `cc-option-bit`, and
+`rustc-option` are portable by default. They use `subprocess.Popen` with
+argument lists internally -- no shell involvement, no `/dev/null` path
+dependency (`os.devnull` is used instead).
+
 ## Real-World Examples
 
 From `arch/x86/Kconfig.cpu`:
@@ -191,6 +314,18 @@ config SHADOW_CALL_STACK
     depends on $(cc-option,-fsanitize=shadow-call-stack -ffixed-x18)
 ```
 
+From a cross-platform project (Mado):
+```
+config CC_IS_CLANG
+    def_bool $(python,assert run(sys.executable, 'scripts/detect-compiler.py', '--is', 'Clang'))
+
+config HAVE_SDL2
+    def_bool $(python,assert run('pkg-config', '--exists', 'sdl2'))
+
+config CROSS_COMPILE_ENABLED
+    def_bool $(python,assert os.environ.get('CROSS_COMPILE'))
+```
+
 ## See Also
 
 - [Kconfig Language](https://docs.kernel.org/kbuild/kconfig-language.html) - Complete syntax specification