Dynamic Configuration for Cutting Plane Separators via Reinforcement Learning on Incremental Graph

This is the code of Dynamic Configuration for Cutting Plane Separators via Reinforcement Learning on Incremental Graph. Mingxuan Ye, Jie Wang, Fangzhou Zhu, Zhihai Wang, Yufei Kuang, Xijun Li, Weilin Luo, Jianye Hao, Feng Wu. NeurIPS 2025.

A learning-based policy that configures which separators to activate and how many separation rounds to run during MILP solving, using incremental graph representations of the evolving instance.

---

Highlights

• Round-wise dynamic configuration
  At each separation round and node, the policy selects a subset of separators and a maximum round budget.

• Incremental graph encoding
  We encode the delta graph/state across rounds (rows/cols/cuts/separator context) to avoid repeated full-graph computation.

• Decoder-only Transformer policy
  A lightweight transformer consumes the sequence of round embeddings and outputs (i) a separator-activation mask and (ii) a categorical decision for the maximum separation round.

• Plug-and-play with SCIP
  Implemented via PySCIPOpt (custom fork) that exposes the needed hooks for per-round configuration.

---

Environment & Installation

Tested with Python 3.8, CUDA 11.7, PyTorch 1.13.

Solver dependencies

- SCIP 8.0.0
- PySCIPOpt 4.1.0 (DIY), please refer to https://gitee.com/heronland/pyscipopt-dynsep

# 1) Create & activate environment
conda create -n scip800-py38 python=3.8
conda activate scip800-py38

# 2) PyTorch (CUDA 11.7)
pip install torch==1.13.0+cu117 torchvision==0.14.0+cu117 torchaudio==0.13.0 \
  --extra-index-url https://download.pytorch.org/whl/cu117

# 3) Utilities
pip install tqdm timer gtimer sqlalchemy python-dateutil pandas tensorboard ipdb

# 4) (Linux) verbs dependency sometimes required by SCIP/PySCIPOpt builds
sudo apt-get update && sudo apt-get install -y libibverbs-dev

# 5) Graph library
pip install torch-geometric

# 6) PySCIPOpt (custom fork with DynSep hooks)
pip install git+https://gitee.com/heronland/pyscipopt-dynsep.git

---

Quick Start

Train DynSep on a SetCover benchmark with a 300-second time limit:

python parallel_reinforce_algorithm.py \
    --config_file configs/easy_setcover_ppo_config.json \
    --reward_type solving_time \
    --instance_type ${instance} \
    --train_type train \
    --max_rounds 5 \
    --max_rounds_root 5 \
    --batch_size ${batch_size} \
    --scip_seed 1 \
    --time_limit 300

The launcher performs parallel data collection and periodically evaluates the policy.

---

How It Works

• sepasel_agent.py — Separator selection agent
  Interfaces with SCIP each separation round: builds the incremental state, queries the policy, and sets separator frequencies and the maximum round budget; logs transitions for training.

• state_generator.py + data.py — Incremental state builder
  Extracts per-round tensors for rows/columns/cuts and separator context, constructs edges, and packs them into a graph-like data object.

• encoder_net.py — Feature extractor / message passing
  Encodes the current (incremental) LP/MIP state into a fixed-width embedding.

• decoder_only_transformer.py + actorcritic_net.py — Policy & value
  Consumes the sequence of round embeddings and outputs a separator mask and max-round decision, with log-prob utilities for PPO.

• algorithms.py + rollout_storage.py — PPO training loop
  Manages advantage computation, clipped objectives, value loss, optimizer steps, checkpointing, and mini-batching.

• parallel_reinforce_algorithm.py — Entry point
  Spawns parallel samplers, handles seeding, training/eval modes, and logging.

---

Repository Layout

.
├── parallel_reinforce_algorithm.py   # training/evaluation launcher
├── sepasel_agent.py                  # SCIP-facing agent for per-round config
├── state_generator.py                # builds incremental round-wise states
├── data.py                           # data container for graph-like tensors
├── encoder_net.py                    # feature encoder / message passing
├── decoder_only_transformer.py       # decoder-only transformer blocks
├── actorcritic_net.py                # policy & value heads
├── rollout_storage.py                # buffer, returns, mini-batch iterator
├── algorithms.py                     # PPO implementation
└── configs/                          # example configs (instance types, etc.)

---

Reproducibility

• Fix SCIP randomness with --scip_seed; Python/NumPy/PyTorch seeds are set in the launcher.
• Keep dataset splits/shuffles fixed; report mean (± std) over multiple seeds.
• Log config and commit hash with each run; store checkpoints and logs under unique run IDs.

---

Datasets

We use datasets from https://drive.google.com/drive/folders/1LXLZ8vq3L7v00XH-Tx3U6hiTJ79sCzxY?usp=sharing

Citation

If you find this code useful, please consider citing the following paper.

@inproceedings{
ye2025dynsep,
title={Dynamic Configuration for Cutting Plane Separators via Reinforcement Learning on Incremental Graph},
author={Mingxuan Ye and Jie Wang and Fangzhou Zhu and Zhihai Wang and Yufei Kuang and Xijun Li and Weilin Luo and Jianye Hao and Feng Wu},
booktitle={Advances in Neural Information Processing Systems 38, NeurIPS 2025},
year={2025},
}

License

This project is released under the MIT License.