[ENHANCEMENT] RobotCode Language Server Performance Improvements

[mikeleppanen]

Problem Statement

For large Robot Framework projects, the Language Server experiences noticeable delays that impact developer productivity:

Startup Performance

Library Loading: Each library import spawns a separate Python subprocess for introspection. This process creation overhead compounds quickly when a workspace uses many libraries.

Namespace Initialization: While libraries and resources are cached to disk, namespaces (which contain resolved imports, keyword references, and variable definitions) must be rebuilt from scratch every time VS Code restarts. This is the primary bottleneck for large workspaces - users must wait for the entire workspace to be re-analyzed on every session start.

Incremental Edit Performance

Linear Lookups: When a library or resource file changes, the language server iterates through ALL open documents to find which ones are affected. In large workspaces, this O(n) behavior creates noticeable delays during editing.

Over-Invalidation: Any import-related change currently invalidates ALL namespaces, even those that don't depend on the changed file. This causes unnecessary re-analysis work.

The goal is to make the Language Server feel instant for common operations, regardless of workspace size.

Proposed Optimizations

flowchart TB
    subgraph "Current Flow"
        A[VS Code Starts] --> B[Load Libraries]
        B -->|Per-lib process spawn| C[Init Namespaces]
        C -->|Rebuilt every restart| D[Ready]
    end

    subgraph "Optimized Flow"
        E[VS Code Starts] --> F[Load Libraries]
        F -->|Shared executor + parallel| G[Load Cached Namespaces]
        G -->|Disk cache hit| H[Ready]
        G -->|Cache miss| I[Init & Cache]
        I --> H
    end

    style C fill:#f99
    style G fill:#9f9

Loading

Library Loading Optimization

• Reuse ProcessPoolExecutor instead of spawning new process per library
• Parallel loading with configurable workers (default: 4)
• Resource disk cache extending existing library cache

O(1) Dependency Lookups

Replace O(n) document iterations with reverse dependency graphs:

flowchart LR
    subgraph "Before: O(n)"
        A1[File Changed] --> B1[Iterate ALL docs]
        B1 --> C1[Check each import]
        C1 --> D1[Refresh affected]
    end

    subgraph "After: O(1)"
        A2[File Changed] --> B2[Lookup in graph]
        B2 --> C2[Refresh affected]
    end

Loading

• Track: importers, library users, variables users
• Instant lookup of affected documents when dependencies change

Namespace Disk Caching (Major Impact)

The biggest improvement - cache resolved namespace state to disk:

flowchart TB
    subgraph "Cache Structure"
        direction TB
        ROOT[.robotcode_cache/] --> PY[Python version/]
        PY --> RF[Robot Framework version/]
        RF --> LIB[libdoc/]
        RF --> RES[resource/]
        RF --> NS[namespace/ NEW]
    end

    subgraph "Cache Invalidation"
        direction TB
        CHECK{Valid?} -->|mtime match| HIT[Load from cache]
        CHECK -->|mtime changed| MISS[Rebuild & save]
        CHECK -->|dependency changed| MISS
    end

Loading

Cache invalidation triggers:

• Source file modification
• Any imported library/resource/variables file changed
• Robot Framework version change
• Python environment change

Architecture Overview

flowchart TB
    subgraph "Language Server"
        IM[ImportsManager]
        DC[DocumentsCacheHelper]
        NS[Namespace]
    end

    subgraph "Disk Cache"
        LC[(Library Cache)]
        RC[(Resource Cache)]
        NC[(Namespace Cache)]
    end

    subgraph "Dependency Tracking"
        IMP[Importers Graph]
        LIB[Library Users]
        VAR[Variables Users]
    end

    IM --> LC
    IM --> RC
    DC --> NC
    DC --> IMP
    DC --> LIB
    DC --> VAR
    NS --> IM
    NS --> DC

Loading

Bottom line:

I have implemented these changes in a local fork of RobotCode and tested against our large internal test codebase:

Scenario	Before	After
Initial analysis (cold)	45 minutes	45 minutes (no change expected)
Subsequent startup (warm cache)	45 minutes	3 minutes
Incremental edits	Noticeable delay	Almost instant

This has been thoroughly tested, and everything is functioning as expected.

[d-biehl]

first, thank you for your analysis.

Second, I need some time to go through all your comments.

can you create a PR with your changes? that would be really helpfull!

[mikeleppanen]

Yes, I can create a PR. It will take some time, but I will try to be as quick as possible.

[d-biehl]

great!!!

[mikeleppane]

@d-biehl Here is the PR: #556