Skip to content

[doc] Custom maps design#4882

Open
saxena-anurag wants to merge 20 commits intomicrosoft:mainfrom
saxena-anurag:user/anusa/custom_maps_proposal_2
Open

[doc] Custom maps design#4882
saxena-anurag wants to merge 20 commits intomicrosoft:mainfrom
saxena-anurag:user/anusa/custom_maps_proposal_2

Conversation

@saxena-anurag
Copy link
Copy Markdown
Contributor

@saxena-anurag saxena-anurag commented Dec 12, 2025

Fixes #4375

Description

This PR adds proposal for implementing custom maps (#4375).

Testing

NA

Documentation

This PR adds doc.

Installation

No

Comment thread docs/ExtensibleMaps.md Outdated
Comment thread docs/ExtensibleMaps.md Outdated
Comment thread docs/ExtensibleMaps.md Outdated
Comment thread docs/ExtensibleMaps.md Outdated
Comment thread docs/ExtensibleMaps.md Outdated
Comment thread docs/ExtensibleMaps.md Outdated
Comment thread docs/ExtensibleMaps.md Outdated
Comment thread docs/ExtensibleMaps.md Outdated
Comment thread docs/ExtensibleMaps.md Outdated
Comment thread docs/ExtensibleMaps.md Outdated
Comment thread docs/ExtensibleMaps.md Outdated
Comment thread docs/ExtensibleMaps.md Outdated
mtfriesen
mtfriesen previously approved these changes Dec 12, 2025
@saxena-anurag saxena-anurag changed the title custom maps proposal [doc] Custom maps proposal Dec 12, 2025
@saxena-anurag saxena-anurag marked this pull request as ready for review December 12, 2025 20:50
mtfriesen
mtfriesen previously approved these changes Dec 12, 2025
Comment thread docs/ExtensibleMaps.md Outdated
Comment thread docs/ExtensibleMaps.md Outdated
Global maps will use IDs from 1 to 4095. Extensible maps will use IDs 4096 onwards.

- **Global Maps (1-4095)**: Reserved for core eBPF runtime map types (hash, array, etc.)
- **Extensible Maps (4096+)**: Available for extension-implemented custom map types

This comment was marked as resolved.

Comment thread docs/CustomMaps.md Outdated
Comment thread docs/CustomMaps.md Outdated
Comment thread docs/CustomMaps.md Outdated
Comment thread docs/CustomMaps.md Outdated
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR proposes a design for implementing custom maps in eBPF-for-Windows, enabling extensions to register and manage their own map types (e.g., BPF_MAP_TYPE_XSKMAP) through a new NMR provider interface while leveraging the core runtime's existing map implementations.

Key changes:

  • Introduces a new NMR provider interface for custom maps with provider registration data and dispatch tables
  • Partitions the map type ID space: IDs 1-4095 for core maps, 4096+ for custom maps
  • Documents the map creation flow, lifecycle management, and how existing APIs remain unchanged

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread docs/CustomMaps.md Outdated
Comment thread docs/CustomMaps.md Outdated
Comment thread docs/CustomMaps.md Outdated
Comment thread docs/CustomMaps.md Outdated
@saxena-anurag
Copy link
Copy Markdown
Contributor Author

saxena-anurag commented Jan 8, 2026

Once custom map types are registered, why have separate ranges in the first place?

@dthaler -- Even though it is recommended, extensions can choose to not register the new map type. Keeping a different range helps ebpfcore know if this is a native map type, or it needs to attach to an NMR provider for this map type.

@dthaler

This comment was marked as resolved.

@saxena-anurag
Copy link
Copy Markdown
Contributor Author

Once custom map types are registered, why have separate ranges in the first place?

@dthaler -- Even though it is recommended, extensions can choose to not register the new map type. Keeping a different range helps ebpfcore know if this is a native map type, or it needs to attach to an NMR provider for this map type.

How is that different from helper function IDs?

It is not. Helper function IDs are also partitioned today. Program type specific helper functions need to start from 0xFFFF

@dthaler

This comment was marked as resolved.

@dthaler

This comment was marked as resolved.

@saxena-anurag
Copy link
Copy Markdown
Contributor Author

Once custom map types are registered, why have separate ranges in the first place?

@dthaler -- Even though it is recommended, extensions can choose to not register the new map type. Keeping a different range helps ebpfcore know if this is a native map type, or it needs to attach to an NMR provider for this map type.

How is that different from helper function IDs?

It is not. Helper function IDs are also partitioned today. Program type specific helper functions need to start from 0xFFFF

Just to drill into the last part... two program type specific helper functions can have the same id because (I think) helper functions are always indexed by (program type, function id) not just by (function id), correct? Is the proposal that the separate range of map type ids must be specific to a single program type, and always indexed by (program type, map type), never just (map type)? Would such map types ever be accessible from user-mode? If so, using what API(s)?

No the map type ID would be globally unique across program types, hence the recommendation to register it in eBPF repo, to avoid a conflict.

@dthaler

This comment was marked as resolved.

@saxena-anurag
Copy link
Copy Markdown
Contributor Author

Clarification needed on type ID collision behavior

The doc states that extensions "must" register map types via PR to avoid collisions, but doesn't explain what happens at runtime if:

 - A custom map type is used with no provider loaded
 - Two extensions register for the same type ID

Suggested clarification - Consider adding a note in the "Map Type Enum Partitioning" section (around line 135):

 **Runtime Behavior:**

 - If no provider is registered for a custom map type, NMR client registration will fail to find a
   provider and map creation will return an error.

 - If multiple extensions register as providers for the same custom map type ID, the behavior is
   undefined - NMR will offer all matching provider interfaces and eBPFCore will attach to the
   first one. This is why registering map types via PR is mandatory: it prevents collisions through
   code review rather than runtime enforcement.

This makes it clear that the PR-based registration process is the collision prevention mechanism, not runtime validation.

Updated text.

@github-actions
Copy link
Copy Markdown
Contributor

This pull request isn't linked to any GitHub issue. Please reference an issue with a keyword such as Fixes #123, Closes #456, etc., so the work can be tracked.

@github-actions
Copy link
Copy Markdown
Contributor

This pull request isn't linked to any GitHub issue. Please reference an issue with a keyword such as Fixes #123, Closes #456, etc., so the work can be tracked.

Comment thread docs/CustomMaps.md Outdated
Comment thread docs/CustomMaps.md
Comment thread docs/CustomMaps.md Outdated
Comment thread docs/CustomMaps.md
Co-authored-by: Dave Thaler <dthaler1968@gmail.com>
@github-actions
Copy link
Copy Markdown
Contributor

This pull request isn't linked to any GitHub issue. Please reference an issue with a keyword such as Fixes #123, Closes #456, etc., so the work can be tracked.

@github-actions
Copy link
Copy Markdown
Contributor

This pull request isn't linked to any GitHub issue. Please reference an issue with a keyword such as Fixes #123, Closes #456, etc., so the work can be tracked.

Comment thread docs/CustomMaps.md
Comment thread docs/CustomMaps.md Outdated
Comment thread docs/CustomMaps.md Outdated
Comment thread docs/CustomMaps.md Outdated
Comment thread docs/CustomMaps.md Outdated
@saxena-anurag saxena-anurag requested a review from dthaler April 6, 2026 20:02
@saxena-anurag saxena-anurag modified the milestones: 2602, 2604 Apr 7, 2026
Comment thread docs/CustomMaps.md
Comment thread docs/CustomMaps.md Outdated
Comment thread docs/CustomMaps.md
Comment thread docs/CustomMaps.md
Comment thread docs/CustomMaps.md
@saxena-anurag
Copy link
Copy Markdown
Contributor Author

@dthaler I have updated this PR, and also created PR #5173 for the corresponding code changes.

Comment thread docs/CustomMaps.md Outdated
dthaler
dthaler previously approved these changes Apr 14, 2026
Copy link
Copy Markdown
Collaborator

@dthaler dthaler left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks ok modulo one wording nit.

Co-authored-by: Dave Thaler <dthaler1968@gmail.com>
@saxena-anurag
Copy link
Copy Markdown
Contributor Author

@dthaler can you re-approve the PR. I had pushed another commit based on the review comments.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Development

Successfully merging this pull request may close these issues.

Add extensible map types

7 participants