feat(xdp): Enhance XDP backend with improved concurrency and performance metrics

[mvp-2003]

• Added new configuration parameters for TX and RX fill and completion ring sizes.
• Introduced enhanced concurrency options for XDP operations, including batch sizes and adaptive batching.
• Implemented a performance testing example to demonstrate the benefits of the new XDP backend.
• Created a batch processing utility for efficient transmission of packets.
• Developed functional and unit tests to validate backward compatibility and new features.
• Updated configuration parsing to support new parameters and ensure proper validation.

[anandbonde]

@mvp-2003: Currently, we don't support running CI on branches outside of our repo. And PRs cannot be pulled unless the CI passes. The only way to work around this is to create a new branch in our repo and submit a PR that way.

Also, could you please write a small reasoning behind why you think this should be present in the mainstream codebase?

[anandbonde]

Added a general comment on this PR. Please take a look.

[mvp-2003]

@anandbonde Hello sir. Just to be sure, I understood the instructions clearly, you wanted me to create a new branch on the origianal repo and raise the PR from my forked repo to the new branch of the original repo and also give a proper documentation and state the reason for coming up with this approach, right?

[anandbonde]

@anandbonde Hello sir. Just to be sure, I understood the instructions clearly, you wanted me to create a new branch on the origianal repo and raise the PR from my forked repo to the new branch of the original repo and also give a proper documentation and state the reason for coming up with this approach, right?

@mvp-2003: I've added you as a contributor to the repo, now you can create a new branch on the original repo and submit PR(s) from there. This will allow the CI actions to run on that branch — it's a mandatory check without which PRs cannot be checked in.

Re. doc: It would be good to document what drove this change... are you addressing/fixing an existing issue or are you trying to solve a "local" problem that you are facing and want to contribute those changes back to the community?

[mvp-2003]

@anandbonde Hello sir. While trying to create a new branch from the "dev" branch, I am getting this error: "Create protected branch 'xdp-performance-improvement' check failed: You're not authorized to push to this branch. Visit https://docs.github.com/repositories/configuring-branches-and-merges-in-your-repository/managing-protected-branches/about-protected-branches for more information."
Could you please recheck if I was provided with the necessary permissions? Thank you.

[mvp-2003]

Hello @anandbonde
Did you get a chance to look into this matter, sir?

[anandbonde]

@anandbonde Hello sir. While trying to create a new branch from the "dev" branch, I am getting this error: "Create protected branch 'xdp-performance-improvement' check failed: You're not authorized to push to this branch. Visit https://docs.github.com/repositories/configuring-branches-and-merges-in-your-repository/managing-protected-branches/about-protected-branches for more information." Could you please recheck if I was provided with the necessary permissions? Thank you.

I’m seeing that you already have permissions. Can you try ssh auth with the GitHub repo if https isn’t working. Let me know if it doesn’t work for you.

[mvp-2003]

Hello @anandbonde
I tried as you asked. I cloned the repo through SSH and created a new branch and as clicked on push to publish the branch, i get this error:
Total 0 (delta 0), reused 0 (delta 0), pack-reused 0 (from 0)
remote: error: GH006: Protected branch update failed for refs/heads/con-xdp-1437.
remote:
remote: - You're not authorized to push to this branch. Visit https://docs.github.com/repositories/configuring-branches-and-merges-in-your-repository/managing-protected-branches/about-protected-branches for more information.
To github.com:microsoft/demikernel.git
! [remote rejected] con-xdp-1437 -> con-xdp-1437 (protected branch hook declined)
error: failed to push some refs to 'github.com:microsoft/demikernel.git'

I don't think I am allowed to create branches. It may also be possible that there are some rulesets configures depending on roles. Can you check once again. Thank you.

[anandbonde]

Hello @anandbonde I tried as you asked. I cloned the repo through SSH and created a new branch and as clicked on push to publish the branch, i get this error: Total 0 (delta 0), reused 0 (delta 0), pack-reused 0 (from 0) remote: error: GH006: Protected branch update failed for refs/heads/con-xdp-1437. remote: remote: - You're not authorized to push to this branch. Visit https://docs.github.com/repositories/configuring-branches-and-merges-in-your-repository/managing-protected-branches/about-protected-branches for more information. To github.com:microsoft/demikernel.git ! [remote rejected] con-xdp-1437 -> con-xdp-1437 (protected branch hook declined) error: failed to push some refs to 'github.com:microsoft/demikernel.git'

I don't think I am allowed to create branches. It may also be possible that there are some rulesets configures depending on roles. Can you check once again. Thank you.

@mvp-2003, sorry for the delay, can you try now? I've explicitly added you in the list of contributors as well now and that should allow creating for new branches.

[mvp-2003]

I'm sorry @anandbonde it looks like I am still blocked. How possible would it be for you to create a branch for me and just tell the branch name here so that I can raise the PR. I think that would be a simpler approach with a guaranteed success.

[mvp-2003]

@anandbonde sir? Any updates?

[anandbonde]

I'm sorry @anandbonde it looks like I am still blocked. How possible would it be for you to create a branch for me and just tell the branch name here so that I can raise the PR. I think that would be a simpler approach with a guaranteed success.

@mvp-2003 That will not work because you are not authorized to "push". I'll take a look.

[update]: I've now re-enabled the permissions for your account. I'm confident that this should work. Please retry and let me know if it doesn't work.