tools: import a conversion tool to and from VuXML #237

khorben · 2024-04-23T15:46:46Z

This is relevant to FreeBSD's ports, and possibly to any other project using the VuXML format in order to track vulnerabilities. (http://vuxml.freebsd.org)

The objective is to help FreeBSD offer security advisories in the OSV format, for ports first but possibly also for base components in the future. The corresponding ecosystem string (most likely FreeBSD) will be requested in a dedicated pull request.

Sponsored by: The FreeBSD Foundation

andrewpollock · 2024-10-16T06:15:37Z

Hi @khorben, this is exciting progress. I can review this from the OSV side of things, is there a VuXML SME who can review that side of things?

Also, if you look at tools/redhat for inspiration, having some test data to validate behaviour is also helpful.

khorben · 2025-05-02T23:59:39Z

Hi Andrew, sorry for the late answer. I suppose by SME you meant "Subject Matter Expert", in which case I think FreeBSD's ports-secteam@ (https://wiki.freebsd.org/PortsSecteam) might be the best point of contact.
I am looking into importing some test data for validation.

andrewpollock · 2025-05-12T06:52:11Z

Would you mind addressing the DCO check failure?

andrewpollock

I'm confused by the UBUNTU-CVE-2025-3454 test files?

I think it'll be a no-op change, but it'd be good to update the schema version in the test OSV records to the current version.

tools/vuxml/README.md

khorben · 2025-05-12T17:51:34Z

Would you mind addressing the DCO check failure?

Thanks for the heads up, I think this should be addressed now.

khorben · 2025-05-12T17:56:04Z

I'm confused by the UBUNTU-CVE-2025-3454 test files?

I have organised the test data as follows:

testdata/VUXML contains the original VUXML files, and their converted output into OSV's JSON
testdata/OSV is the other way around

Should I change or clarify anything in the commits?

khorben · 2025-05-12T21:50:08Z

I'm confused by the UBUNTU-CVE-2025-3454 test files?

Oh do you mean that you expected test files about FreeBSD?

First, IIRC VuXML is distribution-agnostic, and not tied to a specific ecosystem.
The FreeBSD project does not provide OSV files (yet); the objective is to allow the project to do so in the future; this is currently being investigated. OTOH, it can already be useful to import OSV data from other projects, and to publish it as VuXML fort the FreeBSD ports after review.

Likewise, it can be useful to convert VuXML data into OSV for the preparation work mentioned above. However, this still requires the corresponding ecosystem for FreeBSD, and choosing its exact name(s) might still be tricky: there is the base system, to distinguish from the third-party ports, and soon also from the packages for the base system.

I think it'll be a no-op change, but it'd be good to update the schema version in the test OSV records to the current version.

I have now updated the schema version in the output to OSV as 1.7.0.
I believe this is now as expected; let me know otherwise.

illuusio · 2025-05-14T07:14:25Z

When turning off DTD validation then it works (dtd_validation=False) with: https://vuxml.freebsd.org/freebsd/vuln.xml

tools/vuxml/convert_vuxml.py

illuusio · 2025-05-14T07:55:55Z

I would also add purl for packages when ecosystem have package manager it's nice addition to information

illuusio · 2025-05-14T08:01:51Z

When Output directory is issued then (if I understand correctly) naming should something like: FBSD-2025-05-14.json or FBSD-VUID.json. My humble opinion is that they should be arranged by package name or yearly so there should not be so much in one dir.

illuusio · 2025-05-14T08:27:00Z

It seems that <affects> is right next to <topic> in FreeBSD VuXML-file. It shouldn't affect anything but if parser is very bad then it does not work.

illuusio · 2025-05-14T08:37:43Z

tools/vuxml/convert_osv.py

@@ -0,0 +1,271 @@
+#!/usr/bin/env python
+#


Just suggesting to have SPDX-header for license also

Thank you for the suggestion, I have now added the SPDX headers.

SPDX header seems to be correct

oliverchang

thank you very much for working on this, and for adding the tests! (and apologies for the delay in reviewing)

oliverchang · 2025-06-03T09:14:36Z

tools/vuxml/testdata/VUXML/CVE-2025-29087.json

@@ -0,0 +1,42 @@
+{
+    "schema_version": "1.7.0",
+    "id": "409206f6-25e6-11f0-9360-b42e991fc52e",


this needs to have a FREEBSD- or some other prefix so it's clear it comes from the FreeBSD database.

Please also define the chosen prefix in https://ossf.github.io/osv-schema/#id-modified-fields

oliverchang · 2025-06-03T09:15:10Z

tools/vuxml/testdata/VUXML/CVE-2025-29087.json

+    "affected": [
+        {
+            "package": {
+                "ecosystem": "FreeBSD:ports",


In a separate PR, could you please add a ecosystem definition to https://ossf.github.io/osv-schema/#affectedpackage-field ?

Sorry about the late answer from my side as well. The choice of ecosystem definition is a bit complicated on FreeBSD, notably because of the distinction between the base system and the software packages. No decision has been made yet AFAIK, but I will bring it up again internally.

tools/vuxml/testdata/VUXML/CVE-2025-29087.json

andrewpollock · 2026-01-27T05:15:43Z

@khorben I see some exciting progress with the latest release of the OSV Schema, and thought I'd check in on progress on this PR?

This is relevant to FreeBSD's ports. Note that: * There is no official ecosystem for FreeBSD ports yet * A few elements of the VuXML DTD are not covered (e.g., \<uscertsa>, \<uscertta>) * Cancelled VuXML events are not represented, for lack of information (withdrawal date...) * Some versions do not match reality and need a correct increment (\<gt>) Sponsored by: The FreeBSD Foundation Signed-off-by: Pierre Pronchery <[email protected]>

This is relevant to FreeBSD's ports. Sponsored by: The FreeBSD Foundation Signed-off-by: Pierre Pronchery <[email protected]>

XXX this only happens when outputting to the standard output. This is relevant to FreeBSD's ports. Sponsored by: The FreeBSD Foundation Signed-off-by: Pierre Pronchery <[email protected]>

This is relevant to FreeBSD's ports. Sponsored by: The FreeBSD Foundation Signed-off-by: Pierre Pronchery <[email protected]>

Sponsored by: The FreeBSD Foundation Signed-off-by: Pierre Pronchery <[email protected]>

khorben · 2026-01-28T11:14:51Z

I just rebased the changes on top of the latest main branch, and imported further changes from @illuusio (from https://github.com/illuusio/freebsd-osv/blob/main/bin/convert_vuxml.py).
This should address some of the remaining issues.
Let us know if we need to make further improvements!

Sponsored by: The FreeBSD Foundation Signed-off-by: Pierre Pronchery <[email protected]>

khorben force-pushed the khorben/vuxml branch from 60797b2 to b7ea020 Compare October 15, 2024 15:05

khorben marked this pull request as ready for review October 15, 2024 15:06

khorben force-pushed the khorben/vuxml branch 3 times, most recently from 4bd95a4 to 8614a9b Compare May 3, 2025 01:01

andrewpollock reviewed May 12, 2025

View reviewed changes

tools/vuxml/README.md Show resolved Hide resolved

khorben force-pushed the khorben/vuxml branch 2 times, most recently from 3e0b468 to d3c580a Compare May 12, 2025 17:50

khorben force-pushed the khorben/vuxml branch 2 times, most recently from 18503b2 to 44d9763 Compare May 12, 2025 21:39

illuusio reviewed May 14, 2025

View reviewed changes

tools/vuxml/convert_vuxml.py Outdated Show resolved Hide resolved

illuusio reviewed May 14, 2025

View reviewed changes

oliverchang reviewed Jun 3, 2025

View reviewed changes

khorben added 5 commits January 28, 2026 10:52

tools: import a conversion tool to vuxml

27caec0

This is relevant to FreeBSD's ports. Sponsored by: The FreeBSD Foundation Signed-off-by: Pierre Pronchery <[email protected]>

tools: output a JSON object for single entries

10dcb14

XXX this only happens when outputting to the standard output. This is relevant to FreeBSD's ports. Sponsored by: The FreeBSD Foundation Signed-off-by: Pierre Pronchery <[email protected]>

tools: import test data for the vuxml conversion tool

9aea535

This is relevant to FreeBSD's ports. Sponsored by: The FreeBSD Foundation Signed-off-by: Pierre Pronchery <[email protected]>

tools: add SPDX headers to the vuxml conversion tools

8d9615b

Sponsored by: The FreeBSD Foundation Signed-off-by: Pierre Pronchery <[email protected]>

khorben force-pushed the khorben/vuxml branch from 62ccc24 to 8d9615b Compare January 28, 2026 09:53

khorben force-pushed the khorben/vuxml branch 3 times, most recently from 7986c4b to de0e650 Compare January 28, 2026 11:12

tools: import changes from illuusio for convert_vuxml.py

9f4a519

Sponsored by: The FreeBSD Foundation Signed-off-by: Pierre Pronchery <[email protected]>

khorben force-pushed the khorben/vuxml branch from de0e650 to 9f4a519 Compare January 28, 2026 11:20

tools: import a conversion tool to and from VuXML #237

Are you sure you want to change the base?

tools: import a conversion tool to and from VuXML #237

Conversation

khorben commented Apr 23, 2024

Uh oh!

andrewpollock commented Oct 16, 2024

Uh oh!

khorben commented May 2, 2025

Uh oh!

andrewpollock commented May 12, 2025

Uh oh!

andrewpollock left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

khorben commented May 12, 2025

Uh oh!

khorben commented May 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

khorben commented May 12, 2025

Uh oh!

illuusio commented May 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

illuusio commented May 14, 2025

Uh oh!

illuusio commented May 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

illuusio commented May 14, 2025

Uh oh!

illuusio May 14, 2025

Choose a reason for hiding this comment

Uh oh!

khorben May 29, 2025

Choose a reason for hiding this comment

Uh oh!

illuusio Jun 3, 2025

Choose a reason for hiding this comment

Uh oh!

oliverchang left a comment

Choose a reason for hiding this comment

Uh oh!

oliverchang Jun 3, 2025

Choose a reason for hiding this comment

Uh oh!

oliverchang Jun 3, 2025

Choose a reason for hiding this comment

Uh oh!

khorben Jan 28, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

andrewpollock commented Jan 27, 2026

Uh oh!

khorben commented Jan 28, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

khorben commented May 12, 2025 •

edited

Loading

illuusio commented May 14, 2025 •

edited

Loading

illuusio commented May 14, 2025 •

edited

Loading