Add Mermaid system, components and container diagrams#2222
Add Mermaid system, components and container diagrams#2222gildub wants to merge 4 commits intoguacsec:mainfrom
Conversation
Reviewer's GuideAdds a new Mermaid-based UML classDiagram documenting the Trustify data model, including entities, relationships, and key design patterns for PURLs, SBOMs, relationships, statuses, versions, and licensing. ER diagram for core Trustify data modelerDiagram
advisory {
UUID id
UUID issuer_id
timestamp published
timestamp modified
timestamp withdrawn
String identifier
String title
String version
String document_id
jsonb labels
UUID source_document_id
bool deprecated
}
vulnerability {
String id
String title
timestamp published
timestamp modified
timestamp withdrawn
timestamp reserved
text[] cwes
}
advisory_vulnerability {
UUID advisory_id
String vulnerability_id
String title
String summary
String description
timestamp discovery_date
timestamp release_date
timestamp reserved_date
text[] cwes
}
sbom {
UUID sbom_id
String node_id
String document_id
timestamp published
String[] authors
jsonb labels
UUID source_document_id
text[] data_licenses
}
product {
UUID id
UUID vendor_id
String name
String cpe_key
}
product_version {
UUID id
UUID product_id
UUID sbom_id
String version
timestamp timestamp
}
base_purl {
UUID id
timestamp timestamp
String type
String namespace
String name
}
versioned_purl {
UUID id
UUID base_purl_id
String version
timestamp timestamp
}
qualified_purl {
UUID id
UUID versioned_purl_id
jsonb qualifiers
jsonb purl
timestamp timestamp
}
status {
UUID id
String slug
String name
String description
}
purl_status {
UUID id
UUID advisory_id
UUID status_id
UUID base_purl_id
UUID version_range_id
String vulnerability_id
UUID context_cpe_id
}
product_status {
UUID id
UUID advisory_id
String vulnerability_id
UUID status_id
UUID product_version_range_id
UUID context_cpe_id
String package
}
license {
UUID id
String text
text[] spdx_licenses
text[] spdx_license_exceptions
}
purl_license_assertion {
UUID id
UUID license_id
UUID sbom_id
UUID versioned_purl_id
}
cpe_license_assertion {
UUID id
UUID license_id
UUID sbom_id
UUID cpe_id
}
cpe {
UUID id
String part
String vendor
String product
String version
String update
String edition
String language
String sw_edition
String target_sw
String target_hw
String other
}
advisory ||--o{ advisory_vulnerability : has
vulnerability ||--o{ advisory_vulnerability : reported_in
product ||--o{ product_version : has
product_version }o--|| sbom : documented_by
base_purl ||--o{ versioned_purl : has_versions
versioned_purl ||--o{ qualified_purl : has_qualifiers
advisory ||--o{ purl_status : affects
advisory ||--o{ product_status : affects
status ||--o{ purl_status : classifies
status ||--o{ product_status : classifies
license ||--o{ purl_license_assertion : asserted_for
license ||--o{ cpe_license_assertion : asserted_for
purl_license_assertion }o--|| versioned_purl : applies_to
purl_license_assertion }o--|| sbom : in_context
cpe_license_assertion }o--|| cpe : applies_to
cpe_license_assertion }o--|| sbom : in_context
Class diagram for PURL hierarchy and license assertionsclassDiagram
class base_purl {
+UUID id
+timestamp timestamp
+String type
+String namespace
+String name
}
class versioned_purl {
+UUID id
+UUID base_purl_id
+String version
+timestamp timestamp
}
class qualified_purl {
+UUID id
+UUID versioned_purl_id
+jsonb qualifiers
+jsonb purl
+timestamp timestamp
}
class sbom {
+UUID sbom_id
+String node_id
+String document_id
+timestamp published
+String[] authors
+jsonb labels
+UUID source_document_id
+text[] data_licenses
}
class sbom_package {
+UUID sbom_id
+String node_id
+String version
}
class sbom_package_purl_ref {
+UUID sbom_id
+String node_id
+UUID qualified_purl_id
}
class cpe {
+UUID id
+String part
+String vendor
+String product
+String version
+String update
+String edition
+String language
+String sw_edition
+String target_sw
+String target_hw
+String other
}
class sbom_package_cpe_ref {
+UUID sbom_id
+String node_id
+UUID cpe_id
}
class license {
+UUID id
+String text
+text[] spdx_licenses
+text[] spdx_license_exceptions
}
class purl_license_assertion {
+UUID id
+UUID license_id
+UUID sbom_id
+UUID versioned_purl_id
}
class cpe_license_assertion {
+UUID id
+UUID license_id
+UUID sbom_id
+UUID cpe_id
}
%% PURL hierarchy
base_purl "1" --> "*" versioned_purl : has_versions
versioned_purl "1" --> "*" qualified_purl : has_qualifiers
%% SBOM package references
sbom "1" --> "*" sbom_package : contains
sbom_package "1" --> "*" sbom_package_purl_ref : has
sbom_package "1" --> "*" sbom_package_cpe_ref : has
%% CPE relations
cpe "1" --> "*" sbom_package_cpe_ref : referenced_in
cpe "1" --> "*" cpe_license_assertion : has
%% License assertions
license "1" --> "*" purl_license_assertion : asserted_for
license "1" --> "*" cpe_license_assertion : asserted_for
purl_license_assertion "*" --> "1" versioned_purl : applies_to
purl_license_assertion "*" --> "1" sbom : in_context
cpe_license_assertion "*" --> "1" sbom : in_context
Class diagram for advisory, vulnerability, product, and status modelclassDiagram
class advisory {
+UUID id
+UUID issuer_id
+timestamp published
+timestamp modified
+timestamp withdrawn
+String identifier
+String title
+String version
+String document_id
+jsonb labels
+UUID source_document_id
+bool deprecated
}
class advisory_vulnerability {
+UUID advisory_id
+String vulnerability_id
+String title
+String summary
+String description
+timestamp discovery_date
+timestamp release_date
+timestamp reserved_date
+text[] cwes
}
class vulnerability {
+String id
+String title
+timestamp published
+timestamp modified
+timestamp withdrawn
+timestamp reserved
+text[] cwes
}
class vulnerability_description {
+UUID id
+String vulnerability_id
+UUID advisory_id
+String lang
+String description
+timestamp timestamp
}
class weakness {
+text id
+text description
+text extended_description
+text[] child_of
+text[] parent_of
+text[] starts_with
+text[] can_follow
+text[] can_precede
+text[] required_by
+text[] requires
+text[] can_also_be
+text[] peer_of
}
class cvss3 {
+int minor_version
+UUID advisory_id
+String vulnerability_id
+cvss3_av av
+cvss3_ac ac
+cvss3_pr pr
+cvss3_ui ui
+cvss3_s s
+cvss3_c c
+cvss3_i i
+cvss3_a a
+double score
+cvss3_severity severity
}
class cvss4 {
+int minor_version
+UUID advisory_id
+String vulnerability_id
+cvss4_av av
+cvss4_ac ac
+cvss4_at at
+cvss4_pr pr
+cvss4_ui ui
+cvss4_vc vc
+cvss4_vi vi
+cvss4_va va
+cvss4_sc sc
+cvss4_si si
+cvss4_sa sa
}
class organization {
+UUID id
+String name
+String cpe_key
+String website
}
class product {
+UUID id
+UUID vendor_id
+String name
+String cpe_key
}
class product_version {
+UUID id
+UUID product_id
+UUID sbom_id
+String version
+timestamp timestamp
}
class product_version_range {
+UUID id
+UUID product_id
+UUID version_range_id
+String cpe_key
}
class status {
+UUID id
+String slug
+String name
+String description
}
class purl_status {
+UUID id
+UUID advisory_id
+UUID status_id
+UUID base_purl_id
+UUID version_range_id
+String vulnerability_id
+UUID context_cpe_id
}
class product_status {
+UUID id
+UUID advisory_id
+String vulnerability_id
+UUID status_id
+UUID product_version_range_id
+UUID context_cpe_id
+String package
}
class version_range {
+UUID id
+String version_scheme_id
+String low_version
+bool low_inclusive
+String high_version
+bool high_inclusive
}
class version_scheme {
+String id
+String name
+String description
}
class base_purl {
+UUID id
+timestamp timestamp
+String type
+String namespace
+String name
}
class cpe {
+UUID id
+String part
+String vendor
+String product
+String version
+String update
+String edition
+String language
+String sw_edition
+String target_sw
+String target_hw
+String other
}
class sbom {
+UUID sbom_id
+String node_id
+String document_id
+timestamp published
+String[] authors
+jsonb labels
+UUID source_document_id
+text[] data_licenses
}
%% Advisory and vulnerability relations
advisory "1" --> "*" advisory_vulnerability : has
advisory "1" --> "*" cvss3 : has_scores
advisory "1" --> "*" cvss4 : has_scores
advisory "1" --> "*" purl_status : affects
advisory "1" --> "*" product_status : affects
advisory "1" --> "*" vulnerability_description : has
vulnerability "1" --> "*" advisory_vulnerability : reported_in
vulnerability "1" --> "*" vulnerability_description : has
vulnerability "1" --> "*" cvss3 : has_scores
vulnerability "1" --> "*" cvss4 : has_scores
weakness "1" --> "*" vulnerability : categorizes
%% Product hierarchy
organization "1" --> "*" product : owns
product "1" --> "*" product_version : has
product "1" --> "*" product_version_range : has
product_version "*" --> "0..1" sbom : documented_by
product_version_range "*" --> "1" version_range : uses
%% Status tracking
purl_status "*" --> "1" base_purl : applies_to
purl_status "*" --> "1" status : has
purl_status "*" --> "1" advisory : from
purl_status "*" --> "1" version_range : version_constraint
purl_status "*" --> "0..1" cpe : context
product_status "*" --> "1" status : has
product_status "*" --> "1" advisory : from
product_status "*" --> "1" product_version_range : applies_to
product_status "*" --> "0..1" cpe : context
%% Version scheme
version_range "*" --> "1" version_scheme : uses_scheme
File-Level Changes
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
![sourcery-ai[bot]](https://avatars.githubusercontent.com/in/48477?s=60&v=4){kind=small}
There was a problem hiding this comment.
Hey - I've left some high level feedback:
- Consider adding a short legend/section explaining the meaning of the data types used in the diagram (e.g., String vs text vs jsonb, timestamp vs timestamptz) so that readers can more easily map them to the actual database types and semantics.
- Some entities mix String-based identifiers and UUID-based identifiers for
idfields (e.g.,vulnerability.idas String vs most others as UUID); it may help to briefly call out where String IDs represent external identifiers (like CVE IDs) to avoid confusion with internal primary keys.
Prompt for AI Agents
Please address the comments from this code review:
## Overall Comments
- Consider adding a short legend/section explaining the meaning of the data types used in the diagram (e.g., String vs text vs jsonb, timestamp vs timestamptz) so that readers can more easily map them to the actual database types and semantics.
- Some entities mix String-based identifiers and UUID-based identifiers for `id` fields (e.g., `vulnerability.id` as String vs most others as UUID); it may help to briefly call out where String IDs represent external identifiers (like CVE IDs) to avoid confusion with internal primary keys.Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #2222 +/- ##
==========================================
+ Coverage 68.37% 68.38% +0.01%
==========================================
Files 422 422
Lines 24681 24681
Branches 24681 24681
==========================================
+ Hits 16875 16879 +4
+ Misses 6886 6876 -10
- Partials 920 926 +6 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
|
Wouldn't an ER diagram be a better fit for this? Also, why does it have another diagram as PNG? And how do we ensure this stays in sync? Which tool was used to create this? Can the resolution be enhanced. |
You're right, I've changed it to an ER diagram. This was actually an intermediary stage as I had to create the domain model (added now) by reverse engineer the data model.
Having diagrams exported as a png files provides extra visualization options offering flexibility to communicate about the project. |
gildub
force-pushed
the
data-model-diagram
branch
3 times, most recently
from
7537a59 to
a8ac001
Compare
gildub
changed the title
gildub
force-pushed
the
data-model-diagram
branch
2 times, most recently
from
5d6d8f0 to
6c5567c
Compare
|
Thanks for explaining this. I still don't see the benefit of having the (quite small) PNG, compared to the complexity it adds. And the problems when things go out of alignment. I think it's easy enough to view the diagram on GitHub, and at least in IntelliJ too. I'd assume VScode has something similar. So which case would require having this PNG? |
|
The PNG (or SVG, JPEG) are definitely optional and I understand everyone might have different needs to use them or not. I removed the images and let's keep only the instruction for anyone to generate them if needed. |
Could it be that you forgot to push those changes, as they still seem to be part of the PR. |
|
Just to understand the process, whenever we do changes in the codebase, how to we update these diagrams? |
gildub
force-pushed
the
data-model-diagram
branch
from
6c5567c to
49372a2
Compare
|
@ctron, sorry I missed the change, now the generated images are removed. Regarding the process of updating the diagrams, I believe that should be initiated from a new ADR. |
gildub
force-pushed
the
data-model-diagram
branch
from
49372a2 to
58ae3a3
Compare
Yes, how would that relate to the folder https://github.com/guacsec/trustify/blob/main/docs/design/ |
|
nice content 👍 |
|
@ctron, I believe the ADR documents approach was put in place afterward the start of the project, therefore there is no "initial" ADR which paved the design document. The intent of this PR is to fill the gap and give us that initial design document. Following ADRs will have the responsibility to amend the main design document to reflect what they bring to the table. That said, when adding ADR we could discover that some part of the main design would be better if dropped or broken down to avoid excessive work, then we'll adapt. |
gildub
force-pushed
the
data-model-diagram
branch
from
58ae3a3 to
e416b1f
Compare
|
The point I was trying to make was: This PR visualizes a good portion of the architecture. The folder I linked, describes parts, but much less. Now having both, this would probably go out of date pretty soon, if that not already happened. Also meaning, that future ADRs would need to work through all those documents and make changes where relevant. Not a bad thing, but making things much more complicated, which will result in much more wrong information. If we want to consolidate, then we should consolidate, but for me, this would mean removing existing files and incorporating content which makes sense, removing content that no longer makes sense. For example, this PR described the entities of the product related things: https://github.com/guacsec/trustify/pull/2222/changes#diff-143377cacdfc230e96d3048e1070942816f3c09f0646394306edb95d3f90ec4bR227-R254 But we already have that: https://github.com/guacsec/trustify/blob/main/docs/design/products.md Now we do have two, possibly conflicting, documents. In the same folder. This should be cleaned up IMHO. Or, we do have a description on how the SBOM tables work: https://github.com/guacsec/trustify/blob/main/docs/design/sbom.md However, this PR adds the some information: https://github.com/guacsec/trustify/pull/2222/changes#diff-143377cacdfc230e96d3048e1070942816f3c09f0646394306edb95d3f90ec4bR140-R209 … but lacks the description. This means, that we are spreading information, and it will become harder and harder to maintain this. Also, if we make it mandatory to update these documents when creating ADRs (which I'm not against) then I'd like to see some feedback from previous ADR authors about this. |
| --- | ||
| title: Trustify Domain model | ||
| --- | ||
| flowchart TB |
| sbom ||--o{ sbom_node : "contains" | ||
| sbom ||--o{ sbom_package : "has_packages" | ||
| sbom ||--o{ sbom_file : "has_files" | ||
| sbom ||--o{ sbom_node_checksum : "has_checksums" |
There was a problem hiding this comment.
I don't think that's correct. The sbom_node_checksum related to the sbom_node.
| sbom ||--o{ sbom_package : "has_packages" | ||
| sbom ||--o{ sbom_file : "has_files" | ||
| sbom ||--o{ sbom_node_checksum : "has_checksums" | ||
| sbom ||--o{ sbom_external_node : "references_external" |
There was a problem hiding this comment.
The sbom_external_node relates to the sbom_node.
| sbom ||--o{ sbom_node_checksum : "has_checksums" | ||
| sbom ||--o{ sbom_external_node : "references_external" | ||
| sbom ||--o{ sbom_external_node : "referenced_by" | ||
| sbom ||--o{ package_relates_to_package : "defines_relationships" |
There was a problem hiding this comment.
package_relates_to_package relates to sbom_package.
| sbom ||--o{ sbom_external_node : "references_external" | ||
| sbom ||--o{ sbom_external_node : "referenced_by" | ||
| sbom ||--o{ package_relates_to_package : "defines_relationships" | ||
| sbom ||--o{ purl_license_assertion : "asserts_purl_license" |
There was a problem hiding this comment.
Not exactly sure how that relates, but pretty sure it does not relate to the sbom.
| sbom ||--o{ sbom_external_node : "referenced_by" | ||
| sbom ||--o{ package_relates_to_package : "defines_relationships" | ||
| sbom ||--o{ purl_license_assertion : "asserts_purl_license" | ||
| sbom ||--o{ cpe_license_assertion : "asserts_cpe_license" |
There was a problem hiding this comment.
Not exactly sure how that relates, but pretty sure it does not relate to the sbom.
|
|
||
| qualified_purl ||--o{ sbom_package_purl_ref : "referenced_in" | ||
|
|
||
| %% Relationships - SBOM |
There was a problem hiding this comment.
Just doing a review of the SBOM section, I think it got a lot of things wrong. However, they seem to be described in https://github.com/guacsec/trustify/blob/main/docs/design/sbom.md … Those should be aligned, and in a single spot. Otherwise we will have conflicting information in the same folder.
The other sections should be re-checked as well.
gildub
force-pushed
the
data-model-diagram
branch
from
e416b1f to
24b2afb
Compare
3 participants
The design/README.md articulates together the design information of the Trustify project using Mermaid providing diagram as code with the following :
mermaid-clitool in order to regenerate images (PNG, SVG or PDF) along with the default theme and scale parameters.