Skip to content

Commit d09e0a4

Browse files
Merge pull request #20 from pmeerw/ds-attrib
DATASETS.md: fix attribution, provide some links
2 parents 67a581c + ab6b54a commit d09e0a4

1 file changed

Lines changed: 25 additions & 12 deletions

File tree

DATASETS.md

Lines changed: 25 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -36,11 +36,10 @@ CoBRA classifies each input expression into one of four semantic classes and sel
3636

3737
### SiMBA Datasets
3838

39-
Source: [SiMBA](https://github.com/pgarba/SiMBA-)
40-
4139
#### Expression Templates (E-Series)
4240

4341
Each file contains 1,000 obfuscated linear MBA expressions plus a header comment. All 16,000 expressions across 16 files simplify at 100%.
42+
From the [SiMBA](https://github.com/DenuvoSoftwareSolutions/SiMBA) paper.
4443

4544
| Dataset | Variables | Expressions | Simplified | Rate |
4645
|---------|:---------:|:-----------:|:----------:|:----:|
@@ -64,23 +63,36 @@ Each file contains 1,000 obfuscated linear MBA expressions plus a header comment
6463

6564
#### PLDI Research Datasets
6665

66+
Originally from the [MBA-Solver](https://github.com/softsec-unh/MBA-Solver) repository, published at [PLDI'21](https://pldi21.sigplan.org/details/pldi-2021-papers/43/Boosting-SMT-Solver-Performance-on-Mixed-Bitwise-Arithmetic-Expressions).
67+
Source: [SiMBA++](https://github.com/pgarba/SiMBA-)
68+
6769
| Dataset | Total Lines | Parsed | Simplified | Notes | Rate |
6870
|---------|:-----------:|:------:|:----------:|-------|:----:|
6971
| `pldi_linear.txt` | 1,012 | 1,008 | **1,008** | 4 comment headers skipped | **100%** |
7072
| `pldi_poly.txt` | 1,009 | 1,008 | **1,008** | 1 comment header skipped | **100%** |
71-
| `pldi_nonpoly.txt` | 1,004 | 1,003 | **1,003** | 1 comment header skipped | **100%** |
73+
| `pldi_nonpoly.txt` | 1,005 | 1,003 | **1,003** | 1 comment header skipped | **100%** |
7274

7375
#### Other SiMBA Datasets
7476

75-
| Dataset | Expressions | Parsed | Simplified | Rate |
77+
Originally from the [MBA-Blast](https://github.com/softsec-unh/MBA-Blast) repository, published at [USENIX Security'21](https://dl.acm.org/doi/10.1145/3453483.3454068).
78+
Source: [SiMBA++](https://github.com/pgarba/SiMBA-)
79+
80+
| Dataset | Total Lines | Parsed | Simplified | Rate |
7681
|---------|:-----------:|:------:|:----------:|:----:|
77-
| `test_data.txt` | 10,000 | 10,000 | **10,000** | **100%** |
7882
| `blast_dataset1.txt` | 63 | 62 | **62** | **100%** |
7983
| `blast_dataset2.txt` | 2,501 | 2,500 | **2,500** | **100%** |
8084

85+
[Neureduce](https://github.com/fvrmatteo/NeuReduce) dataset, published at [EMNLP'20](https://aclanthology.org/2020.findings-emnlp.56.pdf).
86+
Source: [SiMBA++](https://github.com/pgarba/SiMBA-)
87+
88+
| Dataset | Total Lines | Parsed | Simplified | Rate |
89+
|---------|:-----------:|:------:|:----------:|:----:|
90+
| `test_data.txt` | 10,000 | 10,000 | **10,000** | **100%** |
91+
92+
8193
### GAMBA Datasets
8294

83-
Source: [GAMBA](https://github.com/DenuvoSoftwareSolutions/GAMBA)
95+
Source: [GAMBA](https://github.com/DenuvoSoftwareSolutions/GAMBA) has a characterisation of the datasets.
8496

8597
| Dataset | Origin | Total Lines | Parsed | Simplified | Unsupported | Rate |
8698
|---------|--------|:-----------:|:------:|:----------:|:-----------:|:----:|
@@ -92,11 +104,12 @@ Source: [GAMBA](https://github.com/DenuvoSoftwareSolutions/GAMBA)
92104
| `qsynth_ea.txt` | QSynth | 501 | 500 | **466** | 34 | **93.2%** |
93105
| `mba_flatten.txt` | MBA-Flatten | 3,008 | 2,060 | **2,060** | 0 | **100%** |
94106

95-
- **loki_tiny**: 25 sections covering add, subtract, AND, OR, XOR at depths 1-5. All 25,000 are 2-variable linear MBAs.
96-
- **mba_obf_nonlinear**: 500 polynomial + 500 linear expressions, all with linear ground-truth targets. All 1,000 pass full-width verification.
97-
- **syntia**: All 500 expressions simplify via the orchestrator's decomposition and lifting passes.
98-
- **qsynth_ea**: The most challenging dataset. 466 of 500 expressions simplify. The 34 unsupported expressions are mixed bitwise-arithmetic expressions where CoB is boolean-correct but diverges at full width, and polynomial recovery (d=1..4) also fails — a genuine representation gap in carry-sensitive boolean-null residuals.
99-
- **mba_flatten**: 3,008 lines across 7 sections (2-4 variable linear, sub-expression, and unsolvable-by-other-tools categories). 948 lines skipped (section headers, 3-field sub-expression rows, and expressions that fail parse). All 2,060 parseable expressions simplify at 100%.
107+
- **loki_tiny**: 25 sections covering add, subtract, AND, OR, XOR at depths 1-5. All 25,000 are 2-variable linear MBAs. From the [Loki](https://github.com/RUB-SysSec/loki) repository, published at [USENIX Security'22](https://www.usenix.org/conference/usenixsecurity22/presentation/schloegel).
108+
- **neureduce.txt**: 10000 linear expressions with 2 to 5 variables. From the [Neureduce](https://github.com/fvrmatteo/NeuReduce) dataset, published at [EMNLP'20](https://aclanthology.org/2020.findings-emnlp.56.pdf).
109+
- **mba_obf_nonlinear** and **mba_obf_linear**: 1000 polynomial/nonpolynomial + 1000 linear expressions, all with linear ground-truth targets. All pass full-width verification. From the [MBA-Obfuscator](https://github.com/nhpcc502/MBA-Obfuscator) repo, published at [ICICS'21](https://dl.acm.org/doi/10.1007/978-3-030-86890-1_16).
110+
- **syntia**: All 500 expressions simplify via the orchestrator's decomposition and lifting passes. From the [QSynth](https://github.com/werew/qsynth-artifacts) repo, published at [BAR'20](https://archive.bar/pdfs/bar2020-preprint9.pdf).
111+
- **qsynth_ea**: The most challenging dataset. 466 of 500 expressions simplify. The 34 unsupported expressions are mixed bitwise-arithmetic expressions where CoB is boolean-correct but diverges at full width, and polynomial recovery (d=1..4) also fails — a genuine representation gap in carry-sensitive boolean-null residuals. From the [QSynth](https://github.com/werew/qsynth-artifacts) repo, published at [BAR'20](https://archive.bar/pdfs/bar2020-preprint9.pdf).
112+
- **mba_flatten**: 3,008 lines across 7 sections (2-4 variable linear, sub-expression, and unsolvable-by-other-tools categories). 948 lines skipped (section headers, 3-field sub-expression rows, and expressions that fail parse). All 2,060 parseable expressions simplify at 100%. From [MBA-Flatten](https://tinyurl.com/y5l948pu), published in [Security and Communication Networks'22](https://onlinelibrary.wiley.com/doi/10.1155/2022/7307139).
100113

101114
### OSES Dataset
102115

@@ -146,7 +159,7 @@ All simplified results are validated via spot-check (random-input evaluation) at
146159

147160
| Source | URL | Datasets |
148161
|--------|-----|----------|
149-
| SiMBA | https://github.com/pgarba/SiMBA- | E-series, PLDI, BLAST, test_data |
162+
| SiMBA++ | https://github.com/pgarba/SiMBA- | E-series, PLDI, BLAST, test_data |
150163
| GAMBA | https://github.com/DenuvoSoftwareSolutions/GAMBA | loki_tiny, neureduce, mba_obf_*, mba_flatten, syntia, qsynth_ea |
151164
| QSynth | (via GAMBA) | qsynth_ea |
152165
| NeuReduce | (via GAMBA) | neureduce |

0 commit comments

Comments
 (0)