Skip to content

Commit 66756d2

Browse files
m-jahnm-jahn
authored
Merge pull request #2 from MPUSP/dev
feat: adding mappers, various improvements
2 parents a4831f3 + 87385b7 commit 66756d2

17 files changed

Lines changed: 625 additions & 42 deletions

‎.gitignore‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,7 @@ logs/**
44
.snakemake
55
.snakemake/**
66
.test/results/*
7+
.test/config/*
78
.vscode/*
89
*.Rdata
910
*.RData

‎.test/config/config.yml‎

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,3 +12,41 @@ get_genome:
1212
"RefSeq": "CDS",
1313
"Protein Homology": "CDS",
1414
]
15+
16+
fastp:
17+
extra: ""
18+
19+
mapping:
20+
tool: "bwa_mem2"
21+
bowtie2:
22+
index: ""
23+
extra: ""
24+
bwa_mem2:
25+
extra: ""
26+
sort: "samtools"
27+
sort_order: "coordinate"
28+
sort_extra: ""
29+
samtools_sort:
30+
extra: "-m 4G"
31+
samtools_index:
32+
extra: ""
33+
34+
mapping_stats:
35+
gffread:
36+
extra: ""
37+
rseqc_infer_experiment:
38+
extra: ""
39+
rseqc_bam_stat:
40+
extra: ""
41+
deeptools_coverage:
42+
genome_size: 1000
43+
extra: ""
44+
45+
variant_calling:
46+
bcftools_pileup:
47+
uncompressed: False
48+
extra: "--max-depth 1000 --min-BQ 20"
49+
bcftools_call:
50+
uncompressed: False
51+
caller: "-c"
52+
extra: ""

‎.test/config/samples.tsv‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
sample description read1 read2
2+
spya20 S. pyogenes A20 data/spy_A20_R1.fastq.gz data/spy_A20_R2.fastq.gz

‎.test/data/spy_A20_R1.fastq.gz‎

928 KB
Binary file not shown.

‎.test/data/spy_A20_R2.fastq.gz‎

1020 KB
Binary file not shown.

‎config/README.md‎

Lines changed: 53 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -3,16 +3,17 @@
33
This workflow is a best-practice workflow for mapping of reads to reference genomes, minimalistic and simple.
44
The workflow is built using [snakemake](https://snakemake.readthedocs.io/en/stable/) and consists of the following steps:
55

6-
1. Download genome reference from NCBI (`ncbi tools`), or use manual input (`fasta` format)
6+
1. Download genome reference from NCBI (`ncbi tools`), or use manual input (`fasta`, `gff` format)
77
2. Check quality of input read data (`FastQC`)
88
3. Trim adapters and apply quality filtering (`fastp`)
9-
4. Determine experiment type (`rseqc`)
10-
5. Map reads to reference genome using:
11-
1. Bowtie2 _or_
12-
2. BWA-MEM2 _or_
13-
3. STAR
14-
6. Evaluate mapping quality, quantify variations
15-
7. Collect statistics from tool output (`MultiQC`)
9+
4. Map reads to reference genome using:
10+
1. (`Bowtie2`)[http://bowtie-bio.sourceforge.net/bowtie2/manual.shtml] _or_
11+
2. (`BWA-MEM2`)[https://github.com/bwa-mem2/bwa-mem2] _or_
12+
3. (`STAR`)[https://github.com/alexdobin/STAR] (not implemented yet)
13+
5. Determine experiment type, get mapping stats (`rseqc`)
14+
6. Generate `bigwig` or `bedgaph` coverage profiles (`deeptools`)
15+
7. Quantify variations and SNPs (`bcftools`)
16+
8. Collect statistics from tool output (`MultiQC`)
1617

1718
## Running the workflow
1819

@@ -23,17 +24,53 @@ The sample sheet listing read input files needs to have the following layout:
2324

2425
| sample | description | read1 | read2 |
2526
| ------- | ----------- | ------------------- | ------------------- |
26-
| sample1 | strain XY | sample1.R1.fastq.gz | sample1.R2.fastq.gz |
27+
| sample1 | strain XY | sample1_R1.fastq.gz | sample1_R2.fastq.gz |
2728
| ... | ... | ... | ... |
2829

2930
### Parameters
3031

3132
This table lists all parameters that can be used to run the workflow.
3233

33-
| parameter | type | details | default |
34-
| --------------- | ---- | ------------------------------ | -------------------- |
35-
| **samplesheet** | | | |
36-
| path | str | path to samplesheet, mandatory | "config/samples.tsv" |
37-
| **get_genome** | | | |
38-
39-
TODO: finalize parameters.
34+
| parameter | type | details | default |
35+
| ------------------------------ | ------- | -------------------------------------------------------- | ------- |
36+
| **samplesheet** | string | path to the sample sheet in tsv format | |
37+
| **get_genome** | object | genome retrieval options | |
38+
| database | string | database to use for genome retrieval, 'ncbi' or 'manual' | |
39+
| assembly | string | assembly version to use for genome retrieval | |
40+
| fasta | string | path to a custom FASTA file (optional) | |
41+
| gff | string | path to a custom GFF file (optional) | |
42+
| gff_source_type | array | mapping of GFF source types to feature types | |
43+
| **fastp** | object | Fastp options | |
44+
| extra | string | additional arguments to Fastp | |
45+
| **mapping** | object | mapping options | |
46+
| tool | string | mapping tool to use, one of 'bowtie2', 'bwa_mem2' | |
47+
| bowtie2 | object | Bowtie2 options | |
48+
| bowtie2.index | string | additional arguments to bowtie build | |
49+
| bowtie2.extra | string | additional arguments to bowtie align | |
50+
| bwa_mem2 | object | BWA-MEM2 options | |
51+
| bwa_mem2.extra | string | additional arguments to bwa-mem2 | |
52+
| bwa_mem2.sort | string | sorting tool to use, e.g. 'samtools' | |
53+
| bwa_mem2.sort_order | string | sorting order to use | |
54+
| bwa_mem2.sort_extra | string | additional arguments to the sorting tool | |
55+
| samtools_sort | object | Samtools sort options | |
56+
| samtools_sort.extra | string | additional arguments to Samtools sort | |
57+
| samtools_index | object | Samtools index options | |
58+
| samtools_index.extra | string | additional arguments to Samtools index | |
59+
| **mapping_stats** | object | mapping statistics options | |
60+
| gffread | object | GFFread options | |
61+
| gffread.extra | string | additional arguments to GFFread | |
62+
| rseqc_infer_experiment | object | RSeQC infer_experiment.py options | |
63+
| rseqc_infer_experiment.extra | string | additional arguments to RSeQC infer_experiment.py | |
64+
| rseqc_bam_stat | object | RSeQC bam_stat.py options | |
65+
| rseqc_bam_stat.extra | string | additional arguments to RSeQC bam_stat.py | |
66+
| deeptools_coverage | object | DeepTools bamCoverage options | |
67+
| deeptools_coverage.genome_size | integer | genome size in base pairs | |
68+
| deeptools_coverage.extra | string | additional arguments to DeepTools bamCoverage | |
69+
| **variant_calling** | object | variant calling options | |
70+
| bcftools_pileup | object | BCFtools pileup options | |
71+
| bcftools_pileup.uncompressed | boolean | whether to output uncompressed BCF files | |
72+
| bcftools_pileup.extra | string | additional arguments to BCFtools pileup | |
73+
| bcftools_call | object | BCFtools call options | |
74+
| bcftools_call.uncompressed | boolean | whether to output uncompressed VCF files | |
75+
| bcftools_call.caller | string | use '-c' for consensus or '-m' for multiallelic | |
76+
| bcftools_call.extra | string | additional arguments to BCFtools call | |

‎config/config.yml‎

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,3 +12,41 @@ get_genome:
1212
"RefSeq": "CDS",
1313
"Protein Homology": "CDS",
1414
]
15+
16+
fastp:
17+
extra: ""
18+
19+
mapping:
20+
tool: "bwa_mem2"
21+
bowtie2:
22+
index: ""
23+
extra: ""
24+
bwa_mem2:
25+
extra: ""
26+
sort: "samtools"
27+
sort_order: "coordinate"
28+
sort_extra: ""
29+
samtools_sort:
30+
extra: "-m 4G"
31+
samtools_index:
32+
extra: ""
33+
34+
mapping_stats:
35+
gffread:
36+
extra: ""
37+
rseqc_infer_experiment:
38+
extra: ""
39+
rseqc_bam_stat:
40+
extra: ""
41+
deeptools_coverage:
42+
genome_size: 1000
43+
extra: ""
44+
45+
variant_calling:
46+
bcftools_pileup:
47+
uncompressed: False
48+
extra: "--max-depth 1000 --min-BQ 20"
49+
bcftools_call:
50+
uncompressed: False
51+
caller: "-c"
52+
extra: ""

‎config/schemas/config.schema.yml‎

Lines changed: 147 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,147 @@
1+
$schema: "http://json-schema.org/draft-07/schema#"
2+
description: configuration file for the analysis pipeline
3+
properties:
4+
samplesheet:
5+
type: string
6+
description: path to the sample sheet in tsv format
7+
get_genome:
8+
type: object
9+
properties:
10+
database:
11+
type: string
12+
description: database to use for genome retrieval, 'ncbi' or 'manual'
13+
assembly:
14+
type: string
15+
description: assembly version to use for genome retrieval, e.g. 'GCF_000307535.1'
16+
fasta:
17+
type: [string, "null"]
18+
description: path to a custom FASTA file (optional)
19+
gff:
20+
type: [string, "null"]
21+
description: path to a custom GFF file (optional)
22+
gff_source_type:
23+
type: array
24+
description: mapping of GFF source types to feature types
25+
required:
26+
- database
27+
- assembly
28+
- gff_source_type
29+
fastp:
30+
type: object
31+
properties:
32+
extra:
33+
type: string
34+
description: additional arguments to pass to Fastp
35+
mapping:
36+
type: object
37+
properties:
38+
tool:
39+
type: string
40+
description: mapping tool to use, one of 'bowtie2', 'bwa_mem2'
41+
bowtie2:
42+
type: object
43+
properties:
44+
index:
45+
type: string
46+
description: additional arguments to bowtie build
47+
extra:
48+
type: string
49+
description: additional arguments to bowtie align
50+
bwa_mem2:
51+
type: object
52+
properties:
53+
extra:
54+
type: string
55+
description: additional arguments to bwa-mem2
56+
sort:
57+
type: string
58+
description: sorting tool to use, e.g. 'samtools'
59+
sort_order:
60+
type: string
61+
description: sorting order to use
62+
sort_extra:
63+
type: string
64+
description: additional arguments to pass to the sorting tool
65+
samtools_sort:
66+
type: object
67+
properties:
68+
extra:
69+
type: string
70+
description: additional arguments to pass to Samtools sort
71+
samtools_index:
72+
type: object
73+
properties:
74+
extra:
75+
type: string
76+
description: additional arguments to pass to Samtools index
77+
required:
78+
- tool
79+
mapping_stats:
80+
type: object
81+
properties:
82+
gffread:
83+
type: object
84+
properties:
85+
extra:
86+
type: string
87+
description: additional arguments to pass to GFFread
88+
rseqc_infer_experiment:
89+
type: object
90+
properties:
91+
extra:
92+
type: string
93+
description: additional arguments to pass to RSeQC infer_experiment.py
94+
rseqc_bam_stat:
95+
type: object
96+
properties:
97+
extra:
98+
type: string
99+
description: additional arguments to pass to RSeQC bam_stat.py
100+
deeptools_coverage:
101+
type: object
102+
properties:
103+
genome_size:
104+
type: integer
105+
description: genome size in base pairs
106+
extra:
107+
type: string
108+
description: additional arguments to pass to DeepTools bamCoverage
109+
required:
110+
- gffread
111+
- rseqc_infer_experiment
112+
- rseqc_bam_stat
113+
- deeptools_coverage
114+
variant_calling:
115+
type: object
116+
properties:
117+
bcftools_pileup:
118+
type: object
119+
properties:
120+
uncompressed:
121+
type: boolean
122+
description: whether to output uncompressed BCF files
123+
extra:
124+
type: string
125+
description: additional arguments to pass to BCFtools pileup
126+
bcftools_call:
127+
type: object
128+
properties:
129+
uncompressed:
130+
type: boolean
131+
description: whether to output uncompressed VCF files
132+
caller:
133+
type: string
134+
description: use '-c' for consensus or '-m' for multiallelic
135+
extra:
136+
type: string
137+
description: additional arguments to pass to BCFtools call
138+
required:
139+
- bcftools_pileup
140+
- bcftools_call
141+
required:
142+
- samplesheet
143+
- get_genome
144+
- fastp
145+
- mapping
146+
- mapping_stats
147+
- variant_calling

‎config/schemas/samples.schema.yml‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
$schema: "http://json-schema.org/draft-07/schema#"
2+
description: entries for the sample sheet
3+
properties:
4+
sample:
5+
type: string
6+
description: sample name/identifier
7+
description:
8+
type: string
9+
description: metadata or other information about sample
10+
read1:
11+
type: string
12+
description: path to fastq.gz files, read 1
13+
read2:
14+
type: string
15+
description: path to fastq.gz files, read 2 (optional)
16+
17+
required:
18+
- sample
19+
- description
20+
- read1

‎workflow/Snakefile‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,10 @@ configfile: "config/config.yml"
77
# -----------------------------------------------------
88
include: "rules/common.smk"
99
include: "rules/process_reads.smk"
10+
include: "rules/bowtie2.smk"
11+
include: "rules/bwamem2.smk"
12+
include: "rules/mapping_stats.smk"
13+
include: "rules/bcftools.smk"
1014
include: "rules/qc.smk"
1115

1216

0 commit comments

Comments
 (0)