Avoid appending old log files

[bschilder]

Reprex

 gwas_paths <- MungeSumstats::import_sumstats(ids = meta$id, 
                                       save_dir = save_dir, 
                                       nThread = 50, 
                                       parallel_across_ids = TRUE,
                                       force_new_vcf = TRUE,
                                       force_new = TRUE,
                                       #### Record logs
                                       log_folder_ind = TRUE,
                                       log_mungesumstats_msgs = TRUE)

Issue: part 1

If you run format_sumstats with the log file options set, but then try to later re-munge some of those files, the log files from the previous runs get appended with the output from the new runs.

Here, you can see where it finished the old run and started the new run right around the middle ("Importing VCF file: ..."):

Issue: part 2

Also, I noticed that If I start the above code, but then stop it midway, do some stuff in the R console , and then run it again, it will start recording my output from the console! (not the output of the munging process). This also gets appended to any pre-existing log files.

---

Session info

Details

[Al-Murphy]

I don't know if there is a good way around part 2 issue, this is an annoyance with the sink() function, you have to close connections to the file you are logging results to:

https://stackoverflow.com/questions/18730491/sink-does-not-release-file

[bschilder]

I wonder if closing all connections with this function might work:
https://www.rdocumentation.org/packages/base/versions/3.6.2/topics/on.exit

Specifically, i find base::closeAllConnections() to be the most comprehensive

[Al-Murphy]

Yep but you have to do it manually, if you quite MungeSumstats or it fails you have to call that yourself, no way for the function to I don't think

[bschilder]

Ok, I think I see what you mean. so like you have to wrap whatever function with on.exit(), as opposed to putting it within the function itself.

I'll keep looking around and see if there's a solution out there.