Guidance about conducting scientific research.
Scientists produce deliverables like publications and software artifacts. Creating the deliverables assumes many research skills and awareness of procedural principles. The goal of this list is to make those assumed skills and principles explicit.
For example, a research paper is a specialized form of writing. Junior researchers, who want to write research papers, must first internalize the 'specification' of a research paper. Ideally, junior researchers learn these skills during doctoral studies. However, in reality, doctoral study experiences vary wildly in environment, mentorship quality, etc. Therefore, this collection provides a standardizing public resource, for junior researchers and advisors, that is independent of immediate circumstances. Most entries are drawn from computer science.
This list is related to other official awesome lists (like scientific writing and research tools), but is unique in its emphasis of guidance over technology. There are also several complementary web collections1 mostly written by a single author. In contrast, this list presents advice from multiple experts, in rich formats, and enables collaborative refinement over time. Apart from books, the list items are open access.
- Written Communication
- Giving Talks & Presentations
- Software Artifact & Technical Guides
- Peer Review & Publishing Ethics
- Grants & Funding
- Research Job Search
- Meta Discussions
- Books about Scientific Skills
- Unwinding
- Advice to Authors of Extended Abstracts - What is an extended abstract and what information should it contain.
- Conclusions - Advice on writing effective conclusions.
- Defining the Role of Authors and Contributors - General recommendations for resolving uncertainties about authorship.
- How to Read a Technical Paper - Advice on selecting the reading, scheduling the reading, and taking notes.
- How to Write Mathematics - An expository essay on how to write mathematics (spoiler: there is no recipe).
- How to Write a Technical Paper - Strategies for effective writing of research papers.
- How to read a paper - The "three-pass method" for reading research papers.
- Imagine the Reader - Insights about writing, when approaching it from the perspective of the reader.
- Learn Technical Writing in Two Hours per Week - A student guide explaining the mechanics of a technical writing group.
- Mathematical Writing - Transcribed course lecture notes on technical writing and the effective presentation of mathematics and computer science.
- Semantic Line Breaks - A writing style where each substantial unit of thought is separated.
- Shitty First Drafts - An illuminating characterization of the process of writing drafts.
- Small Guide to Making Nice Tables - A focused how-to to crafting more legible tables in scientific writing.
- Teach Technical Writing in Two Hours per Week - A teaching guide for motivations and mechanics of running a writing group.
- The Craft of Writing Effectively - Compelling advice to improve scholarly writing, independent of discipline.
- Three Sins of Authors in Computer Science and Math - Advice on avoiding some common flaws in introductions and conclusions.
- What it's like to be a POPL referee; or how to write an extended abstract so that it is more likely to be accepted - Writing advice through the eyes of a referee.
- Writing Papers - A legendary lecture about writing research papers.
- Writing and Speaking with Style - An immersive course in effective technical writing and speaking.
- Writing good software engineering research papers - A mini tutorial about writing research papers, illustrated through statistics of submitted papers.
- How (not) to give a great research talk - Tips for crafting lively and engaging presentations.
- How I Give a Talk - Using storytelling techniques to create engaging oral presentations.
- How to Give a Great Research Talk - Simple actionable advice for giving better talks.
- How to Present a Paper in Theoretical Computer Science - Thematically organized advice on designing and delivering presentations.
- How to Speak - Ideas and techniques to incorporate into your speaking toolkit.
- How to give a technical presentation - Written advice about preparing and practicing for a scientific presentation, including talk-delivery and handling the Q&A session.
- Making a technical poster - A list of things to consider while preparing and presenting a technical poster, with example posters.
- Ten simple rules for effective presentation slides - A rule-based "checklist" for validating your slide design.
- The Elevator Pitch: Presenting Your Research in Conversation - Tips for crafting and delivering an elevator pitch in different situations.
- ACM Artifact Review and Badging - The current badging guidelines for judging scientific software artifacts.
- Artifact Evaluation: Tips for Authors - Ten experience-based tips, with justification and examples, for creating software artifacts.
- BenchExec - A framework for reliable benchmarking of non-interactive tools, with built-in resource control and a table generator for visualizing results.
- Benchmarking Crimes - A synopsis of the many ways an experiment design or analysis can go wrong.
- Can you trust your experimental results? - A general framework for validating experimental designs; a technical report developed based on the Evaluate 2011 workshop.
- EAPLS Artifact Badges - The European badging scheme for software artifact evaluation.
- Empirical Evaluation Guidelines - A checklist to evaluate soundness of scientific experiment setup, developed.
- Empirical Standards for Software Engineering research - The official evidence standards for conducting and reporting studies in software engineering; developed.
- Guide for Accelerating Computational Reproducibility in the Social Sciences - A structured guidebook toward assessing and improving computational reproducibility.
- Handbook for Reproduction and Replication Studies - A practical how-to guide for how to carry out a reproduction or a replication study.
- Proof Artifacts: Guidelines for Submission and Reviewing - Proof artifacts are a special category of scientific software and thus have their own presentation standards; the guidelines are maintained.
- Reliable benchmarking: requirements and solutions - Motivations for reliable benchmarking and presentation of BenchExec.
- STABILIZER: statistically sound performance evaluation - Addressing the bias that commonly arises in measurements of effect size, i.e., the magnitude of change in performance.
- Scientific benchmarking of parallel computing systems: twelve ways to tell the masses when reporting performance results - 12 rules of best practices for reporting empirical evaluation results.
- A Guide for New Program Committee Members at Theoretical Computer Science Conferences - A guide to help new program-committee members to understand how the system works, with useful tips and guidelines.
- A Guide for New Referees in Theoretical Computer Science - Advice about peer reviewing research publication drafts, for students and other inexperienced referees.
- After grad student suicide, misconduct findings, university suspends professor - A cautionary tale and a recount of events that exposed a serious case of research fraud.
- Beall's List - A cached copy listing potential predatory journals and publishers.
- Collusion rings threaten the integrity of computer science research - Experiences discovering attempts to subvert the peer-review process.
- Cornell Food Researcher's Downfall Raises Larger Questions For Science - This should be the cautionary tale discussed in Research Methods 101 across many disciplines.
- Green, Gold, and Diamond Open Access - A guide to understanding the different publication access models.
- How NOT to review a paper: the tools and techniques of the adversarial reviewer - A delightful exposition about reviewing, from the perspective of the adversarial reviewer.
- Introduction to Research Metrics - Explanation of research metrics like impact factor, h-index, g-index, and altmetrics.
- Low-CO2 research paper - An initiative to advertise the low environmental impact of research work.
- Mistakes Reviewers Make - Reducing the learning curve for new reviewers, and helping them improve their reviewing, through exposing some common mistakes.
- No free view? No review! - A public pledge to abstain from peer reviewing for editors who hide publications behind paywalls.
- Reviewer-Author Collusion Rings and How to Fight Them - A list of measures to prevent and mitigate potential collusion rings.
- Some Lessons on Reviews and Rebuttals - Advice about writing reviews and review rebuttals.
- The task of the referee - A structured framework for evaluating drafts and writing referee reports (reviews).
- Checklist for effectively writing grants in Stat-ML - A generally applicable list of suggestions for assistant professors.
- How to write a great research proposal - Discussion about how to effectively present scientific project ideas, including: how to approach a proposal, the judgment criteria, and common shortcomings.
- Academic Interviews (Faculty Positions) - Outline of expectations for faculty interviews and campus visits.
- Academic Interviews and Offers Panel Discussion - MIT professors speak about academic job search process and effective interview strategies.
- Application Materials for a Faculty Job Search - Descriptions of commonly requested statements when you applying for faculty jobs in any department and/or discipline in the United States.
- Demystifying PhD Admissions in Computer Science - A comprehensive manual for navigating computer science PhD admissions in the United States.
- Faculty Job Interview Questions - A concise handout of potential job interview questions for a faculty position.
- Faculty Job Search Guide - A structured mini-course on early faculty job search in the United States, with timelines and advice on preparing application documents.
- Getting an academic job - Information about university faculty job application and interview process in the United States.
- Guide for the Tenure-Track Job Market in Computer/Information Sciences - About seeking a faculty position for research-oriented tenure track positions in the United States.
- Negotiating Your Job Offer(s) Inside & Outside Academia - Includes a list of various aspects that can be negotiated during a job offer.
- Tips for Computer Science Faculty Applications - A guide with examples about how to prepare a faculty application package for computer science positions.
- Dancing With Professors: The Trouble With Academic Prose - A classic dissection about the challenges of academic writing, from 1993 New York Times Book Review.
- Getting Your Research Adopted - Advice about how to help others discover your research.
- How Science Goes Wrong - Discussion structural bias of science, driven by e.g., failures to replicate studies and flawed peer review.
- How to succeed in graduate school: a guide for students and advisors: part I of II - About optimizing the outcomes in a student-advisor relationship.
- The Three Golden Rules for Successful Scientific Research - Timeless grounding advice for those seeking respectable success in scientific research.
- The machines are fine. I'm worried about us. - Thoughtful reflections about how large language models are influencing science.
- Houston, We Have a Narrative: Why Science Needs Story - An handbook that teaches scientists how to tell effective stories about their work and why the narrative structure is needed.
- Style: Lessons in Clarity and Grace - Exercises for developing writing skills.
- Stylish Academic Writing - Advice on crafting clear, active sentences with concrete nouns and precise diction.
- The Chicago Manual of Style Online - An excellent guide to punctuation and many other writing issues.
- The Craft of Research - A comprehensive classic guide, with insights about, e.g., effective problem framing and use of flow in crafting persuasive arguments.
- The Elements of Eloquence: How to Turn the Perfect English Phrase - An excellent reference for elegant phrasing.
- The Sense of Style: The Thinking Person's Guide to Writing in the 21st Century - A modern guide that challenges to rethink writing in many settings, including science.
- Co-authorship distance computation - Discover connections between two authors in computer science (fun at conferences).
- PhD comics - Comics of many research-related experiences in graduate school.
- The PhD Simulator - A simulation game of the life and hurdles of an aspiring researcher.
- xkcd - A web comic, with many thoughtful insights, often about science.
A PDF version of this list if available here.
Is something missing? Your contributions can make the list better. Start by reading the contribution guidelines.
Footnotes
-
Advice by Jason Eisner, Jason I. Hong, Michael Ernst, etc. ↩