Capstone: From Research Question to Publishable Project

Module J: Research methods and capstone. Lesson 20 of 20, 30 min.

Abstract

The capstone turns the Advanced tracks into one piece of research that others can read, run and build on. This lesson covers framing a falsifiable question, a logged literature review, reproducible data and models, ethics and consent, six example briefs, a 12-week plan, the assessment rubric, and how to write and demo for AES, ICASSP, ISMIR, DAFx and ICAD.

By the end you can

  • Frame a falsifiable research question with population, comparison, measure and smallest effect of interest
  • Conduct and document a structured literature review that identifies a defensible research gap
  • Package reproducible artifacts with experiment configs, seeds, datasheets, model cards and FAIR data practices
  • Plan, execute and present a capstone against a rubric, targeting an appropriate venue with an honest immersive demo

The capstone is where everything in Academy Advanced turns into one piece of work that other people can read, run and build on. This lesson walks through the research workflow from a sharp question to a submitted paper: literature review, reproducible experiments, ethics, milestones, venue choice and the immersive demo that sells the result. Treat it as both a guide and the contract your project will be assessed against.

What makes a capstone publishable

Most student projects that fail review fail for the same few reasons. The question is vague. The contribution is not placed against prior work. The evaluation cannot support the claim. Or nobody else can reproduce the result. None of these is about talent. Each one is a process failure, and each can be prevented with a specific habit.

A publishable capstone in immersive audio has five properties:

  • A falsifiable question. "Is a personalized HRTF rated as more externalized than a generic one for upmixed pop music?" can be answered. "Explore HRTFs" cannot.
  • A clear contribution. This can be a new method, a new dataset, a new evaluation, or a careful replication. Replications and negative results count if they are rigorous.
  • Evaluation matched to the claim. Perceptual claims need listening tests, which the listening test methods lesson in this module covered. Algorithmic claims need baselines, ablations and held-out data.
  • Reproducible artifacts. Code, configurations, seeds, data documentation and stimuli are released so that someone else can rerun the work.
  • Communication that fits the venue. The paper is written for a specific community and the demo is designed for its audience.

The rest of this lesson turns those properties into a workflow, six example briefs that draw on modules A to G, a 12-week milestone plan and the rubric used to grade your submission.

The rest of this lesson, its lab and self-check are part of Academy Advanced, a paid course. See who has access.

Key readings

  • Pineau, J., Vincent-Lamarre, P., Sinha, K., Larivière, V., Beygelzimer, A., d'Alché-Buc, F., Fox, E., & Larochelle, H. (2021). Improving Reproducibility in Machine Learning Research (A Report from the NeurIPS 2019 Reproducibility Program). Journal of Machine Learning Research, 22(164), 1-20.
  • Wilkinson, M. D., Dumontier, M., Aalbersberg, I. J., et al. (2016). The FAIR Guiding Principles for scientific data management and stewardship. Scientific Data, 3, 160018.
  • Gebru, T., Morgenstern, J., Vecchione, B., Vaughan, J. W., Wallach, H., Daumé III, H., & Crawford, K. (2021). Datasheets for Datasets. Communications of the ACM, 64(12), 86-92.
  • Mitchell, M., Wu, S., Zaldivar, A., Barnes, P., Vasserman, L., Hutchinson, B., Spitzer, E., Raji, I. D., & Gebru, T. (2019). Model Cards for Model Reporting. Proceedings of the Conference on Fairness, Accountability, and Transparency (FAT*).
  • Page, M. J., McKenzie, J. E., Bossuyt, P. M., et al. (2021). The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ, 372, n71.