Martin Klissarov

Martin Klissarov

I am a PhD student supervised by Prof. Doina Precup and Prof. Marlos C. Machado at McGill University and Mila. I am particularly interested in scalable human-inspired approaches to intelligence.

Humans are able to continually adapt in diverse and creative ways, can AI agents do the same?

My research is currently centered on the discovery and construction of temporal abstractions [skills / tools / programs] to answer this question. In particular, I'm interested in building abstractions from the ground-up by learning representations from interaction, as well as from the top-down by building on abstract knowledge contained in large models (e.g. LLMs).

Recently, I've been increasingly invested in the importance of building agents whose behaviour we can interpret and align.

Twitter | GitHub | Google Scholar | Email | CV

News

19/08/2024 Motif was featured in a recent article about AI agents and open-endedness by Air Street Press.

04/03/2023 Co-organizing a workshop on "Training Agents with Foundation Models" at RLC in August. We have some exciting things coming, stay tuned!

25/10/2023 The work resulting from my Meta internship, Motif, has been accepted at ICLR 2024! 🎶

25/10/2023 We have released the code for Motif, a general method to distill prior knowledge from LLMs into actionable RL agents. We have also published a short blogpost that describes Motif.

05/06/2023 Started a research intern at Meta Montreal with Shagun Sodhani, Mikael Henaff and Amy Zhang!

25/04/2023 Our work with Marlos, Deep Laplacian-based Options, has been accepted to ICML 2023! 🏝

31/01/2023 The project we worked on with Marlos C. Machado during my DeepMind intern has been released on arXiv! We scale up a promising approach for option discovery that yields strong exploration performance.

08/10/2022 Selected as a Top Reviewer at NeurIPS 2022! We all have a part to play in making our research community better, as much as time allows us.

14/09/2022 Our work on leveraging meta learning for emphatic algorithms was accepted at NeurIPS 2022!

11/07/2022 Started a research internship at DeepMind Alberta with Marlos C. Machado!

16/04/2022 Selected as a Highlighted Reviewer for ICLR 2022! Hopefully those reviews helped the authors.

15/10/2021 Selected as an Outstanding Reviewer (top 8%) for NeurIPS 2021! Amazing to be recognized for the time and thought put into the reviewing service.

28/09/2021 "Flexible Option Learning" has been accepted as a Spotlight to NeurIPS 2021! The paper and code will be out soon.

23/09/2021 Started a research internship at Amazon Web Services under the supervision of Alex Smola and Rasool Fakoor.

01/04/2021 Started a research internship at Microsoft Research Montreal under the supervision of Mehdi Fatemi and Romain Laroche.

Research

	Motif: Intrinsic Motivation from Artificial Intelligence Feedback Martin Klissarov, Pierluca D'Oro, Shagun Sodhani, Roberta Raileanu, Pierre-Luc Bacon, Pascal Vincent, Amy Zhang and Mikael Henaff International Conference on Learning Representations (ICRL), 2024 FMDM Workshop at NeurIPS 2023, Spotlight ALOE Workshop at NeurIPS 2023, Spotlight pdf \| bibtex \| code \| twitter \| blogpost
	Deep Laplacian-Based Options for Temporally-Extended Exploration Martin Klissarov and Marlos C. Machado International Conference on Machine Learning (ICML), 2023 webpage \| pdf \| bibtex \| code \| twitter \| blogpost
	Adaptive Interest for Emphatic Reinforcement Learning Martin Klissarov, Rasool Fakoor, Jonas Mueller, Kavosh Asadi, Taesup Kim and Alex Smola Neural Information Processing Systems (NeurIPS), 2022 DARL workshop at ICML 2022, Spotlight pdf \| bibtex
	Flexible Option Learning Martin Klissarov and Doina Precup Neural Information Processing Systems (NeurIPS), 2021, Spotlight pdf \| bibtex \| code
	Reward Propagation using Graph Convolutional Networks Martin Klissarov and Doina Precup Neural Information Processing Systems (NeurIPS), 2020, Spotlight pdf \| bibtex \| code
	Options of Interest: Temporal Abstraction with Interest Functions Khimya Khetarpal, Martin Klissarov, Maxime Chevalier-Boisvert, Pierre-Luc Bacon, Doina Precup Association for the Advancement of Artificial Intelligence (AAAI), 2020 webpage \| pdf \| bibtex \| code
	Variational State Encoding Martin Klissarov, Riashat Islam, Khimya Khetarpal, Doina Precup The Multi-disciplinary Conference on Reinforcement Learning and Decision Making (RLDM), 2019 pdf \| bibtex
	When Waiting is not an Option: Learning Options using the Deliberation Cost Jean Harb, Pierre-Luc Bacon, Martin Klissarov, Doina Precup Association for the Advancement of Artificial Intelligence (AAAI), 2018 pdf \| bibtex \| code
	Learning Options End-to-End for Continuous Action Tasks Martin Klissarov, Pierre-Luc Bacon, Jean Harb, Doina Precup Neural Information Processing Systems (NIPS) Hierarchichal Reinforcement Learning Workshop, 2017 pdf \| bibtex \| code

Education

	McGill University2020 - Present PhD in Computer Science Supervisor: Prof. Doina Precup and Prof. Marlos C. Machado
	McGill University2018 - 2020 MSc in Computer Science Supervisor: Prof. Doina Precup

Reviewing

TMLR: 2022 / 2023
NeurIPS: 2020 / 2021 (Outstanding reviewer) / 2022 (Top reviewer) / 2023
ICML: 2021 / 2022 / 2023
AAAI: 2021 / 2022
ICLR: 2022 (Highlighted reviewer) / 2023
CoLLAs: 2022

Organisation Comittee

CoLLAs: 2022

Selected Awards

Natural Sciences and Engineering Research Council of Canada (NSERC) Alexander Graham Bell Canada Graduate Scholarship (2020-2023)
Fonds de Recherche du Québec - Nature et Technologie (FRQNT) Masters Research Scholarship (2018-2020)
McGill Graduate Excellence Award (2018-2020)
Ordre des Ingénieurs Foundation Excellence Scholarship (2016)

A special thanks to this template.