Senior Researcher
INRIA Lille - Nord Europe, SequeL team (Sequential Learning)

---

Research interests:

• Compressed Learning, random projections

• Bandits, Experts, and online learing
  
  • Optimistic planning, tree search
  • bandits in metric spaces
  • bandits with infinitely many arms

• Reinforcement Learning (RL) and approximate dynamic programming (DP):
  
  • Analysis of RL and DP with Lp norms
  • Sample complexity bounds
  • RL and DP with function approximation

• Reinforcement Learning in continuous time
  • Numerical approximations of Hamilton-Jacobi-Bellman equations
    
  • Link to the theory of viscosity solutions
  • Variable resolution discretizations

• Policy gradient
  • Sensitivity analysis in continuous time
  • Sensitivity analysis in POMDPs via particle filters

• Variance reduction techniques for value function and policy gradient estimation

---

 Publications

---

 

Collective activities:

• PASCAL2 site INRIA Lille, since October 2009.
• ANR EXPLO-RA (EXPLOration - EXPLOitation for efficient Resource Allocation. Applications to optimization, control, learning, and games) 2009-2011
• ANR CO-ADAPT (Brain computer co-adaptation for better interfaces), 2010 - 2013.
• ARC MaBI, 2010 - 2011
  
• PASCAL 2 Pump Priming Programme Sparse Reinforcement Learning in High Dimensions, 2010 - 2011
  
• Associated Team with RLAI University of Alberta, 2009 - 2010
• ARC CODA: Contrôle Optimal d'un Digesteur Anaérobie, 2007 - 2008
• Associated researcher with CREA (Centre de Recherche en Epistémologie Appliquée), Ecole Polytechnique, from 2007.

 

Organization of scientific events

• ICML 2009 workshop On-line Learning with Limited Feedback (Sponsored by PASCAL 2)

• European Workshop on Reinforcement Learning, 2008. A post selection of 21 papers have been published by Springer in this LNCS Volume.

• Co-chair of ADPRL 2007 (IEEE Symposium on Approximate Dynamic Programming and Reinforcement Learning), celebrating the 50th anniversary of Richard Bellman's pioneering work on Dynamic Programming in 1957. April 1-5, 2007, Hawaii, USA.

• ICML/COLT 2006 Workshop Kernel Machines and Reinforcement Learning, June 29, 2006, Pittsburgh, USA.

 

Teaching (Master Maths Vision Apprentissage ENS Cachan)

---

PhD Students:

• Pierre-Arnaud Coquelin (actuellement président de Vekia)
• Robin Jaulmes
• Jean-François Hren
• Sébastien Bubeck
• Odalric-Ambrym Maillard (co-supervized with Philippe Berthet)
• Alexandra Carpentier

---

Contact:

Address:
Rémi Munos, SEQUEL project, INRIA Lille - Nord Europe,
40 avenue Halley, 59650 Villeneuve d'Ascq, FRANCE

Email: remi (dot) munos (at) inria (dot) fr
Tel: (0 or 33)3 59 57 79 06
Fax: (0 or 33)3 59 57 78 50