Peter Nguyen

Peter (Phat) Nguyen is a researcher at MIT CSAIL, advised by Prof. Daniela Rus. Peter completed his Bachelor's degree in Computer Science with a minor in Mathematics at the University of Massachusetts Amherst, where he was a researcher at the Computer Vision Lab, advised by Prof. Erik Learned-Miller.

My research interest lies at the intersection of computer vision, machine learning, and robotics. My goal is to bridge the gap between high-level cognition and low-level planning and control.

Google Scholar  /  CV  /  LinkedIn  /  Github  /  X

peter nguyen's photo
Research

My area of research interest is in computer vision, machine learning, and robotics.

ReGen: Generative Robot Simulation via Inverse Design
Phat Nguyen, Tsun-Hsuan Wang, Zhang-Wei Hong, Erfan Aasi, Andrew Silva, Guy Rosman, Sertac Karaman, Daniela Rus
ICLR 2025
project page / arXiv / bibtex

  @inproceedings{
    nguyen2025regen,
    title={ReGen: Generative Robot Simulation via Inverse Design},
    author={Phat Nguyen, Tsun-Hsuan Wang, Zhang-Wei Hong, Erfan Aasi, Andrew Silva, Guy Rosman, 
            Sertac Karaman, Daniela Rus},
    booktitle={The Thirteenth International Conference on Learning Representations},
    year={2025},
    url={https://openreview.net/forum?id=EbCUbPZjM1}
  }
Text-to-Drive: Diverse Driving Behavior Synthesis via Large Language Models
Phat Nguyen, Tsun-Hsuan Wang, Zhang-Wei Hong, Sertac Karaman, Daniela Rus
IROS 2024, (Oral Presentation)
project page / arXiv / bibtex

  @INPROCEEDINGS{10801754,
    author={Nguyen, Phat and Wang, Tsun-Hsuan and Hong, Zhang-Wei and Karaman, Sertac 
            and Rus, Daniela},
    booktitle={2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)}, 
    title={Text-to-Drive: Diverse Driving Behavior Synthesis via Large Language Models}, 
    year={2024},
    volume={},
    number={},
    pages={10495-10502},
    keywords={Training;Large language models;Natural languages;Cognition;Trajectory;Driver behavior;
      Intelligent robots;Autonomous vehicles},
    doi={10.1109/IROS58592.2024.10801754}
  }
Robust Frame-to-Frame Camera Rotation Estimation in Crowded Scenes
Fabien Delattre, David Dirnfeld, Phat Nguyen, Stephen Scarano, Pedro Miraldo, Michael Jeffrey Jones, Erik Learned-Miller
ICCV 2023
project page / arXiv / bibtex

  @article{Delattre2023RobustRotation,
    author    = {Fabien Delattre, David Dirnfeld, Phat Nguyen, Stephen Scarano, Michael J. Jones, 
                Pedro Miraldo, Erik Learned-Miller},
    title     = {Robust frame-to-frame camera rotation estimation in crowded scenes},
    journal   = {2023 IEEE/CVF International Conference on Computer Vision (ICCV)},
    year      = {2023}
  }
Teaching
Sept. 2023,

TA for CS326: Web Programming, UMass

May 2023,

TA for CS240: Reasoning Under Uncertainty, UMass

Jan. 2023,

TA for CS389: Machine Learning, UMass (Received distinguished TA award)

Activities and Leadership

Previously, I co-founded Stella Agritech to create technologies that enhance the efficiency of agriculture. I also led ISHCMC's sustainability program with Tanya Meftah, which was nominated by the European Chambers of Commerce for the Best Sustainable Business Initiative in Vietnam in 2019. I was also part of the UMass Rocket Team between 2021-2022.

UMass Aerospace and Rocket Team, NASA SLI 2022, Payload Software Engineer

Building the onboard flight inertial reference system (IRS) to continuously compute the launch vehicle's dead reckoning position in a GPS-denied environment with a 75 meter (250 ft) resolution


   


Some inspiration from Jon Barron.