Sean Hardesty Lewis
I am a predoctoral researcher at MIT studying how intelligent agents learn transferable representations of complex environments and the people acting within them. My work often uses large-scale physical and virtual worlds to study how representations of environments and behavior generalize across settings.
Previously, I completed my M.S. in Information Science at Cornell Tech, where I was advised by Wendy Ju and Anthony Townsend. There, I created Searchable.City, a semantic representation built from over 50 million street-view images that enables natural-language search across the physical world.
I also created Human Operator, a vision-language system that translates observations into real-time motor actions executed through a human body, which reached over 30 million views online.
Before Cornell, I received my B.S. in Mathematics from UT Austin, where I worked with Junfeng Jiao on AI systems for city-scale sensing and simulation.

email:
end@mit.edu
selected work in progress
selected publications
Searchable.City: An Open-Vocabulary Semantic Atlas
Sean Hardesty Lewis
Minecraft to 3D: A Pipeline for High‑Fidelity Reconstruction of Minecraft Worlds
Sean Hardesty Lewis
A Decentralized Digital Twin via Crowdsourced Sensing and Browser-Based Edge Computation
Sean Hardesty Lewis, Junfeng Jiao, Yiming Xu, Jihyung Park, Connor Phillips
news
- Sep 2026
Transferable visual representations was accepted to the SIGSPATIAL 2026 UMFM Workshop. See you in Riverside! 🇺🇸
- July 2026
Presented Searchable.City at SIGGRAPH 2026 in Los Angeles and received the DCAJ award. See you in Tokyo! 🇯🇵
- May 2026
Demonstrated Daunt, a real-time runtime for embodied AI agents in virtual worlds, to 600+ people at Cornell Tech.
- Mar 2026
Won the MIT Media Lab HARD MODE hackathon with Human Operator, using VLMs and EMS to control human bodies.
- Oct 2025
Happy that StreamTwin was accepted to the Deployable AI Workshop at AAAI 2026. See you in Singapore! 🇸🇬
- Sep 2025
Thrilled to share that OpenCityCorpus was accepted to the NeurIPS AI4Science workshop.
- Sep 2025
Proud of my identical twin brother Ryan on starting his PhD at MIT!
- Aug 2025Presented Minecraft to 3D at SIGGRAPH 2025 in Vancouver. 🇨🇦
- Aug 2024Started my M.S. at Cornell Tech in New York City.
- May 2024Graduated from UT Austin with a B.S. in Mathematics.
- Feb 2024Our work on emergency AI, SafeMate, was funded by the City of Austin.