Logo of Huzzle


Research Intern - Speaking the World into Existence

Logo of Microsoft


2mo ago

🚀 Summer Internship


AI generated summary

  • You must be enrolled in a PhD program in Computer Science or related field with 2 years of research experience, including peer-reviewed publications. Experience in AI systems, Unity, and collaborating with product teams is required.
  • You will conduct research on applying generative AI to creative domains, focusing on mixed reality and gaming. Responsibilities include developing a research platform and creating scenarios for new technology.

Summer Internship

Software EngineeringRedmond


  • The Speaking the world into existence research effort aims to apply the recent progress in large language models (LLMs) to prompt-based creation of interactive three-dimensional (3D) scenes. Integrating LLMs with a game engine should enable not only faster development of 3D content in various domains, such as gaming, mixed reality applications or animated films, but also allow for spontaneous user-generated content in the course of an interactive experience. Our previous work has also demonstrated that giving a large Artificial Intelligence (AI) model the ability to act in a simulated environment with feedback has the potential to improve the outputs of generative models by grounding them in the real world. One of the fundamental research directions of this project is to make large multimodal models more reliable in the domain of human-scale activity, by not only incorporating what has been said about the world, but also testing results in a simulation of the world.
  • Beyond application in gaming and building compelling virtual worlds, easier creation of interactive 3D content and simulations by non-coding users also opens applications to education (e.g. allowing teachers to create immersive virtual reality (VR) lessons in a short time), rapid creation of interactive training scenarios (imagine a group of first responders spinning up a simulation in the 30 minutes it takes them to arrive at the location of a disaster so that they can be prepared for the potential difficulties at the location) and many other applications in various domains such as real-time generation of virtual environments for therapy or creative applications. 


  • Required Qualifications:
  • Currently enrolled in a PhD program in Computer Science or a related STEM field. Committed candidates enrolled in a master’s program might also be considered.
  • At least two years of research experience, including peer-reviewed publications, researching a topic closely related to the above description, such as AI systems for mixed reality, natural language processing, deep learning, generative models, mixed and virtual reality, multimodal interaction, etc.
  • At least one year of experience in building applications with the help of game engines, particularly Unity.
  • Other Requirements:
  • Interns are expected to be physically located in their manager’s Microsoft worksite location for the duration of their internship.
  • In addition to the qualifications above, you’ll need to submit a minimum of two reference letters for this position. After you submit your application, a request for letters may be sent to your list of references on your behalf. Note that reference letters cannot be requested until after you have submitted your application, and furthermore, that they might not be automatically requested for all candidates. You may wish to alert your letter writers in advance, so they will be ready to submit your letter.
  • Preferred Qualifications:
  • Demonstrated ability to develop original research agendas.
  • Able to collaborate effectively with other researchers and product development teams.
  • Experience in applying multimodal machine learning in a real interactive system.
  • Experience programming multimodal systems that interact with users, particularly by integrating multiple machine-learned components such as computer vision, speech recognition, dialogue handling, natural language generation, etc.
  • Efficient interpersonal skills, cross-group, and cross-culture collaboration.
  • Ability to think unconventionally to derive creative and innovative solutions.

Education requirements

Currently Studying

Area of Responsibilities

Software Engineering


  • Research Interns put inquiry and theory into practice. Alongside fellow doctoral candidates and some of the world’s best researchers, Research Interns learn, collaborate, and network for life. Research Interns not only advance their own careers, but they also contribute to exciting research and development strides. During the 12-week internship, Research Interns are paired with mentors and expected to collaborate with other Research Interns and researchers, present findings, and contribute to the vibrant life of the community. Research internships are available in all areas of research, and are offered year-round, though they typically begin in the summer.
  • For this Research Internship (Summer 2024), our group is seeking PhD Research Interns with a similar passion for research on applications of the newest generative AI models to creative domains, with particular interest in mixed reality and gaming, as well as improving large multimodal generations by grounding them in the real world. Intern responsibilities will include (1) helping to develop a research platform integrating large multimodal models within a rendering engine, (2) building compelling scenarios exploring the various potential uses of this new technology.


Work type

Full time

Work mode