Xiaoyun Margaret Zhang

• LinkedIn
• Email
• GitHub

• [email protected]
• (574) 310-9583

PhD Candidate in Architectural Design and Computation, Massachusetts Institute of Technology
ND BArch'17, MADU'19 ; MIT SMArchS'21

Research: Probabilistic Machine Learning of Creative Visual Thinking

Envisage I & II examine measurable physiological attributes of eye-movement such as vector speed and foveal vision areas and their emergent relationships to embedded architectural intentions.PhD Thesis: ROSAS, or Reciprocal Observational Stochastic Action-Space model, translates temporal-spatio eye-movement patterns into architects' perception of design intentions. This model-based approach leads to a Neural Network where machines learn to see like architects in understanding design representations.

• More

About me

I am a PhD candidate in Computation at the Department of Architecture, MIT. I am from Beijing, China, and I hold a BArch and a MADU from the University of Notre Dame, and a SMArchS Computation from MIT. I received MIT Presidential Fellow of year 2022 - 2023. I dedicate in studying compositional system and procedure of architects' vision in Design Intelligence and incorporating visual thinking into deep learning and computational design processes. Carrying on the hand-drawn tradition in architectural design of the Renaissance and École des Beaux-Arts, I am now exploring architects eye movement in the context of perception and creativity, for both theoretical and applicational approaches in probablisitic machine learning.I enjoy drawing fantasy scenes and characters, making costumes, planting roses, drinking tea, designing kimonos, listening to folk music from around the world, and developing happy and sympathetic collaborations between human and machines.

Publications & Designs

• More

Experience

• eCAADe 2025 Proceedings

• Home

Observe Like an Architect:
A neural network model that recognizes embedded design intentions in plan sketches through eye-tracking

Abstract Architects’ visual behavior during the interpretation of architectural representations reflects the “eye-mind link” central to design thinking. This paper proposes a novel dual-branch neural network framework that captures and utilizes dynamic eye-tracking data to embed and recognize design intentions from architectural plan sketches. We introduce a learnable “intention vector field” layer within a convolutional neural network (CNN), trained on eye-tracking data that records goal-directed gaze patterns corresponding to three architectural intentions: shape, composition, and circulation. The data collection pipeline enables eye movements mapping of multiple design intentions to one plan sketch. We evaluate the model’s performance with and without the vector field layer to assess its contribution. Results demonstrate that the learnable layer significantly improves classification accuracy and enables the network to internalize intention-specific gaze dynamics—mirroring the “eye-mind link” through interpretable learned features.Key Words Architectural Design Intention, Neural Networks, Eye-tracking, Visual Data Representation, Eye-mind Link.

Gallery

• SiGraDi 2024 Proceedings

• Home

Eye-tracking architects: investigating design intentions in plan drawings

Abstract Architects have a long tradition of communicating design ideas visually by embedding them in representations such as sketches and drawings. With the advances in physiological recording devices such as eye trackers, architects have obtained another means of exploring how their vision works when performing design tasks, and discovering how their eye movements reflect creative design thinking. In this paper we investigated architects’ eye movements during design intention retrieval by examining existing studies of eye-tracking in architecture and proposing two new types which were observed in early scientific explorations and architectural writings. Finally, we devised a tool to visualize eye movements when architects observe design sketches by Palladio and Le Corbusier, and analyzed gaze directions and foveal vision clusters from our custom calibrated eye-tracking tool. Our evidence of two types of eye movements suggested that distinctive eye movement patterns were temporally and spatially meaningful to their corresponding design intentions.Key Words Design cognition, Eye-tracking, Architectural drawings and sketches, Design intentions, Visual search

Gallery

• Collage gallery

• Home

Institute of Dunhuang Arts, Beijing, China:
Architectural heritage and its roles in a modern metropolis

Anchoring our celebrative daily lives
In a global world, as shared building traditions becoming more accessible, architects should engage multiple architectural heritages in a design process that produces a sense of place through the use of modern materials, contemporary construction procedures, clarity of structure, local and reginal vernaculars and monumental forms. Architects should consider both traditional and future-oriented adaptations to successfully aim for a healthier and more engaging urban condition that sustains and evolves.Architecture has been given the tasks of creating and sustaining great spaces for people to visit, study, experience, reflect, live, and rest. People, spaces, lights, colors, feelings and times alike, we perceive the world around us by attending to details of life that relates to our very daily activities. Buildings are manifested for the very common lives of all: streets that connect apartments to work places, family-owned shops that sell food for late-night students, and local supermarket that fills with treats and sales...The hearts of out everyday like reside in the coziest spaces that architecture can provide and cherish.The forms and spaces take the very hearts of daily life and celebrates its warmth and tranquility. This thesis focuses on integrating those life moments with traditional motifs and senses of belonging to a culture of humility, diversity and an embracement of the histories through tectonic details and composition of spaces, hierarchy and spatial narratives.

Gallery

Modeling Visual Creativity: Teach machines to see like an architect
Vitruvius argued in his “De Architectura” that education of a good architect must include a
variety of scientific and social studies that range from geography to astronomy. Contemporary
architecture applications benefit from tools developed for simulation, rendering a previous
understanding of the other disciplines into an architect-friendly interface that cohere to major
design pipelines such as programming, modeling, rendering, and post-occupancy analysis.
Design of buildings often associates with a real-world embodiment that expresses ideology and
expectations. On the other hand, architectural design is also a way of interpreting human thinking
as they project their relationship with civil and natural realms into the future, which is both our
second nature and one essential part of type-III reasoning that makes us “human”. Especially
when a new digital revolution is rising from the horizon, architects shall contribute their
understanding of the entire human creative endeavor as it is constantly reasoned using models
and equipment digital and analogical.

Current Focuses:

To explore what visual behavior means at this starting point of visual creativity as an "invaluable
design tool", our study explores a possibility of integrating measurable visual
behavior into “intentions”. Intentions such as shape, composition, circulation, depth, positive and
negative, and hierarchy are abstract concepts that architecture students learn and apply when
studying and practicing designs. To understand how these “intentions” are
obtained by observing sketches, one opens up a portal into understanding establishing links
between what is seen and what is thought. What’s more, in the current time where Artificial
Intelligence has been integrated into many human creative endeavors, it is crucial to examine this
visual reasoning processes’ relationship with neural networks which assist creative thinking.

Publications:
• Zhang, X., Nagakura, T. (2025). Observe Like an Architect: A neural network model that recognizes embedded design intentions in plan sketches through eye-tracking. 43rd Education and research in Computer Aided Architectural Design in Europe Conference Proceedings.
• Zhang, X., Nagakura, T. (2024). Eye-tracking Architects: Investigating design intentions in plan drawings. XXVIII International Conference of the Ibero-American Society of Digital Graphics Proceedings.
• Brown, C.J., Yao, S., Zhang, X., Brown, C.J. et al. (2023). Visualizing digital architectural data for heritage education. IS&T Conference on Visualization and Data Analysis.
• Dressen, A., Camerlenghi, N., Krusche, K.U., Zhang, X. (2022). Digital Humanities Workshop I: Learning about 3D models and virtual realities through two projects. The Renaissance Society of America Virtual.
• Zhang, X. (2021). Envisage: Investigating design intentions and visual perception through eye-tracking of architectural sketches. Massachusetts Institute of Technology. SMArchS Thesis.

Exhibitions:
• Nagakura, T., Tu, H., Peng, W., Cheng, C., Zhang, J., Zhang, X., Sung, W., Ishikawa, K. et al (2024). Singapore GeAR. SIGGRAPH 2024.
• Zhang, X. Nagakura, T. (2021). Metropolitan Museum of Art, Art of Japan. Medieval Japanese Buddhist Statues of the Temple Room. Interactive digital exhibition of 4k statues.
• Nagakura, T., Zhang, X., Sung, W. (2020). Full-scale AR Reconstruction of Alvar Aalto’s Resident, Aino and Alvar Aalto Shared Visions: Small is beautiful – ideal homes for everyone. Gallery A-Quad. Tokyo, Japan.

You have arrived at the fun section! Here I included more than academic showcases...such as...?

Academic:Fall 2025:
Smoothing Eye-tracking data streaming in Unity using custom real-time filters.
Revising eye-tracking experiment pipelines to embed streaming data into Dynamic Scene Graph representation.Upcoming in 2026:
Exploration and documentation of LangZhong, the home of Spring Festival.

Activities:We like ShinyStuff:
Animating costume weapon props using Arduino, robotic components, and programmable LED strips.CosWear!:
Making of FFXIV Online Edenmorn Coat of Maiming using traditional tailoring material and 3D-printed fashion accessories.