MINDSEED

Abstract

Mindseed is an experiment that uses virtual reality and machine learning tools to facilitate the generation of new ideas through visualization, language, immersive experience, and play. I see the future of the project as a digital framework that uses virtual reality, a web platform, and natural language processing to create a world where you can plant, grow, and harvest your ideas.

You can watch a video of the project presentation at ITP’s thesis week below.

Thesis Presentation Video:

TECHNICAL DETAILS

Frontend: Javascript, ReactJS, ThreeJS, Heroku

Backend: Python, Tensorflow, Spacy, MongoDB, Ngrok, Heroku

VR: Unreal Engine 4

Hardware: PC, Oculus Quest 2, HTC Vive

Introduction

For my thesis, I built a prototype to assist myself with my creative process. We all know what a struggle it is to come up with ideas. They take time, patience, and inspiration. Imagine if we could make the process of coming up with an idea more entertaining and engaging. What if we could find a way to grow ideas as if we were growing plants or trees. Through these past 14 weeks, I’ve found a way to prototype this notion. Mindseed is a digital framework that uses virtual reality, a web platform, and natural language processing to create a world where you can plant, grow, and harvest your ideas.

Personal Statement

 So, I’ve been saving ideas for a while. I started using online platforms to record them in 2017, including google drive, keep, notes, and even emails, slowly building a personal archive of my thoughts. Since then, I’ve relied on this archive to develop new projects and recently, to think about my thesis project. I started my thesis exploration by frequently reviewing this archive and adding new ideas to it. 

After some time, I realized many of these concepts were connected, and some became much more interesting when I mixed parts and pieces of them together.I enjoyed this process of mixing concepts, and I was excited by some of the results. so I asked myself,  How I can translate this experiment with language into a digital experience? How can it benefit our creative process?

Research

Creativity is one of our most valuable skills, and our creative process is an essential part of innovation. This interest in innovation and problem-solving has led to the development of ideation methods and techniques like brainstorming and cheatstorming; It has fueled the success of collaboration tools such as Miro and inspired experimentation with online sites like Ideas.AI.Tools like these directly impact how we process information, engage with our ideas, and structure our creative process. 

Until recently our handheld screens and computer monitors have been the most utilized medium to interact with our data. As valuable as they can be, sometimes we feel detached from our digital information because it seems disembodied and shapeless. Today, we have new ways to visualize and experience our data. With the rise of technologies like mixed reality and machine learning, we can think of creating experiences that allow a more profound sense of embodiment, user personalization, and immersion.

I chose virtual reality to create an immersive experience with a more embodied sense of interaction. Inside this digital environment, we can experiment with any audiovisual representation to give life to our visual metaphors. On the other hand, using Natural language processing we have created tools that use semantic analysis and text generation to create meaningful communications that sometimes appear human. Mindseed is an experiment that uses virtual reality and machine learning tools to facilitate the generation of new ideas, using data visualization, language, immersive experience, and play.

Design / Production / Implementation

 The project has a simple premise: an idea is a seed. You plant your seed through a website by typing in a sentence that summarizes it. For example, “A website that helps people better understand their personal data. The website takes care of planting your seed and gives you back a set of coordinates indicating where you can find it later, once it has had the chance to grow. 

You then go into the VR world, where you enter the “Forest of Ideas.” Each of the ideas you planted earlier has become a full-grown fruit tree, with each fruit as a concept that makes up your idea. You can find the tree you planted with the coordinates you were given earlier, or you can walk around the forest and explore. 

 You go around the forest, picking around fruits from the ideas you like. When you feel you have enough, you mix them up and get a new seed to plant. This unique idea inherits some of the concepts you used to make it and will grow into an entire idea after some time.

You’ll receive a text message later on confirming when your new seedling has grown into a tree, as well as its coordinates and an id number for you to track it later.

Since this is all connected to a database, you’ll be able to mix different generations of ideas and track how they are related to each other. In further iterations, this will allow you to create a map of the roots that connect your thoughts.

Through my research, I found creative people prefer to generate ideas in solitary and later share these ideas with others. This is why I decided to incorporate multiplayer into the VR experience, allowing users to share limited interaction in the same space. Mindseed allows for individual reflection and also for cooperative play.

User Experience

There is a website where people can plant their ideas. It is currently functional, hosted online. The interface is simple; you type in your idea, choose a color for your seed, and then watch an animation of your seed being planted. You will also receive a set of coordinates. These are the coordinates of your tree inside the VR world.

The forest experience, where we can interact with our ideas, is made for VR. Additionally, it is also accessible to people who don’t have a VR headset and use a computer. It consists of two parts: A tutorial where someone can become familiarized with the controls, the environment, and how to interact with their ideas. The second part is the Forest, where you can explore alone or with others.

It took 14 weeks to develop Mindseed. The framework consists of three parts. The Frontend is the website where users can type in their ideas and see their seeds grow. The Backend takes care of processing the information, serving as a bridge between the website and the VR forest, making the experience of planting an idea seamless and synchronized. It also makes the simulated world seem continuously evolving, keeping track of each idea’s growth process.

User Experience

Onboarding

Mindseed has an onboarding experience that consists of a lobby where the user can learn how to move around, use the controls, and learn the dynamics of how to grow an idea. You can see a short video below.

Web Platform

You plant an idea.

Multiplayer VR

You plant a seed for a new idea from other ideas.

Text message notification

Your new idea is ready.

Multiplayer VR

Profit.

Some results from the more fine-tuned algorithm:

  • A project that uses random oscillating patterns to create frame-by-frame animations in 3D.
  • A custom built sound controller that uses wind energy to mine cryptocurrency.
  • A permanent installation for Berlin’s Futurism exhibition that explores prized possession in their native habitat.
  • An interacive installation that facilitates collabortation between a human and a puddle of alcohol.
  • A dancer’s body is extended and manipulated as a tool to quantify the world.

Process

Below you can see a video with a timeline of the development process, as well as some inspiration for the project.

Conclusions and Next Steps

The process took several iterations, user testing, breaking, and building.

And it was the process that truly had me excited. Figuring out how to use these tools to generate unexpected text was surprisingly fun. Playing with all these VR dynamics, introducing smoke, and light truly felt magical. And seeing the project evolve from something that looked like a rough idea I wrote down into something so personal and relatable, makes me want to develop it even further into the future.

  1. BE CREATIVE ALONE. BE CREATIVE TOGETHER – My research has led me to believe that coming up with ideas benefit from a solitary process. However, collective and constructive feedback can make a seed grow and evolve into true innovation.
  2. TIME AS A BRIDGE – While we strive for seamless real-time communication, we sometimes forget the value of time as an essential part of our processes. Time is not something we should eliminate, but rather something we should work with to give our metaphors more power.
  3. GO FOR THE LOW HANGING FRUIT – Our world is full of sensors and information. Make the most out of the information you have available to make the experience evolve with the person using it.
  4. LANGUAGE IS PLAY
    Our language determines how we think, how we communicate, and what we create. We should think of language as a playful artifact, a poetic machine, and the center of our creative ideation process.
  5. WE CHANGE. OUR IDEAS CHANGE.  We should be able to track these ideas and connect them with our past thoughts.  What better way to connect with ourselves than collaborating with our past to build our future!

For the next steps, I want to make the forest something you can mold into your own, a memory palace where you can freely explore your ideas and organize them spatially. I also believe each tree and fruit should look unique and recognizable, similar to the idea of a phenotype that expresses an inner trait. Appearance should have meaning. Through this process, several people have expressed their interest in using Mindseed. I want to further develop this into an experiment that can be experienced by more people.