Culture Vision – iMedia Capstone Project

For my Interactive Media Capstone project, I explored Interactive Media Applications in Foreign Language Acquisition.

As an Undergraduate Spanish minor, I’ve always wondered if new technology could change the ways that we learn languages.

Project Purpose

Research suggests that the advent of technology changes the way that we learn new things. For high school and college students, available foreign language course materials and resources have remained mainly the same for several years. This project explores how various forms of Interactive Media can is applied to foreign language education, and offers an example of a supplementary language-learning tool that uses Augmented Reality.

You can read my full literature review at finalmanuscript.

Understanding the User

Prior to the development of an interface, I needed to figure out what my application would do and who my audience would be. Only then would I understand how to maximize their benefit from my app. In my literature review, I focus heavily on Foreign Language Acquisition in the Classroom. I decided to focus my efforts on young teachers and college-aged students. Next, I spent my time taking stock of the audiences, potential considerations. I developed a User Persona to profile an example of a user that would benefit from my research.

Following the development of the User Persona, I conducted interviews with Jose Bravo de Rueda, a Foreign Language Professor and Kayla Fisher, a Foreign Language Student. I learned their respective ideas and expectations of a foreign language learning app.

I brainstormed ideas of potential tools or interfaces that would be beneficial to a user with similar to Kay. The main points that I gathered were:

The application would not have a lengthy delayed-gratification model for the benefits that the User would receive
A more useful application is designed to supplement omnipresent tools and applications
It would need to be mobile, as it would be useful in ‘on-the-fly’ circumstances
The application should be able to benefit the user in everyday situations (vocabulary being particularly useful)

Kay’s personal interests and everyday nuances provided me with a more focused perspective of a user’s everyday life. Using more intricate pieces of information obtained from the interviews that may be useful for my potential users.

The application would have a limited sign-up or sign-in interface
The limited sign-in interface would limit the amount of the user’s data that the application collects
Augmented or Virtual objects need to be used in some way
Simple/ Self Explanatory UI
User-Guided/ User-Led Interaction

From Concept to Design

Based on the User Persona and information from my literature review, I believed I now had enough information to begin sketching and storyboarding the application experience. In order to get a better picture of what I was beginning to create, I had to identify what tasks the user would want to complete. After taking inventory, I could thoroughly plan how to provide a seamless task completion experience.

I would also have to identify potential pitfalls and do some light user testing before I begin the higher-fidelity designs and application development. Bearing in mind that the application would feature vocabulary, I thought of an AR vocabulary application.

With Google Cardboard in mind, I created a concise application map to show how the user will get into the interface.

Next, I created a low-fidelity wireframe of the landing page. Here, the user will be able to view and switch native and target languages and toggle into the interface.

Finally, I ‘storyboarded’ the application by using a sample image to outline the application’s function. Note that the application recognizes the object and provides the native word banana and the Spanish translation.

After researching development tools for Image and Object recognition, I decided to use Cordova and Wikitude in order to develop the application. For exhibition purposes, my application would serve as a prototype for a Natively developed iOS and Android application. The actual application would use machine learning and a Google Translate or Microsoft Translate API in order to display translated words over an object.

After gaining an understanding of the constraints that Cordova and Wikitude provided, I began to design the application.

During the development process, I realized that maintaining the contrast between the overlaying text and its image target could be difficult depending on the setting.

In order to combat this, the overlay will include a rectangular black background with the world appearing over it in white text with tall and thinner characters to improve readability.

Furthermore, I decided the best image overlay would display the word in the target language while also providing the user with audio of the word being pronounced.

After ensuring that the application was viable, I began to style the application and accompanying media. I wanted the interface and the accompanying informative website to have an easily digestible white background with matte variants of the reds and blues present in the flags of many Spanish-speaking countries.

The decision to tone down the colors was made following a critique from prospective users that vibrant colors may take the user’s attention away from the content.

In the spirit of the projects academic roots, I refrained from using fonts that could potentially alter the tone of the content opting for the ever-so-reliable and mobile-friendly Helvetica with Avenir. Avenir possesses minimally varying font weights that are suitable for heavy passage reading.

The Next Steps with Augmented Reality

Where Do I Want To Go With Augmented Reality?

I’m not entirely sure whether or not I want to get into developing Augmented Reality, but I sure do think it’s cool! I’ve been muddling with the idea of taking up Vuforia as another Interactive Media hobby. Being limited to 3D models for AR development seems very restricting.

My Augmented Reality Fears

However, I’m pretty confident that AR development tools like Unity will find a way to make the means that we are currently using obsolete in a few years. Another fear that I have is that a company like Adobe will find a new more accessible way to develop AR that will, unfortunately, saturate the industry. My fear of the AR industry saturation could be a silly one, but we’ve seen film evolve from a niche activity. A sector that previously sported a high learning curve has transformed into a brand new medium. The majority of the people use this new medium to connect with the outside world.

What I’m Optimistic About

The idea of being able to take an environment and slap it on top of real-life sounds like something out of science fiction. Unfortunately for the skeptics, that technology is here, and it is taking off. My hope for AR is that it’s potential will be explored beyond general monetization of the experience. The reason why I became interested in AR was that I saw that it could change experiences. Transforming the educational, purchasing and operation experience can potentially affect everyone and the way that the world operates. I see AR as a present-day social network in terms of potential. This technology can change the world, and all we need to do is figure out how.

Is Augmented Reality Entertainment Still Reality?

Congratulations!

You’ve made it past my SEO nightmare of a title! I wanted to introduce this topic because I feel as though many overlook Augmented Reality applications in entertainment outside of the gaming community. One of my classmates and I discussed the possibility of experiencing a movie in AR. Unfortunately, we were unable to see the idea to fruition. While that thought is likely to stay in limbo, for the time being, it’s got me thinking.

What Are We Trying to Improve With Augmented Reality?

After revisiting that idea in my head, I realized it’s about how more than what. Trey and I were trying to improve how we can experience entertainment. Most advancements in entertainment that thrive make changes to how we are consuming the content. Netflix, Hulu, and Amazon Prime Instant Video made their big breaks because they changed the way that we consumed film and television videos. The entertainment that we’re receiving has not changed at all! Augmented reality makes it possible for us to experience the SAME things in new ways! The reason why movie theaters have not died yet is that it provides an experience beyond the content.

What’s the Correlation Between Theaters and Augmented Reality?

Movie theater franchises are introducing larger, reclining seats, more snacks (that are still expensive) and better screens to improve our movie watching experience. What if we applied those same principles to role-playing games using AR? For instance, when I was younger, I used to log onto Habbo religiously. I enjoyed meeting new people in a semi-detached reality platform and seeing their creations. Runescape, World of Warcraft, and Second Life also provided a very similar experience, but what if we didn’t have to be completely immersed in a separate environment? We can experience the fun in real life!

Augmented Reality Applications in Business

Companies are currently introducing Augmented Reality as an external contribution to the market. Companies that deal with AR have mostly been dealing with developing an environment or device for AR consumption and releasing their product into the market. Recently I’ve been exploring ways that AR can increase efficiency and push the bottom line from the inside out.

Can Augmented Reality be beneficial to businesses if applied internally?

Thus far, we’ve discussed ways that AR applications in education, medicine, and gamification, but there are many more applications that aren’t quite that abstract. For instance, an interior designer can potentially use Augmented Reality to overlay their designs on their client’s space. We’ve all spent a fair amount of time watching Love It or List It! and Property Brothers getting blown away at their 3D mockups that they show their clients, but what if these clients could see it to scale?

Let’s Dig A Little Deeper

Recently I’ve been learning about Design Sprints, Agile, and Scrum techniques. When we simulated a design sprint in class, I noticed that we ended up going through a lot of sticky notes, sketch paper, and markers. What if a team of employees used an AR overlaid Scrum board for their process. Furthermore, a digital copy of the Scrum board could make it easier for all parties involved to be able to reference the scrum board at will.

While Scrum and Agile techniques currently increase efficiency and productivity in groups, they create a lot of necessary waste. Wouldn’t it be cool if we could use AR to make an efficient process more efficient? I think so…

Augmented Reality’s place in education

Augmented Reality brings the user into the computer-generated world without taking them out of their own.

Not Virtual Reality?

I prefer AR to VR because it allows for users to experience it on devices that most of us already own. The potential for ARs adoption on smartphones and tablets simply seems more likely than VR’s potential smartphone consumption. AR can be used as an educational tool by bringing together theoretical and practical learning environments together to form a uniquely immersive experience.

How can Augmented Reality be used to teach?

AR can provide a practical learning experience by simulating the practical environment where students can simultaneously reference or use their theoretical material.I once imagined an AR application where botany students can aim their device’s camera at a plant and physiological and ecological information about the plant is displayed in real time. Rather than just learning about the plant in the classroom, the students can venture outside of the classroom and learn about the plant while seeing the real thing. As a result, students are more engaged because they can see, smell and touch the plant while acquiring the theoretical knowledge.

So What’s Your Niche?

The theory surrounding Augmented Reality can be applied to foreign-language learning because of its immersive nature. Something I’ve pondered is an app where somebody their device’s camera at an object in real life and see the name of the item in the foreign language displayed on the screen. One feature that would take the application a step further is an audio function where the name of the object is played by the application to help the student practice the proper pronunciation. I’ve dreamed of an environment where the names of objects are displayed in a foreign language for students.

I’m still working out the kinks in my idea, but I’m open to suggesting it.

Mapping the Brain with Augmented Reality

Uhh.. is this possible

Yes, mapping the brain with the use of Augmented Reality is certainly possible. There are still some kinks to be worked out in the technology. And as we all know, all emerging technologies require a boatload of money to gain any worthwhile exposure. I recently discovered this.. umm concept or thing during a conversation with my professor Dr. Moner.

(SN: you should really check out the link on his name, as I navigated to his twitter page to find the link, I found myself captured by a tweet thread where Tweeters continuously made dry puns about Rick and Morty)

He told me that some of his colleagues in the psychology department were using Virtual Reality to map the brain. When he explained this to me I thought: “No way.. that’s not a thing, this isn’t the Jetsons” , but sure enough a Bing search revealed that Virtual and Augmented Reality Brain Mapping is certainly a thing!

Augmented Reality Applications in the Brain get better!

What was even more exciting about my findings were that not only were people using AR to map the brain, but they were doing so with educational intentions! Here, some folks used an Augmented Reality brain to help people remember the different parts of the brain. They also have games to analyze the brain’s responses to auditory stimuli and even one where you catch a ball against an artificial brain that might be smarter than you.

On a more serious note, the ability for brain professionals to map the brain outside of the skull has incredible potential. The amount of medical and educational advancements that AR brain mapping can create are exponential.

As much as I would LOVE to go to Med School, residency, and fellowship. My dreams of starting my own augmented reality brain poking and mapping practice will have to remain just that. I’ll leave the innovation to the professionals and support their big ideas every step of the way!

Augmented Reality and Gamification

One of the more popular applications of Augmented Reality is gaming.

The Game Changer (Pun Intended)

Pokémon Go is one of the most popular Augmented Reality games to have hit the general public’s grasp. This game was a heavy-hitter because it was based on a name and franchise that many millennials know and trust. The product was also introduced on as an application on a device that just about everyone owns.

Why Augmented Reality Gaming Matters

The genius behind the development of the game has practically forced every emerging technology expert to take notice of Augmented Reality gaming as a serious industry. Previously, most people saw Virtual Reality as the sole revolutionary gaming platform, but the success of Pokémon Go and other games like Zombie Go and Father.io have taken the medium to new heights. Having a game that you can take into the real world is already amazing enough, but the games are influencing those of us who may like games more than exercise to get up and get active.

Does the “Go” in Pokémon Go mean anything?

This surge in activity can be attributed to Pokémon Go’s quests and methods of in-game advancement. For example, in order to earn experience for some of your Pokémon, you must walk a certain distance. This led to a lot of people aimlessly walking around their neighborhoods in an effort to advance in the game while unknowingly burning a few calories in the process. Now, some may ask: “Does it really count if they’re getting active because of a game and not just for the sake of wanting to live a healthy lifestyle?” To that, I say, WHO CARES?! People are getting up, getting out and exploring the world around them, or the 3D modeled overlaid world through their smartphones. And for all you life hackers out there, Yes, the app CAN tell if you are driving your car.

There will be no cutting corners.

Final Thoughts

All that I have to ask is “What’s next?”

Can Augmented Reality help save lives?

Augmented Reality and Medical Imagery?

My fascination with Augmented Reality began when I discovered the differenced between Virtual Reality (VR) and AR. I loved the idea of an immersive environment that does not take you out of reality completely. Overlaying real-world images with other information can help people better understand what they have in front of them.

Where does medicine come into play?

A secret fun fact about me is that I have spent the better part of 4-6 months binge-watching Shonda Rhimes’s 14-season medical drama Grey’s Anatomy. Grey’s is a medical drama surrounding a group of surgical interns fresh out of med school and chronicles their adventures as Interns, residents and ultimately attendings. My favorite part of the show is when we see the surgeon’s operating (with all cinematic medical inconsistencies intact). While I haven’t spent much time in a hospital setting, I’ve come to learn that medical imagery and its relative medical specialty Radiology is a huge deal. These professionals already utilize tons of imaging technology already. Most people are familiar with Ultrasounds, MRIs, PET scans and things of the like. After I learned the general ins and outs of ARs, I wondered if it could be applied in the medical field.

Augmented Reality in the Body Cavity?

The most abstract application that I’ve thought of is mostly related to surgery. Surgeons spend the better part of their 20s and early 30s learning about the human body and how they can manually correct problems that may arise from within. General Surgeons almost always know exactly what they’re looking at when they open a person’s abdomen, but what if AR could be there to remind the surgeon what they’re looking at anyway? I’m imagining a device that recognizes an open body cavity and can overlay the names of the internal body parts and even project relevant images over them depending on the scope of the surgery.

I don’t know if I’m reaching for the stars with something that is not relevant or necessary, but I plan to do some light research in AR as it applies to medical imaging and even if there is such a thing.

Regardless, it’s still an exciting thought, and I’ll keep up with this topic in the future. Once I find some proven applications of AR or even VR in the medical field, I’ll create another post that dives a bit deeper into it.

If any of you readers have any insight, I’m always open to learning!

Augmented Reality as a new Frontier

Augmented Reality (AR) essentially bridges the gap between reality and the virtual world.

What is it?

Definitions found online describe AR as a live direct or indirect view of a physical, real-world environment whose elements are overlayed or augmented by computer-generated or extracted real-world sensory input using sound and audio, video, graphics, and GPS. Similarly to Virtual Reality (VR), AR involves an artificial environment. The main difference between the two is that Virtual Reality completely immerses the user in the new environment whereas Augmented Reality allows users to experience both the real world and the virtual world simultaneously.

The opportunities in this field are plentiful as AR is still an emerging technology.

Umm…. What?

Most people are familiar with AR through the widely used game application Pokémon Go, the game allows players to catch fictitious digital monsters or Pokémon in the real world. The game received a lot of buzz and was credited with encouraging many young people to get active.

How Can I Create It?

There are many ways that users can develop Augmented Reality, but it requires a lot of knowledge and dedication because… like I said before, it’s still an emerging technology. Many of the bigger names in AR like Pokémon Go are practically building it as they go. One of the ways that we can develop AR is through an application called Vuforia. Vuforia provides game developers who use Unity the opportunity to take their 3D models and objects and create an AR environment by utilizing features such as image tracking, device tracking and object tracking to make your AR app usable on iOS, Android and A selection of Handheld devices.