Experiential Design - Task 1: Trending Experience

24.9.2024 - 15.10.2024 / Week 01 - Week 04
Tan Zhao Yi / 0363285
Experiential Design / Bachelor of Design (Hons) in Creative Media
Task 1: Trending Experience


INSTRUCTION



LECTURE

Lecture 1: AR Design Workshop

Introduction to XR
  • Mobile phone extends information, VR extends experience, and AR extends visual
Activity: Launch google.com on your phone browser search for dinosaur or dog, cat, tiger View in 3D View in AR (Life Size).

  • Designing for AR: Marker Based / Marker-less
  • Design Components: Design thinking (UX Design) + Human Computer Interaction (Usability + Interaction Design)
  • Design Thinking: Empathise > Define > Ideate > Prototype > Test
Lecture 2: Designing Experiences
  • User Mapping
  • Journey Map

XD
  • Involves designing products, services, processes, events, and environments.
  • Prioritizes quality of user experience and culturally relevant solutions.
  • Involves a cross-discipline approach, not tied to a single design discipline.
  • Considers multiple aspects such as product, packaging, retail environment, and employee presentation.
  • Aims to improve the overall experience of interacting with a product or service.
UX vs XD

  • Empathy Map
    • A tool used to articulate what we know about a particular type of user
    • Externalizes user knowledge in order to create a shared understanding, and aid in decision making

Activity: Journey Map
We form a group of 7 and create our own journey map of a random place. We chose theme park as our topic, then we download a template on Miro to work the journey map out.
Journey Map - Miro link: https://www.figma.com/board/eFuBskPZU65oVaP2rQxiGy/User-Journey-Map-Template-(Community)?node-id=0-1&t=4omyR4IF79QJ8dul-1

Feedback: Not clear enough, could list out the gain point and pain point or else viewers have to read everything again to figure that out.

Tutorial: Marker-based AR Experience
We are creating our first AR experience by using Unity. Mr.Razif has given a detailed guiding video for us in case we couldn't follow in class. In this tutorial, our goal is to install Vuforia package, setting up Vuforia license key and image target database.

We set up our Vuforia license and connect the key to Unity.
Vuforia License

Final Tutorial Outcome Video
Final Outcome Video - Tutorial 1
Lecture 3: User Controls and Button
In Week 3 class, we are creating 3 scenes that can navigate to each other through buttons. The three scenes are: Menu, AR Scene, and Credits.
Working on Unity

Final Tutorial Outcome Video
Final Outcome Video - Tutorial 2
Back to top
Lecture 4: Markerless AR Experience
Previously, we did a marker-base AR experience why we need an ImageTarget to trigger the AR visuals. In our case, we spawn a 3D cube when the image target is detected. We then change it to a video and we added controls to the video.  In this tutorial, we'll look at a marker-less AR experience where we only need to detect the ground/floor/any flat surface, and we will be able to spawn our 3D object.
Switching Platform Setting
I couldn't successfully switch the platform to my phone because of numerous versions and connecting issues, therefore I am using the ground simulator to showcase this tutorial.
Connecting Unsuccessful in Xcode

Final Tutorial Outcome Video
Final Outcome Video - Tutorial 3
Back to top

EXERCISE

Week 1
Imagine the scenario in either of the two places. what would the AR experience be and what extended visualization can be useful? What do you want the user to feel? (Kitchen / Shopping Mall / Gym)

#Shopping Mall
Problem Statement: 
Large shopping malls can be confusing, and finding specific stores, restrooms, or exits can be challenging. Also, shoppers may have trouble locating a specific item or comparing similar items across stores. Meanwhile, navigating and managing parking in large shopping mall car parks is often a time-consuming and stressful experience for visitors. Users frequently struggle to find available parking spots, especially during peak shopping hours, and they may have difficulty locating their vehicle when leaving.

AR can provide a real-time mall navigation system. Users could point their smartphone at the mall's floor, and directional arrows or floating virtual signs would guide them step-by-step to their desired location.
Fig 1.1. Shopping Mall AR Map

AR could display real-time promotions and deals as users walk past stores. Virtual sale signs could appear in the AR view, offering users a quick way to find discounts or offers at their favorite stores.
Fig 1.2. AR displaying real-time promotions

Besides, users could use the AR app to locate their parked car when returning from shopping. By pointing their phone in the car park, a virtual path would guide them directly to their vehicle. The app could also display a floating marker or icon above their car to make it easier to spot in a crowded lot.
Fig 1.3. Find you car using AR
In a nutshell, real-time updates on promotions, deals, and store information provide a sense of informed excitement, turning the shopping experience into a more engaging and enjoyable journey. Additionally, safety features and clear guidance through the car park, especially in crowded areas, help users feel secure and at ease. 

Media Reference:

#Gym
Problem Statement:
Many gym-goers struggle with performing exercises correctly, leading to potential injuries or ineffective workouts. For beginners, using gym machines or equipment can be intimidating and confusing without proper instructions.

AR could provide live feedback on exercise form by using the camera to track the user’s movements. It could overlay visual guides showing correct posture, alignment, and motion, for example a virtual figure demonstrating the proper squat form. When the user deviates, the app could highlight the error with virtual markers like a red dot and provide immediate instructions to adjust.
Fig 2.1. AR highlighting error with virtual marker
In the other hand, beginners could scan the gym equipment with their phone to receive a virtual guide on how to use it correctly. Overlaid visuals could highlight where to adjust the machine, the correct positioning for the user, and recommended settings based on their fitness goals.
Fig 2.2. Lat pull-down tutorial using AR
The desired user feeling focuses on confidence and safety, as the AR system provides real-time feedback and guidance during exercises. This ensures that users can perform movements with proper form, reducing the risk of injury and allowing them to exercise with confidence. By having immediate visual cues and corrections, users feel more secure in their actions, knowing they are supported in maintaining good posture and technique throughout their workout. This reassurance fosters a safer and more effective fitness experience.

Media Reference:


Task 1: Trending Experience

This task is given for us to explore the current and popular trend in the market to have a better understanding of the technologies and the knowledge in creating content with it. We will conduct research and experiment to decide which technologies we should proceed studying in our final project.

A. Research
As this topic is quite strange to me, I started with reading article about Augmented Reality (AR).

1. AR in Mobile Devices
  • Apps like Snapchat and Instagram use AR filters for entertainment.
  • Attracts users of all ages and is widely used in entertainment and eCommerce.
  • Businesses using mobile AR in apps or websites benefit from improved customer engagement and conversions.
2. AR in Automotive Industry
  • Improves driving safety through object detection and scene analysis,
  • Nissan's AR system, I2V, alerts drivers to hazards and helps maintain focus on the road.
  • Reduce accidents and enhance driving comfort.
  • BMW developed an AR app allowing customers to virtually experience new cars at home.
3. AR in Education
  • Provides an interactive solution that fosters imagination in children.
  • Makes learning more engaging and motivates students.
  • Helps students manipulate and view virtual 3D objects, enhancing their understanding of anatomy.
  • Teachers can use AR to reduce distractions and improve focus.
  • AR's accessibility may make it a cost-effective alternative to physical supplies in schools.
4. AR in Healthcare
  • Enhances health services and quality of life, especially during the pandemic by enabling telesurgery and remote patient support.
  • Used for patient and doctor education, and surgical simulations.
5. AR in Remote Assistance
  • MobiDev’s AR demo allows IT specialists to assist customers by overlaying instructions on their screens through smartphone cameras, enhancing remote customer support.
6. AR in Indoor Navigation
  • AR improves navigation in complex indoor spaces like malls and airports, addressing location-related challenges.
  • Provide accurate visitor locations and virtual paths for navigation.
7. AR in Retail and Shopping
  • Enhances customer experiences and reduces product returns.
  • Enables virtual try-ons, improving customer satisfaction, brand loyalty, and conversion rates.
8. AR in NFTs
  • The gaming industry is exploring AR and VR experiences with blockchain-based in-game economies and AR NFTs.
  • AR NFTs linked to physical objects and location-based AR are emerging trends, similar to the popularity of Pokémon Go.
9. AR and AI Integration
  • Advancements in generative AI enhance AR content creation and user experience.
  • OpenAI’s GPT-4o model integrates video and audio with text, improving human-AI interactions.
B. Ideation
After having a brief understanding of the trending AR experience nowadays, I came up with some ideas to study on.

Idea #1: Architectural Visualization in AR
Problem Statement: Traditional architectural designs often fail to provide clients and designers with a clear understanding of spatial layouts and the real-world scale of buildings and interiors. This lack of clarity can result in miscommunication, costly design changes, and dissatisfaction with the final product. Existing 2D plans and 3D models do not fully convey how designs will fit into actual environments, limiting clients' ability to visualize the final result. There is a need for technology that bridges this gap by providing a realistic, interactive preview of architectural designs in real-world spaces.

Architectural visualization in AR involves creating 3D models of buildings or interior spaces that users can view in real-world environments through mobile devices or AR glasses. It allows users to explore designs, get a sense of scale, and interact with virtual structures in real-time.

After consulting with my friend Ryan, who is studying Quantity Surveying (QS), I became acquainted with existing architectural AR software, specifically Building Information Modeling (BIM).
Fig 1.1. BIM
BIM is a digital tool that helps architects, engineers, and builders create and manage detailed 3D models of buildings and infrastructure. It includes information about the design, materials, and construction process, making it easier for teams to collaborate, visualize projects, and identify potential problems before construction begins. Essentially, BIM acts like a blueprint that everyone can see and work from, improving efficiency and reducing errors.

Idea #2: AR Storybook
Problem Statement: Traditional storybooks, while valuable, may not fully engage modern, tech-savvy children who are increasingly drawn to interactive and digital media. As a result, reading engagement has declined, and many young readers struggle to connect with static, printed content. Parents and educators are looking for ways to make reading more interactive and immersive, helping children develop stronger literacy skills while maintaining their interest. Current reading materials lack the multi-sensory stimulation needed to captivate children in a digital world.

AR storybook / comics bring a digital storybook to life. When the user points their phone at a page or image, an animated 2D scene appears, showing additional content or interactive elements related to the story. Or, the 2D characters could come to life with subtle animations, speech bubbles, or even branching storylines based on user interaction.
Fig 1.2. AR Storybook Example

Idea #3: Experience of Exploring Devices in AR
Problem Statement: The online shopping experience for electronic devices often lacks the tactile, in-store experience that allows customers to fully understand the size, appearance, and functionality of a product before purchasing. This can lead to uncertainty, buyer's remorse, and increased return rates. Customers have difficulty visualizing how devices will look in their personal space, and product descriptions and static images often fail to convey important details. A more interactive and immersive shopping solution is needed to bridge the gap between online and in-store experiences, giving customers confidence in their purchasing decisions.

In real-world cases, Apple has implemented AR technology in the Apple Store app, primarily for showcasing products in 3D, allowing customers to view items like iPhones, iPads, and MacBooks from different angles, and in some cases, visualizing how the products would look in a user's environment. 
Fig 1.3. Apple AR Technology
I am thinking to extend the AR features to allow users customise the product in real-time. For example, changing the colours to see how different finishes look in their environment. Also, users can visualise the accessories like cases, keyboard covers in AR.
Idea #4: AR to show Concave and Convex Lenses Principles
Problem Statement: Teaching complex scientific concepts like concave and convex lenses can be challenging with traditional methods, as students often struggle to grasp abstract theories presented through static diagrams and text. The absence of interactive and visual tools limits their understanding of light behavior when passing through different lenses, hindering the application of these principles in real-world contexts.

There is a clear need for an interactive, visual learning tool that enables students to manipulate lenses and observe their effects on light in real time. This approach aligns with the educational trend of AR, which simplifies complex scientific principles through engaging, interactive experiences. By using AR, students can better visualize experiments on how light interacts with various lenses, enhancing their learning engagement.

Back to top

C. Development
After conducting further research, I decided to abandon idea #1 due to my limited knowledge of architecture. Since I want to create a useful AR app, I recognize the need to acquire a foundational to advanced understanding of the particular field. I have since shifted my focus to researching relevant details for my other ideas.

Idea #1: AR Storybook
  • Combines traditional storytelling with immersive AR visuals that bring characters and scenes to life as the user reads.
  • As users point their device at the book, 3D animations, voiceovers, and sound effects appear, enhancing the narrative experience.
  • Allows users to explore different viewpoints within the story, such as seeing scenes from a character’s eyes or zooming into different parts of the environment.
Fig 2.1. Idea #1 - Visual Reference
Pros:
1. Combines text, visuals, and audio for better understanding and retention
2. Makes reading feel like a game, motivating kids to read more
3. Enhance user interaction
4. Portable and convenient
5. Sparks creativity

Cons:
1. Limited accessibility, as it requires a device and sometimes internet
2. Potential device compatibility problems

Idea #2: Explore Devices in AR
  • Users can explore electronic gadgets, like smartphones and laptops in AR before purchasing.
  • Allows customization options such as color, and accessories to visualize the product in a user’s environment.
  • Users can interact with the device in AR, simulating how the device functions (exp: turning it on, swiping through screens, testing features).
  • Users can compare different models side-by-side to understand physical dimensions, design, and key features.
  • Aids in decision-making by providing a realistic 3D preview of the product in the user's space.
Fig 2.2. Idea #2 - Visual Reference
Pros:
1. Realistic product experience
2. Allow customisation
3. Hands-on interaction
4. Space visualisation

Cons:
1. Can't fully replicate the tactile feel of holding and using the actual device

Idea #3: Interactive Learning of Light and Lenses
  • Explains the principles of concave and convex lenses through interactive AR experiences.
  • Users can see light rays passing through different lenses and observe how they are bent, refracted, and focused.
  • Users can adjust the distance of lenses and light sources to see real-time changes in image formation.
  • Offers quizzes or interactive challenges to test understanding of optical principles in a fun and engaging way.
Fig 2.3. Idea #3 - Visual Reference
Pros:
1. Visualise complex concepts, making students to have a better understanding
2. Useful for both classroom and at-home learning with minimal equipment
3. Engaging learning with interactive quizzes

Cons:
1. Interactive elements could shift focus away from core learning concepts

Final Task 1: Trending Experience

Fig 3.1. Final Task 1: Trending Experience - PDF

REFLECTION

Experience
The tutorials for using Unity were particularly satisfying for me, as AR was a completely new topic. I appreciated how the tutorials guided me step by step, helping me grasp the concepts and tools I needed. I successfully solved all the problems I encountered along the way, which boosted my confidence and understanding.

Observations
While exploring these AR applications, I noticed a few key things. First, interactive features, like 3D animations and real-time simulations, keep users interested and help them understand better. Customization options, like choosing colors or adjusting lenses, create a personal connection to the material. Additionally, the ability to compare items side-by-side encourages critical thinking. However, I also saw some challenges, like the need for reliable technology and the risk of technical glitches that can interrupt learning.

Findings
My exploration of AR has led to some important findings. First, AR makes it easier to connect theory with real-life examples, especially in subjects like optics. Second, engaging with interactive content helps users learn actively instead of just watching or reading. Third, the ability to customize AR experiences can cater to different learning styles. However, we must address technology barriers to ensure everyone can access these resources. 

Comments

Post a Comment