Experiential Design - Final Project

18.12.2024 - 1.11.2025 / Week 13 - Week 15
Tan Zhao Yi / 0363285
Experiential Design / Bachelor of Design (Hons) in Creative Media
Final Project


LIST

Final Project


INSTRUCTION


Final Project

Finally, we will synthesise the knowledge gained in task 1, 2 and 3 for application in task 4. We will create integrate visual asset and refine the prototype into a complete working and functional product experience.

The remaining tasks are:
  • Potion spills out the pot effect
  • Animals animation
  • Frog asset
  • UI (Buttons, Screens etc.)
I plan to focus on completing the story section first. After finalizing the prototype, I intend to streamline the content by removing interactions from certain pages that are either unworkable or unnecessary. Currently, I have completed pages 4, 6, 18, and 19. However, I plan to remove interactions for pages 7 and 9 because I realized I couldn't edit the liquid in the pot as intended. Initially, I considered placing a liquid asset on the ground, but I concluded that it is unnecessary if the user cannot see the animation. Additionally, I will remove the interaction on page 10 and page 11, which involved animals feeling sick, as it adds little value. The animals merely lie on the ground and moan, making the interaction redundant.

For now, I will continue working on:
  • P2: The Potion Book
  • P5, 13, 14 & 15: Drag and Drop
  • P8: Animal Running
  • P12: Luna Interaction
  • P16: Brush the Teeth
A. The Potion Book
Fig 1.1. Page 2
My initial concept was to have a potion book appear when the user scanned the image target, creating an interactive and magical experience. However, after searching through the asset store, I couldn’t find a suitable model that matched the look of a classic potion book. To adapt, I decided to modify the idea by floating the potions above an open book, maintaining the magical theme while providing a visually engaging element for the user to interact with.

Assets
First I import all the images needed to Vuforia, then download and import them into my Unity file.
Fig 1.2. Vuforia Engine

After linking Page 2 to the designated image target, I carefully positioned the potion models on top of the book asset to create a visually appealing arrangement. To ensure the models appears when the it was scanned, I set the potion and book models as child objects of the image target.
Fig 1.3. Placing Object
Outcome
Fig 1.4. Outcome Video - The Potion Book
B. Drag and Drop
This section closely resembles Page 4 from the prototype submitted in Task 3. For this task, I will apply the same method used previously but with updated assets to achieve the desired visual effect.

1. Page 5
Fig 2.1. Page 5
As described in the book, the objects to be dragged and dropped include sticky mint and cranky toothache shells—items that don’t actually exist in the real world. To address this, I need to find suitable replacements that align with their intended characteristics and fit the overall theme.
Fig 2.2. Sticky Mint and Toothache Shell
As a result, I replaced the sticky mint with sugar cubes, and toothache shells with chocolate donut with whipped cream. 

Also, from the drag and drop script used in Task 3, I extend it to allow the draggables got destroyed after colliding with the pot.
Fig 2.3. Destroy script

Outcome
Fig 2.4. Page 5 Screen Recording

2. Page 13

Fig 3.1. Page 13
As the story says Luna brought the things that her pets need to try, the interaction of this section is allowing the user to drag the tooth floss milk, beak wash and the toothbrush and put them into the bag. I downloaded a trunk online (link: https://assetstore.unity.com/packages/3d/props/leathertrunk-295428) and added an open and close animation to it, therefore user can click to trigger the animation.
Fig 3.2. Trunk - Open and Close
Then I started to search for the tooth floss milk (milk box), beak wash (glass bottle) and toothbrush from Unity Asset Store.
Fig 3.3. Placing Items

For the script, I created a new one because the existing drag-and-drop script was designed to destroy game objects after colliding with the pot. Since I wanted to place the objects inside the trunk instead of destroying them, I introduced a plane object as a placement point to fix their position. However, this approach didn’t work initially due to the nature of planes in Unity.

Planes in Unity are infinitely thin and use a Mesh Collider by default, which only detects collisions on their surface. This limitation makes collision detection unreliable, especially if the draggable object doesn’t intersect precisely with the plane’s surface. On the other hand, cubes have volume and typically use a Box Collider, which accounts for the entire cube, making collision detection much more dependable.

To address this issue, I replaced the planes with cubes and positioned them at the desired locations within the trunk. This adjustment significantly improved the collision detection and ensured the draggable objects could be accurately placed inside the trunk.

Fig 3.4. Objects floating on the trunk if using Plane

Outcome
Fig 3.5. Page 13 Screen Recording

3. Page 14
Fig 4.1. Page 14
For this page, user need to drag the milk and drop it to the cat. As the downloaded cat asset has a sitting animation, I set the cat default status to "sitting".
Fig 4.2. Cat Animator
I edited the Drag and Drop script and assigned it to the milk box. Then, I added a "Cat" tag to the cat and a "Draggable" tag to the milk box. After configuring the necessary colliders and attaching a Rigidbody to both objects, they functioned seamlessly, replicating the behavior demonstrated on Page 4 of the prototype.
Fig 4.3. Edited Script Part

Outcome
Fig 4.4. Page 14 Screen Recording

4. Page 15
Fig 5.1. Page 15
This section closely mirrors the functionality of the previous one, with some adjustments tailored to the specific interaction. In the script, I introduced a new function that enables the draggable object to collide with the crow. Once the collision occurs, the script triggers the destruction of the draggable game object.
Fig 5.2. Edited Script Part
This implementation ensures a seamless and engaging interaction while maintaining consistency with the mechanics established earlier. By incorporating the collision-based destruction, the scene emphasizes the dynamic and responsive elements of the prototype, enhancing the user experience by making interactions feel more immersive and purposeful.

Outcome
Fig 4.3. Page 15 Screen Recording
C. Animal Running
This section is working on page 8, which mentioned the potion spilts out of the pot and the cat and the toad were frightened and run away.
Fig 5.1. Page 8
I Google searched for 3D toad assets since the Unity Asset Store didn’t offer a suitable option. During my search, I discovered two assets on SketchFab: a frog with a jumping animation and another static 3D frog model. Although the two assets are somewhat similar, I had to use them together as separate resources due to the lack of better alternatives.
First I import the cat and set its "walk" as default status, then increase its speed to make it looks like running. 
Fig 5.3. Cat Script (click to enlarge)
This script makes an object move in a circular path around a target. In the `Update()` method, the script first checks if the `imageTarget` (the point around which the object moves) is set; if not, it stops. It then increases the `angle` by a certain speed every frame, using `Time.deltaTime` to make the movement smooth regardless of the frame rate. If the angle exceeds 360 degrees, it resets the angle back to 0. The script calculates the object's new position using cosine and sine functions based on the `angle`, and it moves the object around the target at a fixed distance (radius). Finally, it makes the object rotate to always face the direction it's moving in by adjusting its rotation based on the angle.

The toad's movement is similar to the cat's, but I have configured it to follow the cat's position. In this setup, the toad's speed and radius are directly influenced by the cat's movement, as shown in Figure 5.4. By referencing the cat's speed and radius, the toad moves in sync with the cat, mimicking its path. 
Fig 5.4. FrogJump Setup
The FrogJump script makes the frog move in a circular path, following a cat's movement. In the Update() method, it first checks if both the imageTarget and catTransform (the cat's position) are set; if either is missing, the script stops. It then gets the cat's current movement angle from the CatWalk script. The frog's angle is calculated slightly behind the cat by subtracting an offset, ensuring the frog stays behind the cat on the circular path. If the angle goes below 0, it wraps around by adding 360 degrees. Using the angle, the frog's position is calculated using cosine and sine functions, keeping it at a fixed distance (radius) from the imageTarget. The frog is also rotated to face forward along the circular path. Lastly, the script checks if the frog's jump animation is playing and ensures it keeps playing the "jump" animation when needed.
Fig 5.5. FrogJump Script

Outcome
Fig 5.6. Page 8 Screen Recording
D. Luna Interaction
The interaction of this page is planned Luna will pops out and says "Oh dear, this did not turn out well. Why did I think it would be fun to cast a naughty spell?" And the sound will replay once user clicks on Luna.
Fig 6.1. Page 12
Since I did not have access to a 3D model of Luna, I opted for an alternative solution by cropping the character directly from the book illustrations. This approach allowed me to incorporate Luna into the project without requiring additional modeling resources or complex 3D asset creation.  

By using the cropped 2D character, I could retain the original artistic style and visual consistency of the storybook, ensuring Luna remained recognizable and appealing to the audience. Although this method lacks the depth and interactivity of a 3D model, it effectively captures the essence of Luna's character and integrates seamlessly into the project. This solution also saved time and effort, enabling me to focus on enhancing other interactive elements while still maintaining a cohesive visual experience.
Fig 6.2. Cropped Luna
I imported an AI-generated voice created using TikTok, as other online platforms required payment for generating a child's voice. To integrate this into the project, I attached an audio source component to the Luna game object. Additionally, I developed a script to handle the functionality, allowing the audio to play when the Luna object is clicked.
Fig 6.3. Audio Script
Additionally, I included a line of instructional text on the screen to guide users in performing the desired action. This ensures clarity and provides a seamless user experience, especially for those who may not intuitively understand the interaction. By offering clear and concise directions, the instructional text serves as a helpful prompt, enhancing usability and reducing potential confusion during the interaction.
Fig 6.4. Instructional Text

Outcome
Fig 6.5. Page 12 Screen Recording
E. Brush the Teeth
Initially, I planned for the interaction to allow users to brush the toad's teeth, with the decay on its teeth disappearing as they cleaned them. However, as I mentioned earlier, I discovered that I couldn't edit the asset myself. This limitation forced me to rethink the approach and find an alternative way to achieve the desired interaction.
Fig 7.1. Page 16
I considered the idea of displaying only the teeth and allowing users to interact with them by brushing away the decay. While this approach could focus attention on the activity, I realized it might not be suitable for children, particularly because the project is designed in 3D.

In a 3D environment, isolating just the teeth without the toad might look unusual or unsettling for a younger audience, as it removes the context of the character and could feel abstract or even eerie. For children, the presence of a friendly and engaging character like the toad is important for maintaining interest and creating a sense of connection. A floating set of teeth, even if interactive, might fail to capture their attention in the way a complete, animated character would.

Given these considerations, I decided to explore other ways to incorporate the brushing interaction while keeping the toad's full presence in the scene to maintain a child-friendly and visually engaging experience.
Fig 7.2. Working Page 16
As a result, I decided to utilize the drag-and-drop method once again for this section of the project. This approach is both versatile and intuitive, allowing users to engage with the interaction in a straightforward manner. By reintroducing this method, I can maintain consistency with other parts of the experience while ensuring that the activity remains accessible and enjoyable for children.

Outcome
Fig 7.3. Page 16 Screen Recording
F. UI
Finally, I started working on the UI part, which is:
  • Home
  • Characters
  • Rewards
The UI screens for the application will also be developed within Unity. To ensure consistency and optimal performance during development and testing, I have selected the iPhone 13 in portrait orientation as the target screen size. This decision stems from the fact that I am using an iPhone 13, making it easier to test and refine the app directly on my device. By aligning the screen size with my personal device, I can accurately assess the visual layout, functionality, and responsiveness, ensuring that the user experience is tailored to real-world conditions on this specific device.

1. Home
Fig 8.1. Home - Prototype
I first exported the background and title assets from the design software. Once exported, I imported these assets into Unity and placed them within a Canvas object. The Canvas serves as the primary container for UI elements in Unity, ensuring proper scaling and alignment across different screen resolutions.
Fig 8.2. Home screen in Unity
Next, I added a Home Manager script to the scene to manage the functionality of the Start and Rewards buttons. This script handles user interactions, ensuring that pressing either button triggers the corresponding actions effectively.
Fig 8.3. Home Manager

2. Characters
Fig 8.4. Characters - Prototype
This section features two scenes. The first scene remains visible for approximately 2 seconds before smoothly fading out to transition into the second scene, which introduces the characters. To maintain a consistent design across the app, I updated the "Nice to meet you!" button to match the style of the Rewards button from the home screen. This ensures a cohesive and unified visual experience for users.
Fig 8.5. Refined "Nice to meet you!" Button
Then I added a fade-in and fade-out effect using a fade panel. This effect is controlled by two coroutines: FadeIn and FadeOut. The FadeIn method starts by setting the panel's transparency  to fully opaque (1). It then gradually decreases the alpha value over time, making the panel fade away smoothly. The change happens incrementally each frame, using "Time.deltaTime" to ensure the fade duration is consistent regardless of frame rate. Once the alpha reaches 0.0, the fade-in effect is complete, and the panel becomes fully transparent.   
Fig 8.6. Character Scene Manager
The FadeOut method works in the opposite direction. It starts with the panel fully transparent (alpha = 0) and gradually increases the alpha value until it becomes fully opaque (alpha = 1), creating the fade-out effect. Both methods use "yield return null" within a "while" loop to pause and update the alpha value over multiple frames, ensuring a smooth transition. The fadeDuration variable defines the total time taken for each effect. This implementation allows seamless scene transitions with a polished visual effect.

3. Rewards
Fig 8.7. Rewards - Prototype
This section highlights the rewards users earn as they progress through the storybook using the app. As users engage with the interactive content and advance through the chapters, they unlock various achievements or collectibles that enhance their reading experience. The rewards not only serve as an incentive to encourage further exploration but also provide a sense of accomplishment and engagement. This feature creates a more dynamic and interactive storytelling experience, transforming the act of reading into an exciting journey where progress is tangibly recognized and celebrated.
Fig 8.8. Rewards section in Unity

UI Outcome
Fig 8.9. UI Outcome
G. Buttons
The buttons implemented in the entire app are:
  • Exit
  • Screenshot
  • Rewards
1. Exit Button
Fig 9.1. shows the script that is attached to the exit button. After users click on it, they will be directed to the Home Screen.
Fig 9.1. Exit Button Script
Fig 9.2. Button Settings

2. Screenshot
For this functionality, I implemented a debug log the replace the actual screenshot feature. This approach allows me to simulate and test the screenshot process during development without capturing actual images, ensuring smoother debugging and functionality validation.
Fig 9.3. Screenshot Button Script
3. Rewards
This functionality is linked to the rewards icon button on each page. When users click on the button, they are seamlessly directed to the Rewards section, providing them with an interactive way to view and track the rewards they have earned throughout their progress.
Fig 9.4. Rewards Button Script

The video below provides a brief walkthrough of the app developed in Unity, showcasing its key features and functionality.
Fig 9.5. Walkthrough Video in Unity

Then, I built the app and imported it into Xcode. This process was challenging because my previous attempt during an exercise didn’t work as expected. Fortunately, I discovered a helpful tutorial video online, which guided me through the steps and allowed me to successfully complete the process.
Tutorial Video Link: https://youtu.be/-Hr4-XNCf8Y?si=HLNDVU2g4TZOiSgB
Fig 9.5. Building in Xcode

Final Project Submission

Fig 10.1. Final Walkthrough Video using iPhone 13

Fig 10.2. Final Presentation Video
Fig 10.3. Final Project Slides


REFLECTIONS

This project tested my patience in ways I didn’t expect. There were so many moments when I got really angry and frustrated because solving one error would often lead to another unexpected problem. Sometimes, no matter how much I searched for solutions, I just couldn’t figure it out. It felt like a never-ending cycle, and it made me doubt myself at times. However, through these struggles, I realized that I needed to change my approach. Instead of trying to force a solution, I had to step back, rethink the problem, and try looking at it from a different perspective.

I also learned that sometimes, the simplest or even “stupidest” solutions work best. Overthinking often complicated things, while trial-and-error or straightforward methods helped me push through roadblocks. This shift in my mindset—being open to unconventional solutions and embracing the learning curve—was a turning point for me. It taught me that challenges are part of the process, and finding creative ways to solve them can be incredibly satisfying. Looking back, even though it was tough, I feel like this project made me more resilient and resourceful.

Comments

Post a Comment