Creating ‘Speech Sound Sleuth’, a Speech Articulation Game

For individuals working to improve their articulation during speech therapy, practicing speech sounds can sometimes seem a bit monotonous. Repeatedly attempting words that feature challenging sounds can be not only frustrating but also repetitive and uninteresting. To make speech therapy more engaging, I created an articulation game inspired by the hidden image activities Where’s Waldo, I Spy, and Highlights magazine’s “Hidden Pictures.” 

Before creating the game, I brainstormed the features I wanted to be included. Here was my vision:

• There should be a variety of scenes from which players can choose.

• Each scene should target specific speech sounds featuring hidden objects related to the sounds. For example, given the L sound, there should be items that contain L in all word positions (e.g., log, elephant, and basketball).

• The game should be timed to add excitement. 

• Whenever a player finds an object, the timer should pause, and a modal box should prompt the player to practice the word associated with the found object several times.

• After practicing, the timer should resume, ending only when all items are found.

• A leaderboard should highlight the top scores.

I took the opportunity while building this game to practice my skills with the MERN stack (MongoDB, Express.js, React, Node.js) and develop new knowledge (e.g., TypeScript).

Building the Frontend Functionality

I designed the interface with a focus on simplicity, consisting of just a few key pages:

• A homepage, where users can select a game scene

• A sound selection page that allows users to pick the target sound to find and practice

• The gameplay page, where the excitement happens

• Leaderboard pages for each game scene and sound category

Of course, the most intricate page was the gameplay page, which would contain the game scene. Upon clicking the game scene, a drop-down menu of the hidden item options would appear. The user would click the hidden item they believed they had found, and then the backend system would validate their choice for accuracy.

I started with the basics: adding the game scene. I used the React Zoom Pan Pinch library to allow the game scene image to be manipulated (e.g., zoomed in/out and panned or dragged). Then, I added the functionality for displaying a drop-down menu on click of the scene. The next part was the most challenging aspect: getting the coordinates of the click. Based on the way I was getting the coordinates, different screen sizes would produce different coordinates. Zooming in and panning using the library caused issues obtaining the coordinates as well. After much thought and research into how to normalize the coordinates across these conditions, my function finally worked, and I could consistently get the coordinates of the click. I left hooking up the backend to validate the coordinates for a little later. 

I then needed to implement a timer and display the target items to be found. I decided to put these features in the header, with the target items located within an accordion to allow them to be shown or hidden as needed.

For the frontend timer, I used a library called React Timer Hook. I configured the timer so that it would start when the image loads, considering the image would be pulled from a database. The timer was tricky to implement because I didn’t just want it to end when all the images were found. Instead, I aimed for the timer to pause each time an item was discovered, providing players with an opportunity to practice the word they had found, and then resume afterward. This timer was just for show. There would also be a timer on the server side to prevent users from hacking their scores.

After setting up the timer, I moved on to the practice modal, which would pop up whenever a player correctly guessed the location of an item. In the modal, the player would be prompted to practice saying the word five times, with an option to click stars to count the number of times the word has been practiced. 

Finally, I set up a form to allow users to input their nicknames to submit to the leaderboard at game completion. I separated the leaderboard by scene and sound. The leaderboard tables were modified components from Material UI. 

Developing the Backend

Within my database, I needed to store the scene, hidden items associated with each scene, and the scores for each scene. I created models, as illustrated below, to achieve this goal. 

Next, I set up routes and controllers. The routes forward requests made at specific endpoints to the appropriate controller functions. The routes for the game include endpoints for validating the user guess, getting game scene details, starting the timer, pausing the timer, resuming the timer, getting the final time, disconnecting the player, adding a score to a leaderboard, and getting the scores for a leaderboard. 

The controller functions handle interaction with the models to return the requested information. An area I struggled with was getting the backend timer to function properly. While the backend timer generally meets the needs of the game, it is still a feature that could be improved in the future. One reason is that the backend timer returns slightly different results than the frontend timer, which seems to be due to processing delays that cause them not to be synchronized. However, given the nature of the game, I was content with the current functioning while the number of users remains low. 

I tested the models and controllers using Supertest/Jest and Mongo Memory Sever, which freshly spins up a MongoDB server solely for testing during development.

Wrap Up

I learned a bunch while navigating my way through this project, especially with learning to use TypeScript and understanding the benefits it provides. While I have decided to move on to a new project for the purpose of continuing to make strides in my skills, there are some areas that I would like to return to in order to improve upon this project, including upgrading the timer functionality and fixing some minor display items such as preventing the target box and dropdown menu from appearing as the player drags/pans the game scene.