Enabling speech-to-text through Godot and REST

Context

My recent game Phonograph is a game where you play as a ghost hunter, communicating with ghosts you cannot see. The main mechanic is that the player speaks into an old phonograph using their actual microphone input. The words you say into your microphone while using the phonograph in game represents you speaking to the ghosts. This is an attempt to make a really interesting and immersive way to make suspense games. And it mimics one of the most fun mechanics in ghost games such as Phasmophobia.

Approach

The approach is very simple: Create a REST service that did the speech-to-text functionality outside of the game. And call that service from Godot. I did try to build Godot using the speech-to-text Godot plugin, but that didn't work well for me. And since I've had a good experience using Google's speech recognition Python API, I thought I would just host a Python REST service that utilized this Python SpeechRecognition library that includes google's library (and others) and Flask, hosted in pythonanywhere. 

Setup Speech-To-Text, Step-by-step in Godot

Setting up the REST service

1. Go to https://www.pythonanywhere.com/details/flask_hosting and sign up. I use pythonanywhere to host Flask apps for free, but you can use any hosting site that you're familiar with - so long as you know how to make REST calls to that particular hosting service. For pythonanywhere, I chose the "Create a beginner account" button to create a free account. Remember to confirm your account via email.

2. Once you've created your account, click "Web", "Add new web app", "Next", "Flask", "Python 3.9" (but you choose whatever version you're comfortable with, the SpeechRecognition API should work with all versions in 3.X).

3. Copy down your Path. It should look like /home/johndoe/mysite/flask_app.py. Then click "Next". You now have a web app set up. Let's put some code in it.

4. Click "Files" (at the top), "mysite", "flask_app.py". Delete all the code and paste the speech-to-text code I've written in there - I've included the code in a public gist here: https://gist.github.com/urodelagames/6ad8a429e73d114e64467988854e8eb0.  And press "Save".

5. Go back to your pythonanywhere.com dashboard (http://www.pythonanywhere.com/user/johndoe). Press "Reload johndoe.pythonanywhere.com.

6.  Copy the link that says "Configuration for johndoe.pythonanywhere.com". Navigate to that link and make sure you get "hello world" as a response. If you do, your REST service is setup!

Setting up the project in Godot

1. Clone the project here: https://github.com/urodelagames/speech-to-text-demo

2. Import the project and open it up in the Godot editor.

3. On the main node, "MicRecord", there's a variable in the inspector named "host". Enter your pythonanywhere host there. It should look like http://johndoe.pythonanywhere.com/ . Remember to replace johndoe with your user in pythonanywhere!

4. Run the project.

5. Press "Record" and start speaking in to your mic. Press "Stop" when you're done. The text result should show below the buttons!

Things to note

The pythonanywhere free tier only lasts for so long before it becomes a paid service. At the time of this devlog, the free tier lasts for 3 months before shutting down.

Additional Resources

https://medium.com/swlh/how-to-host-your-flask-app-on-pythonanywhere-for-free-df...

https://docs.godotengine.org/en/stable/tutorials/audio/recording_with_microphone...