Bugs Found in Hume: Your Personal AI for iOS
Hume AI is an innovative organization that focuses on the development of emotion-aware AI, particularly specializing in voice-based interactions. The company strives to redefine traditional AI systems, focusing not only on speech context but also on how it is delivered, taking into account tone and emotional signals.
Hume presents the users with EVI, an emotionally intelligent voice AI with robust conventional capabilities. It is intended to foster greater alignment with human emotions and responses, marking a distinctive shift from traditional AI models.
Our QA engineers conducted a bug crawl of Hume’s app and identified bugs that could affect the performance and user experience of this novel platform. We present these findings to our readers and invite them to consider how these issues could be resolved to improve interaction and create a more natural AI conversation flow.
Error message is displayed when attempting to sign up using “Continue with Email” feature
Severity:
Critical
Precondition:
The app is installed.
Steps to Reproduce:
- Launch the app.
- Select the “Continue with Email” option.
- Enter a valid email address.
- Enter a valid password (the password must contain at least one uppercase letter, one lowercase letter, one number, and one special symbol).
- Tap the “Continue” button.
Environment:
iPhone 16 (18.2)
iPhone 11 (26.2)
Actual Result:
An error message appears when trying to sign up for the app using the “Continue with Email” feature.
Expected Result:
Users should be able to create an account using the “Continue with Email” option without receiving any error messages.
“Failed to reset” password error is displayed after entering email on “Forgot password” screen
Severity:
Critical
Precondition:
The user has an existing account.
Steps to Reproduce:
- Launch the app.
- Navigate to the “Log in” screen.
- Tap the “Forgot Password” button.
- Enter the email address associated with an existing account.
Environment:
iPhone 16 (18.2)
iPhone 11 (26.2)
Actual Result:
A “Failed to reset” password error message is displayed after entering the email on the “Forgot Password” screen.
Expected Result:
Users should be able to successfully reset their passwords.
Failure to open most recent chat with character after tapping button in “Jump back in” section when using OS 16
Severity:
Major
Precondition:
The user has a few recent chats with different characters.
Steps to Reproduce:
- Launch the app.
- Tap on any chat button within the “Jump back in” section.
Environment:
XS (16.3)
Actual Result:
The most recent chat with a particular character does not open after the user taps the chat button in the “Jump back in” section on OS 16.
Expected Result:
The chat should start immediately after the user taps the chat button in the “Jump back in” section.
AI “Podcast Host” persists in repeating same prompt after non-semantic user responses
Severity:
Major
Steps to Reproduce:
- Launch the app.
- Initiate a chat with the “Podcast Host”.
- Respond to the AI with either short or non-semantic responses.
- Observe the AI’s replies within the chat.
Environment:
iPhone 16 (18.2)
iPhone 11 (26.2)
Actual Result:
The AI continually repeats the same content despite non-semantic responses or short acknowledgements from the user.
Expected Result:
The AI should perform one of the following actions:
- Shift the conversation to a new topic.
- Pose a clarifying question.
- Cease the repetition of identical content.
AI voice responses are recorded as user voice messages in chat history
Severity:
Major
Steps to Reproduce:
- Launch the app.
- Initiate a chat with the “Outgoing Friend”.
- Engage in a conversation exchange by providing a few responses.
- End the chat session.
- Restart the same chat.
- Contribute a few more responses.
- Observe that AI responses were recorded as user responses.
Environment:
iPhone 16 (18.2)
iPhone 11 (26.2)
Actual Result:
User voice messages contain AI-generated audio instead of the actual user input.
Expected Result:
User messages should exclusively capture and display audio stemming from the user’s responses.
“Instructions” screen doesn’t open after tapping “Show me how” button
Severity:
Major
Precondition:
- The app is installed for the first time.
- The user has not set up microphone isolation.
Steps to Reproduce:
- Launch the app.
- Sign in to the app.
- Initiate a chat with any character.
- Tap the “Show me how” button.
Environment:
iPhone 16 (18.2)
iPhone 11 (26.2)
Actual Result:
Tapping the “Show me how” button does not open the “Instructions” screen.
Expected Result:
The “Instructions” screen should open after tapping the “Show me how” button.
“Open Settings” button does not open “Microphone Settings” screen
Severity:
Major
Precondition:
Microphone access has not been granted.
Steps to Reproduce:
- Launch the app.
- Open any chat with AI.
- Tap the “Open Settings” button.
Environment:
iPhone 16 (18.2)
iPhone 11 (26.2)
Actual Result:
Tapping the “Open Settings” button does not lead to the system’s “Microphone Settings” screen.
Expected Result:
The “Microphone Settings” screen should open after tapping the “Open Settings” button.
Unable to change “Onboarding” screen via swiping gesture or paging control
Severity:
Minor
Precondition:
The app is installed for the first time.
Steps to Reproduce:
- Launch the app.
- Attempt to move through the “Onboarding” screen using a swipe gesture.
- Attempt to change pages by tapping the paging control.
Environment:
iPhone 16 (18.2)
iPhone 11 (26.2)
Actual Result:
Unable to change the screen by swiping gesture or clicking on the paging control on the “Onboarding” screen. The onboarding pages can be swiped through by tapping the “Next” button.
Expected Result:
The user should be able to change screens by swiping or by clicking the paging control on the “Onboarding” screen.
Overall, the app feels innovative and engaging, especially the voice-based interaction and emotion-aware approach. The idea of more emotionally intelligent AI conversations is compelling, and the interface feels approachable and easy to start with. That said, I noticed a few bugs that negatively affect the user experience and natural flow of interaction. These could be resolved with comprehensive AI and functionality testing.
