Family App NPS

Improving Customer Satisfaction and NPS Score through Comprehensive User Research and Data Analysis

Project Overview

Client: Smith Micro Software
Project: Tier 1 Carrier Family Safety App
Project Duration: [December 2023] – [July 2024]
Role: Manager, Customer/User Experience
Team: Sr. User Experience Researcher, Data Scientist, Product Manager, UI Designer

Tools & Methods:

    • NPS Survey Analysis
    • App and Play Store Reviews
    • Alchemer (Apptentive) No Love Surveys
    • User Interviews
    • Data Synthesis and Analysis
    • Design Recommendations

1. Background and Context

Project Goal: The primary goal was to enhance user satisfaction and loyalty by improving our app’s Net Promoter Score (NPS). The NPS score is a key indicator of user sentiment and loyalty. Our previous scores were negative on both iOS and Android, signaling significant dissatisfaction among our users. The objective was to identify the root causes of dissatisfaction and implement changes to boost the NPS to positive levels.

Challenge: The app had received consistently poor ratings across both iOS and Android platforms. Our existing NPS data, app store reviews, and additional feedback indicated various pain points and areas needing improvement. Our challenge was to analyze this feedback comprehensively and use it to guide the development of a new app version that users would prefer.

2. Research and Discovery

Analyzing NPS Survey Data: We started by reviewing historical NPS survey data to identify trends and patterns. Key insights included:

    • Frequent Complaints: Users expressed frustration with children circumventing the app, location tracking issues, and a confusing user interface.
    • Feature Requests: Users requested additional functionalities such as blocking the entire device during off times and contacts/calls & text management.

App and Play Store Reviews: We systematically analyzed app reviews from both the iOS App Store and Android Google Play Store to gain deeper insights into user complaints and suggestions:

    • Common Issues: Location accuracy, frequent app log outs, and user interface difficulties were frequently mentioned.
    • Positive Feedback: Users noted they are generally very happy with the app’s parental controls as the features were well-received and highlighted as strengths because of how it helps manage their child’s online use.

Alchemer No Love Surveys: Alchemer (Apptentive) No Love Surveys provided additional qualitative feedback:

    • No Love Dialog: Every 45 days, users are eligible to see the Alchemer Love Dialog box which asks a simple question, “Do you love “Family Safety” app?” If users select yes, they are promoted to rate the app, if they select No, they are taken to a survey.
    • Detailed User Sentiments: Users provided candid feedback on why they “do not love” the app and suggested potential improvements.
    • Sentiment Trends: We identified recurring themes and sentiments that aligned with the issues found in NPS and app reviews.

User Interviews: Conducted user interviews to gain a deeper understanding of individual pain points and experiences:

    • Interview Findings: Users experienced frustration with onboarding processes and found some parental control features difficult to use. There was also a strong demand for enhanced circumvention prevention options.

3. Synthesis and Insights

Key Insights:

    1. Child Circumvention: The inability for the app to prevent the child from removing the VPN or deleting the app were major pain points affecting user satisfaction.
    2. Location Tracking Enhancements: There was a strong desire for the accuracy of the location tracking to be more precise and accurate.
    3. User Interface and Experience: Users struggled with setting up the parental controls and found the features to be difficult to understand.

User Personas: Developed user personas based on our research to represent the primary user types and their needs:

    • Persona 1: [Sarah, Mother of young child (11 and under), Needs: to manage her child’s screen time while ensure inappropriate content is not viewed, Pain Points: Worried that bad content will slip through the crack while child is online]
    • Persona 2: [Mark, Father of older child (12-17), Needs: Location tracking now that their child is more independent and managing screen time, Pain Points: preventing the child from circumventing and disabling the app or removing the VPN in order to bypass the parental controls to have more screen time.]

Customer Journey Mapping: Mapped the user journey to identify critical touchpoints where users experienced issues. This helped prioritize areas for improvement.

    • Touchpoint 1: Users are not setting up and pairing the child’s device properly.
    • Touchpoint 2: Users are unaware of the location tracking settings that need to be both enabled (or disabled) in order to receive the best accuracy possible.
    • Touchpoint 3: Users are unaware of all the features in the app that can help them manage their child’s online use and/or do not understand how to set them up properly to receive the full benefits of the app.

4. Design and Implementation

Redesign Recommendations: Based on our insights, we recommended several key changes:

    • Circumvention Optimization: Enhanced parental controls and introduction of Google’s Accessibility Permission to prevent the child from circumventing as well as blocking the entire app during off times and bedtimes. This feature, once enabled, will prevent the child from disabling the controls and the permission.
    • Feature Education: Created an educational campaign where a series of “Did you know” Firebase in-app messages are presented to the user to educate them on the new features (Accessibility Permission, Block Device, Updated Content Filters, Updated Time Limits) to increase adoption of the new features and so users can see we listen to their feedback and improved the app based on their needs.
      • We also sent out a message about the location settings that needs to be enabled to improve accuracy. We then began to respond to individual users’ feedback through the use of “Custom ID” in Alchemer to let users know we are listening to their feedback and have a solution for their problems.
    • Updating Onboarding: Pending: Currently working on Phase 1 and Phase 2 of an Onboarding overhaul which will remove unnecessary steps and get the user to the Hub faster. Additional education of setting up the features through the use of tool tips, toasts, and pop-up messages are planned for Phase 2.

Prototyping and Testing: Developed prototypes incorporating the recommended changes and conducted usability testing:

    • Testing Feedback: Tested the Accessibility Permission flow with 10 users to verify if the new interface and instructions would have performance improvements.

Iterative Refinements: Made iterative refinements for the next release based on usability test results and further user feedback.

5. Results and Impact

NPS Improvement: After implementing the changes and running a new NPS survey, the scores improved significantly:

    • iOS: Increased 33 points and went from a negative to a positive
    • Android: Increased 20 points and went from a negative to a positive

App Store Ratings: App store ratings also saw notable improvements. Circumvention complaints on Android have decreased and positive feedback increased due to users reported greater satisfaction with the app’s performance.

    • iOS: Increased from 4.4 to 4.5
    • Android: Increased from 3.5 to 4.4

Love Percentage: The percentage of users who tapped “Love” to the pop-up ‘Do you love the app” increased on both platforms.

    • iOS: Increased from 58% to 72%
    • Android: Increased from 54% to 67%

6. Lessons Learned and Future Recommendations

Lessons Learned:

    • Comprehensive Analysis: Analyzing multiple sources of feedback provided a holistic view of user issues.
    • Iterative Design: Iterative design and testing are crucial for refining solutions and ensuring user satisfaction.

Challenges:

    • Customer Restrictions: Due to the customer rejecting implementing the most requested feature, Contacts/Calls & Text because it requires their API integration which was not approved, we had to focus on improving the users experience through education.
    • OS and Technical Limitation: We were able to address the circumvention issue with Android child users by implementing Google’s Accessibility Permission, but were unable to apply the same feature on iOS as they only have the STAPI feature to prevent circumvention but is limited only to children under the age of 13. Data shows that younger children do not circumvent the app like older child who have become more tech savvy and rebellious do.

Future Recommendations:

    • Continuous Monitoring: Continue to monitor user feedback and NPS scores to proactively address new issues.
    • User Engagement: Regularly engage with users through interviews or Alchemer notes to keep up with evolving needs and preferences.
    • Update Onboarding: To prevent issues where the users are not setting up the app correctly or using all the features to the fullest.
  1. Conclusion

By employing a rigorous research methodology and incorporating user feedback into the redesign process, we successfully enhanced our app’s user experience, leading to a substantial increase in NPS scores and App Ratings. This study underscores the importance of user-centered design and continuous improvement in achieving high levels of user satisfaction.