Exploring Content Usefulness and Effectiveness Through Real-Time User Feedback
Background: According to a recent report by Acrolinx, nearly half (47%) of business to business buyers view 3 to 5 pieces of content before contacting a sales representative. Prospective customers search for solutions online by visiting company websites, reading product documentation, and reviewing white papers to learn more about potential products. Based on these findings, creating effective and informative content can impact business decisions and add value to a company. The purpose of the current project is to conduct extensive research on the ServiceNow Product Documentation website (docs.servicenow.com) to explore content usefulness and effectiveness and evaluate the impact on customer satisfaction.
Methods: To collect real-time feedback from users interacting with the Product Documentation website, we deployed a site intercept survey that is triggered based on specific user behaviors. We developed 4 specific blocks of questions based on user behaviors to explore content usefulness and effectiveness. When a user searches on the site and selects a result from the suggested list of options, the search block of questions pops up after a 1-minute delay with questions related to the search experience. The excessive scrolling block of questions pops up after a user scrolls up and down 3 times on a page of content. To evaluate multimedia content, the multimedia block of questions pops up after a 3-minute delay on any pages with YouTube embedded links. Finally, a general block of questions appears after a 1-minute delay on all other content pages. To allow for pooled data analysis across triggers, we included some consistent questions regarding user role, reason for visiting the site, experience with ServiceNow, and any other content resources the user accessed to try to accomplish their goal.
Results: The survey went live on October 27, 2019 and within the first 24 hours, we collected 529 qualified responses. To date, we have collected N=2,450 survey responses with the largest blocks of data coming from the general (N=968) and excessive scrolling (N=825) blocks. Additionally, we have N=427 responses for the search trigger and N=230 for multimedia. The most common reason for visiting the Product Documentation site across all triggers is to solve a problem (25%) or learn how to use an application or feature (37%).
Implications: Next steps are to continue to analyze the data to explore relevant trends. Importantly, we want to use the data to generate specific research questions related to content and identify methods to improve the user experience through ongoing projects to improve Product Documentation. Our long-term vision is to continue to innovate on how we present product content to ensure users are successful in navigating the site to find the content they need to accomplish their goals.
What can the audience expect to learn?
Given the impact product documentation has on purchasing decisions and customer satisfaction, innovative methods to explore content usefulness and usability are essential. Site intercept surveys collect real-time user feedback that is directly attached to a specific content experience and can be leveraged to evaluate key content metrics. These data are benchmarks to explore how content improvement activities affect user experience.
Meet the presenter
Michelle Takemoto is a behavioral scientist with over 10 years of experience using evidence-based principles to drive engagement, adherence, and outcome improvement using both in-person and digital modalities. She graduated with a PhD in public health specializing in human behavior and has expertise in user-centered study design, behavior change, and quantitative and qualitative methodologies. She is a UX researcher at ServiceNow focusing on the product content ecosystem and is proficient in engagement to encourage continued participation with a digital platform using innovative methodologies.
Katie Ottt manages several teams for the Product Content organization at ServiceNow including, Release Notes, HR, and ITSM. She is also the Product Content lead for the Employee Workflow and is leading efforts around research and metrics for the entire Product Content organization. Prior to joining ServiceNow, Katie was at HP where she worked to define and implement requirements for a second-generation content management system, helped merge several documentation teams into a central function, aligned internal processes across functions including localization, CSAT, operations and support, and eventually took over management of the printer documentation, unassisted support, localization, and multi-media teams.