Creating a tool that records user interactions

My team kept running into the same pain point over and over in the beginning stages of every project: even when we had usability analytics in our hands, we still ended up relying on gut feelings and making educated guesses.

Analyzing data gave us a clear picture of what was going on, but it left us guessing why it was going on. We’d gotten tremendous feedback from usability testing, but we felt like that feedback wasn’t authentic since users behave differently when they know they’re being watched.

So we created Jaco, a tool that lets you see what people do on your website or application. No matter how complex your app is or which framework you’re using, Jaco can replay every user session as a video.

“Users behave differently when they know they’re being watched.”

During the process of developing Jaco, we learned a lot about the use case for such capability—and we actually used Jaco to improve Jaco. What follows is the story about how we added Jaco to our traditional workflow, and the added value we’ve gained from it.

Building the bare minimum

We had no idea if Jaco was something people wanted or needed, so we built the bare minimum to test the waters. That meant developing a proof of concept (POC)—and we decided to include 2 basic pages in it:

• The sessions page (our dashboard), which displays a list of all the recorded sessions and a basic functionality for filtering the sessions by date, username, and page URL (within the session)
• The player page, where you can actually play a real user recording and watch them interact with your website. Every user interaction gets listed in a transcript on the right side of the screen and is also referred to in the timeline.

The only goal here was to be able to store and then later on play user sessions. We had no designer working with us and gave minimal attention to user experience at the time.

Getting clients to use our basic product

Now that we had something that functioned, it was time to get our first beta testers. Since our goal was to find startups in various stages to gain a better understanding of our product market fit, we thought about the the best places we could go to find them. So we attended meetups and conventions geared towards startups, and we were able to find enthusiastic beta testers.

“No idea if people want or need your product? Build the bare minimum and test the waters.”

Our first beta testers handed us a list of feature requests and gave us priceless feedback, summarized below:

• Late-stage startups: “Although we learn something new from each session we watch, we have specific questions we want to investigate using Jaco but we lack the tools to do so.” This told us we needed to add more filters and integrations to allow customers with lots of traffic to use Jaco.
• Early-stage startups: “Watching our users interact with our product took the guesswork out of making hard decisions.” This told us that early-stage startups gain a lot of value from Jaco.

At that time we were an early-stage startup, so we decided to try a case study on our own product and use Jaco to go over the recordings of our beta testers.

Using client recordings to improve Jaco

We noticed a pattern related to the way our clients use the player page. When a new user starts using our product, they spend time watching their collected videos from start to end. After a certain period of time, the user starts watching videos at 3x speed, and later on, as they become more familiar with the product, they use the transcript to scan the session’s content. They also use the seek feature in the timeline.

Based on what Jaco showed us, we made 3 changes to our transcript feature:

• We moved it to the left side, where it’s more visible
• Originally we had seek behavior in the timeline (and not in the transcript), but since we noticed several users clicking on an event in the transcript prior to clicking on the correlating mark in the timeline we decided to add this behavior to the transcript as well
• We added different icons and colors to each type of event in the transcript, along with an auto scroll behavior, to make this component pop and encourage user interaction

Within 2 days, 73% of our users turned into player page experts: they started using the transcript to scan the session for specific parts of the recordings, and they used the seek feature to see only the most relevant parts of the recording. This change generated positive feedback from our clients, and we started brainstorming ways we could apply what we learned in the player page to the rest of Jaco.

We came up with the notion to reflect the session timeline (from the player page) in the sessions page, thus helping our clients scan faster through more than one session at a time. We mocked up our ideas and tested them via several 1-on-1 sessions with our clients. After this quick process, we designed the mockups below:

We invested most of the focus in creating a smaller version of the timeline, and we added the information from the transcript when hovering. Presenting the timeline in the sessions page gave our clients the ability to understand the tempo and the level of user engagement in the session with just a glance.

From there, we created a quick prototype. After testing our new concept with clients we concluded that while the feedback regarding the concept for this component was positive, some of our clients felt that our filtering flow was sluggish and overly complicated.

We realized that even though this solution is a good step in the right direction, there was still work to be done. We created new mockups with that feedback in mind, which led to the following design:

Along with improving the sluggish filtering, we wanted a solution that gave the chart more horizontal space and additional room for the information. We decided to remove the left column and build an entirely new interface for our segments and filters feature.

The new filters interface felt much more intuitive, flexible, and easy to use. We used JIRA’s filters as a reference. We loved how JIRA lets you choose how you input a query for segmentation.

Since the spectrum of our client base contains designers and product owners—visual people—on one end, and hackers on the other, this is a perfect solution for us.

And here are the final designs:

Summary

Since completing our personal case study, we’ve learned that Jaco benefits us best when adding it to our workflow in the following way:

• After analyzing all of our information from Google Analytics to figure out our status, we use Jaco to determine how we should improve things
• After every deployment, we use Jaco to view sessions where our new feature was used to get ahead of the curve and estimate the new release performance. This allows us to get validation without the need to wait for a critical mass of traffic for the analytics to be relevant.

This approach allowed us to be data-driven during our early stage when our traffic wasn’t heavy enough to be aggregated, and it’s now helping us move faster and be more data-driven since we actually get live feedback from real clients within an hour after a new rollout.

Jaco is currently in the last few weeks of its free beta phase, and we’re working vigorously to add more integrations and advanced filters. In the near future we intend to add more tools that will allow product owners to get aggregated data regarding the usage of their product from both the sessions and users’ aspects.