5 User Research Lessons From the TV Show Silicon Valley

Matt Daughtry

5 User Research Lessons From the TV Show Silicon Valley

It’s no secret that the HBO series Silicon Valley had its fingers on the pulse of the tech world. Bill Gates said, “If you want to understand Silicon Valley, watch Silicon Valley,” and Evan Spiegel, CEO of Snapchat, called the show “basically a documentary.” Just as it poked considerable fun at developers, designers, product teams, marketers, investors, and more, the show was also unsparing—and enlightening—in its depiction of user research.

Two scenes, one in the second season and one in the third, showcase how the adversarial tech companies Hooli and Pied Piper conduct research to understand how users will react to their products. It's perhaps telling that, for as realistic as this show is perceived by those in the industry, we see so little user research, beyond these two brief vignettes. (Thankfully, the wider trend within the industry appears to be changing; in a recent study, only 6% of researchers said there were no dedicated user experience researchers at their company, down from 19% in 2019.)

While we can guess the 30-minute sitcom format put some restrictions on depicting user research more fully, these hilarious scenes still manage to reveal five lessons that are central to getting the most out of—and conducting—user research.

User-Research Exhibit A

Lesson 1: Understand That Focus Groups Aren’t a Panacea

First, though, let’s discuss the elephant in the room: the companies in Silicon Valley use focus groups to gain insights. Focus groups are not the best approach for answering the kinds of questions that Hooli and Pied Piper should have been asking.

There are potentially some instances where the creative collaboration of a focus group may be appropriate. But focus groups are not great for understanding behavior and user experience. [KB(1] [MD(2] Not only can they lead to groupthink and to perspectives based on hypotheticals rather than actual experience, but you also miss out on a wealth of insights.

Erika Hall, co-founder of Mule Design, noted, “If I achieve one thing with my time here on earth, I might be content if that one thing could be burning to the ground the practice of running focus groups in place of actual user research.”

In a more ideal scenario, the Hooli and Pied Piper CEOs should've never heard these criticisms in a focus group setting. They should've uncovered them earlier in the research process, during one-on-one interviews.

One-on-one interviews are typically much more effective at getting to know your user, understanding a user’s behaviors, motivators, and pain points. These conversations can feel informal, and both invite and encourage participants to share more details and insights.

And for assessing an experience, usability tests are far, far more effective than focus groups at uncovering problems and opportunities. You get to actually observe how someone uses a product or interacts with an experience, rather than hearing about it after the fact.

What Pied Piper and Hooli should have done was conduct user interview research early in the development lifecycle to better understand their customer and build empathy. In understanding the customer’s mindset and typical behaviors, you gain a strategic “north star” throughout that development lifecycle.

And as both companies were iterating on designs, they should have been conducting usability testing. An ongoing focus on usability leads to increased efficiency, reduced development costs, and increased customer satisfaction.

Lesson 2: Follow User Researcher Best Practices

That said, there’s still a lot we can take away from these examples, so let’s give them a pass for narrative convenience; these focus groups were probably funnier than a montage of individual interviews would have been.

In the video above, from the season two episode “Homicide,” we see customers’ frustrated responses to using the new Hooli phone—an in-development smartphone that’s being tested and is dubbed, “Apple Maps bad.” The moderator of the focus group (an amusing caricature of a researcher) manages to adhere to typical facilitation best practices, such as:

Avoiding asking leading questions.
Probing attendees to explain their thoughts and ideas further.
Remaining attentive rather than taking notes during the session.

The show writers certainly satirize a moderator’s tendency to cut down on bias by maintaining a neutral perspective, and this moderator would benefit from relaxing and building a more natural rapport with the participants. But it is critical to avoid leading participants to answers they wouldn’t otherwise arrive at.

Lesson 3: Don’t Be Scared of Incisive—and Sometimes Blistering—Customer Feedback

In the focus group participants, we see a group of people who are somewhat shy to kick off the conversation, but everyone has a strong opinion. It’s common for a moderator to have to get people to open up. That’s why building rapport is so critical.

And their feedback is frank, critical, and spot-on. In the moment, it’s not uncommon for a customer’s feedback to seem nothing but brash and unhelpful. Indeed, this situation is one of those all-too-real moments in this industry: using user feedback to convince key stakeholders (in this case, Hooli CEO Gavin Belson) far more viscerally than a slideshow ever could. In my experience, charts and graphs tend not to be as compelling as hearing a flesh-and-blood users say, “What the ****!?”

But this is why following up on their points and asking additional questions is so important, just as the moderator did here. Even those moments that seem benign or unhelpful may have threads that should be tugged. A good user researcher is like a toddler: they just keep asking why.

Lesson 4: Don’t Get Defensive or Be Offended

It’s not enough to understand that users are upset by something; we want to know why. The moderator’s defense, “I didn’t design this,” is all too real. As a researcher, I’ve been “accused” of making all manner of business or design decisions!

This scene closes with one of the Hooli clients suggesting they not take the feedback to heart. After all, it’s “just one guy’s opinion.” This, too, is an all-too-common reaction to negative user feedback. He is, of course, immediately proven wrong.

Lesson 5: Understand That You Are Not Your User

In the season three episode “Daily Active Users,” we return to the focus group facility for Pied Piper’s product evaluation (that again, probably should have been one-on-one usability interviews rather than focus groups). In this scene, Richard Hendricks, the CEO of Pied Piper, is reluctant to accept the results of the research. Rather, he wants to discount the results, because he sees the value of the product, and more importantly, his peers have given him very positive feedback.

We see highlighted here one of the central maxims that resounds throughout the experience design industry: You. Are. Not. Your. User.

“You are not your user” is a way of restating the false-consensus effect, which is the tendency to assume others share our opinions and experiences. As Arin Bhowmick, chief data officer of SAP and former chief data officer of IBM, noted, “Conducting user research activities helps us identify and address our own biases and misconceptions, which could otherwise be detrimental to the success of the product in terms of user adoption.” If you’ve ever tried to use an interface that felt clunky and overwhelming, like you didn’t know where to start, you probably understand how the users of Richard’s app felt.

But it’s not always just about something feeling “engineered;” more often a product team develops a sort of tunnel vision around certain topics and ideas and forget that their customers don’t share their intimate understanding of their systems.

Harnessing the Power of User Research

User research shines a light on all those little gaps that not only add up to a substandard experience but can also be costly to an organization or brand. When we fail to bring the voice of the customer to the table, we lose the most critical perspective in the entire customer relationship.

For both Hooli and Pied Piper, the user research was a reality check slap to the face, which is perhaps the most true-to-life lesson to take away.

If you’re interested in talking more about how user research can transform your product development, reach out to us at findoutmore@credera.com.