Imagine you’re in a tense situation. You’re going on a first date and want to find out if the other person likes you (or is a serial killer). Your boss calls you into their office to discuss your idea, and you want to know if they’re being completely honest or hiding something. Or perhaps you’re a police officer and want to determine if the driver you stopped is just anxious or potentially a threat.

Which of the following clues about body language could help you?

  • People smile and laugh when they are happy.
  • Strangers cross their arms when they are uncomfortable or defensive.
  • Someone doesn’t meet your eyes and glances away when they are telling a lie.

If you thought they all could help you, you are correct. You are also completely wrong. As it turns out, body language isn’t as predictable as we’ve been told it is.

Malcolm Gladwell’s latest book, Talking to Strangers, reveals why you shouldn’t rely on your ability to read body language. The book shows that the skills you typically use to interpret behavior in people you know don’t translate well to interactions with employees and strangers. Here’s the reason why.

Body language is not conventional.

The first thing you need to know about body language is facial expressions are only conventions. In Western cultures, we believe upturned lips are a smile and indicate someone is happy; downturned lips are a frown and show someone is sad; very wide eyes mean someone is surprised. These conventions are so prevalent we have emojis for them. The only problem is people’s body language and emotions don’t consistently work that way.

If you show these same emoji types of expressions to people across the world (think: people in remote tribes on different continents), these people don’t agree on what the expressions represent about a person’s emotional state. If you observe people around you, you’ll notice your friends’ and family’s body language doesn’t always match their feelings. Your friends don’t always smile when they are happy, frown when they are sad, or have wide eyes when they are surprised. The reality is people will not often reflect what they are feeling like emojis.

Someone might look away because they are nervous, not because they are lying. Another person might fold their arms because they are cold, not because they are shutting people out. And that smile might just be for politeness—they don’t really like you.

Body language is not universal.

The second thing you need to understand is body language is often particular to an individual. If you remember the Amanda Knox murder story, then you understand how body language can be misinterpreted. Amanda Knox was a young American studying abroad in Italy when her roommate was stabbed to death.

Despite zero evidence of Knox being at the crime scene, police pursued her as a suspect (then charged, convicted and later exonerated for the crime). It all came down to the police not liking her behavior. Knox didn’t cry. She wasn’t angry. She even kissed her boyfriend while police were investigating inside the apartment.

However, the police didn’t know Amanda personally. They had no idea of the quirky personality she was known for in the US. They didn’t take into account she hardly knew her roommate or the relationship with her boyfriend was exciting and brand new. Since she didn’t meet expectations of behavior, the Italian police saw her as an indifferent, stone-cold killer.

People’s behavior is not universal. You have to know a person well to decode their body language. You have to understand the context to know why they act the way they do.

So the next time you look at a colleague’s or employee’s body behavior and think you know what’s going on, think again. You might not have all the information to understand why their face looks a certain way or why their body is positioned in a certain fashion. People don’t look or act as we expect, which is why you shouldn’t assume how someone feels just from the way they look. It pays to take your time with people.