That would be entirely up to you and the other player involved! Doubles are physically identical in every way, and they have all the memories of their original. They will generally have the same mannerisms and quirks, or will be aware of them and able to duplicate them if necessary. So it's difficult to tell doubles and originals apart - but it's not impossible. If a double does something their original would never do, or says something sketchy, a friend could easily realize something's up, especially once it becomes public knowledge that evil doubles are running around. Even something as simple as 'hey, you weren't wearing that shirt when I saw you five minutes ago' could potentially tip off a suspicious friend.

So it's a plotting decision for the players, really! Have fun.