The complainant, one of seven young men the MP is said to have sexually assaulted, had been invited by Evans to the Strangers Bar at Commons one evening in May or June 2010.
The complainant said he was introduced to the MP who was standing in a group of people, including former MP Lembit Opik.
Prosecutor Mark Heywood QC asked him what then happened. The complainant said: "We were chatting and Nigel grabbed my genitals, basically. That is when I brushed it (his hand) away and said 'That's not cool'."
He told the jury that Evans then walked away as if nothing had happened. The complainant said he went to tell a friend, also in the bar, about the incident.
When it was put to him by Peter Wright QC, defending, that he made made the incident up, he said: " He should not be in a position of power where stuff like this happens and it is swept under the carpet."
The complainant said no one in the bar appeared to have witnessed the alleged incident and he made no mention of it to anyone else on the night.
The court heard that he and some of the group went on to another bar, along with Evans and both were 'tipsy'.
Former MP Lembit Opik told a court there was a "not very savoury" exchange between himself and the complainant over a round of drinks. He said he thought the man's behaviour and language was "entirely inappropriate".
Evans denies one rape, two indecent assaults and six sexual assaults said to have taken place between 2002 and last year.
The trial, at Preston Crown Court, continues.