It is mentioned several times in scripture that Jesus sometimes hung out with a rough crowd. Mentioned are prostitutes, sinners, tax collectors, drunkards, gluttons, etc. This was the opposite of hanging out with the religious crowd of Pharisees, scribes, elders and other officially religious people. Whenever Jesus was around the religious crowd it was usually concerning conflict and disagreement over Jewish Law and his Kingdom grace message. When he was around the rough crowd everyone seemed to be blessed and comforted.
The issue is not the rough crowd or the religious crowd (and they might be the same people in some circumstances) it is being the same person in attitude and action wherever and with whomever. If there is a conflict in personality it is being some kind of pretend religious adherent and at the same time a real (or pretend) rough crowd person. When we have it both ways neither are effective.
“The Son of Man came eating and drinking, and they say, ‘Here is a glutton and a drunkard, a friend of tax collectors and sinners.’ But wisdom is proved right by her deeds.” Jesus