Even then belonging to the communist party was not illegal.They were card carrying communists in places of power or significant influence. Alger Hiss is a clear example among others that indeed we were infiltrated by a foreign ideological movement every bit as vile as Nazism.
And before, during, and after they were all free to leave and start over in political climates more suited to their preferences.
Persecuting someone on a rumor is more like Salem than actual justice. Did we go after a pro-nazi like Lindberg or Bush?