This answer is long and in two parts.
Part One: The Legal Crap
As far as I know, most "Criminal Anarchy" statutes date back to the cold war or the original red scare. There is also the related Espionage Act, which led to the deportation of many anarchists, including Emma Goldman and Alexander Berkman.
I am no legal scholar (thank Godwin!), but I believe that the legal line (which is just as malleable as the state wants it to be, so "line" is maybe the wrong word here) is that you can think that the world would be better if the United States was completely erased and in place was a collection of bolos (a bolo of bolo?). You can say "it is my opinion that the United States should be replaced by a bolo'bolo." You could go on Anderson Cooper 360 and say, to CNN's entire fake-news consuming viewership, "Well Anderson, you silver fox, the increasingly evident plutocracy and concentration of capital evidenced under the Trump regime, but which has been present throughout United States history, is exactly why we would be better served by splitting off into specific bolo, and then having relations between them in a bolo'bolo."
If, instead, you went to a rally of people who would prefer bolo and said, "fellow ibu, we have suffered too long. to the bolo'bolo!" and then the people who would prefer bolo declared themselves ibu and began actively seceding from the US and forming bolo (bolo-becoming?). That would probably be where you've crossed the line. If you happened to have a stockpile of materials designed to be used in the furtherance of bolo, you would definitely be fucked, and any literature or other evidence that showed you were dedicated to the furtherance of bolo would further sink you.
But I'm not a legal expert, I'm an anarchist.
Part 2: Fuck the Law
Yes, the police use force on nonviolent protesters all the time. The role of the police force is maintaining order for the powers that be, which ultimately boils down to the free functioning of capital, although there are lots of other things that they reinforce, the purpose and function of modern police is grounded firmly in defending the economic status quo. Demonstrations are seen as inhibiting the smooth functioning of the system ("the system" is complicated, I hate the phrase, but I'm using it as shorthand here). At the same time, cracking down on everything runs the risk of widening discontent, leading to even larger disruptions. Whether we are talking rioters, peaceful Black Lives Matter protests, tree sitters, or what have you, at some point the flow of capital must continue, and status quo must be reestablished. That is when the truncheons come out, the pepper spray is deployed, and blast balls get thrown.
Where things cross the line varies, but the line exists in every conflict with power. The need for police to cross that line is mediated by the use of permits and the state sanctioning of demonstrations, which might be one way (but certainly not the only way) to contrast the state reaction to J20 Inauguration Day actions last year to J21 (the Women's March). It is much more complicated than that, obviously, and I am not going to write a whole book on this site...
Regarding the state doing better research, it really depends which parts of "the state" we are talking about. There are some parts of law enforcement and domestic intelligence that have very good understanding of the culture(s), goals, and methods anarchists employ. At the same time, there are frequently assessments that are surprisingly daft. Ultimately, what matters to the state is that we are enemies of them. Frankly, I am not an anarcho-pacifist, but I don't think the state really cares much about what tactics you employ or what particular ideology you hold, you are an enemy, enemies must be contained, marginalized, and/or eliminated.
