Taking this to its logical conclusion you should be able to walk up the hill, push aside the cops, enter the rotunda, occupy the chambers of Congress all within your first amendment rights.
There is nuance and balance to be had in law, your rights end where others begin.