We can also distinguish "free speech" and "freedom of speech".
Many people believe (erroneously, I think) that free speech is the right to say whatever you like about whatever you like, whenever you like. Not quite.
The right to "freedom of speech" is the right to seek, receive and impart information and ideas of all kinds, by any means.
It concerns the posession and communication of information, ideas (including feelings), facts and knowledge. (Information, ideas, facts and knowledge are not exactly the same thing as opinions, though opinions can meet the criterias for any of those four.)
"Free speech" is speech or expression that neither causes harm nor is itself harmed.
As you can see, freedom of expression does not imply that all expression would be free expression, although that's the ideal goal.
Free expression and the right to freedom of expression can be applied to ideas of all kinds (yes, including those that may be deeply offensive), but they always do come with implicit or explicit responsibilities, which makes them inherently restrictive by themselves. They carry an inherent responsibility to not cause harm.
It is because such acts of expression come with inherent responsibilities that the way in which they are performed can be legitimately restricted. Restrictions can be justified if they protect specific public interest or the rights and reputations of others from unjust harm. There is the complication that some harm is echically just and good. For example, harming a person or group's reputation may be an inevitable consequence of publishing some truth about their characteristics or actions (the good of the publication then also depends on wether or not it's actually true).
This much, most people agree on.
The root of the issue then is what exactly constitutes harm?