A man who works to no gain, no matter how free his actions, is still a slave. A man coerced into a line of work who sees a gain from his labor may well be free.
Consider two men. One chose his profession. He was free to make any choice he desired. But now he works only for those over him. Whether the government, or some private employer, he spends his labor, but sees no profit. He produces, but his employer gains. The other man was "forced" by circumstance, or law, or tradition into some form of work. He had no real choice in the matter. However, he works and sees a profit from his labor. What he makes is his, and he disburses it as he sees fit, in exchange for whatever he agrees.
Which of these men has Liberty?
In many ways, the right to accumulate property- the right to work for oneself- is the essence of Liberty. Once I can work for myself (say, by starting up a blog and running Ads on my site. Or by selling my labor to an employer), I am the master of my own destiny. My decisions, and even happenstance, may make my life less than perfect, but I always have the choice of doing something differently. If I cannot work for myself, if I must work only for another's gain, then I am a slave. My life is not my own, because I cannot work for my own property. This applies to money as well as other forms of wealth.
When we seek to restrict private ownership of any kind of property, we are restricting Liberty. Sometimes, in the case of illegal drugs, for instance, Society has made the choice that this particular Liberty-for-Security trade is worth it. Only God and Time will be able to tell for sure. Other times, those restrictions are forced on us without explanation, or without our having a voice. When that occurs, someone is usurping, or attempting to usurp, our Liberty. From gun ownership, to land zoning, to mineral rights, to HOAs, to any number of other things: these are all restrictions on our Liberty.
Not all of those are unreasonable, or unnecessary. But you must remember, whenever you say, "There ought to be a law!" that you are advocating for the reduction of Liberty. That reduction, once made, is hard or impossible to reverse. We should be very careful about under which circumstances we will agree to such reductions.
When you advocate for higher taxes "on the rich," you are advocating for reduced Liberty.
When you ask that your HOA stop someone from building a tool-shed, or erecting a flag-pole: you are advocating for reduced Liberty.
When you request that the Government set prices for Health Care, and for increased regulations on how doctors and hospitals may practice: you are advocating for reduced Liberty.
When you want that Liberty back, the price may be terribly high.