http://www.codeacademy.com just sent me a mail declaring that they have an HTML and CSS course available now (for free).
Other than that, I think you should continue to dabble. Just a bit more structured:
First decide what kind of website you want to create, and then visualize it.
Then decide how you are going to update content on the site. There are alot of options, but basically it's going to be a (L)AMP variant, unless you're creating a presentation or more basic site. This is not a small step. https://en.wikipedia.org/wiki/LAMP_(software_bundle)
There are frontends/frameworks/content management systems you can use, such as django, ruby on rails, wordpress, etc, but I'd recommend starting coding your own source first, and when you have an idea of what you're looking for, start going after them.
Also, learn what XML is and the Document Object Model, even if you don't (think you) use it.
When you've rebuilt the website the 5-10 (or 50, or 200) times, from mockup, to basic dynamic site to whatever it takes to get a good result, you'll have a good understanding of the various components that drive a website.
Cost? Time and a raspberry pi (unless you already have a left-over computer, or get alot of traffic on the site).
Even if you take online courses, you still need to understand the underlying systems supporting what you're creating. At least on Lynda.com, the courses often glance over quite important information, just to keep the flow and attention of the viewer. Only actually practicing (preferrably with the help of a more in-depth text/e- book) will help you understand what is actually happening, why it's happening, and - as you rebuild your website - what parts are actually needed to form the structure/frame of your site, what can be provided from the database and framework, and what you need to "hard code."
Or you can do the "2k" method, too. Watch your online courses, install all the software you think you need, and google all the issues that crop up as you go. Not recommended if you want to become more than a dabbler, though.