I've been interested for a while, as a 'just for fun' project, and I'd like to hear how difficult it is to do. Is it actually a possibility if you run a low/medium traffic website?
For my own "just for fun" project (http://www.babynamemap.com), I wanted to be able to handle more than one visitor at time. :) I'm hosting the RoR app on the cheapest Slicehost VPS for $20/month.
So, if you don't care about many concurrent visitors and decent uptime, then save $20 and host from your home. However, if you actually want the general public to come visit, then get yourself some cheap hosting.
The biggest counter example I know of for a site that started this way and ended up huge is Markus Frind's free dating site, Plenty of Fish.
"I was running the entire site off my home PC and ADSL connection for the first 8 months." (not sure if this is the original site that hosted this interview, but it's the first one Google turned up: http://www.traffick.com/articles/innovators/01-markus-frind.... )
Ran web app from desktop -> ssh reverse proxied through to cheep $20/month ~unlimited bandwidth server. sshfs on desktop so static files that were uploaded go through the desktop and back up to the shared server so they'd get served quickly to users.
Then I got a faster laptop so I decided to run everything but the database off of that. So it was user->shared host->laptop->desktop with database->laptop and all the way back for every request.
Latency wasn't really that bad as long as the shared host was nearby.
I once got on the front page of digg with a home server, but it was before that meant a lot of traffic so I think the server actually survived.
I wouldn't recommend it though. Mine was just a hobby site, and running the server was part of the fun. For anything more than that you are better off with professional hosting.
If you want to gain some sysadmin knowledge it's probably the single best way to start learning. It's just like having a machine in the shittiest datacenter possible, that just happens to be very conveniently located.
And as far as I know, getting SDSL or a T1 is gonna set you back a few hundred a month, at which point you're better off getting a dedicated box.
Start with a VPS -- even if 30 bucks a month sounds like a big chunk of change, consider your other overhead (rent, groceries, etc). Not a big price to pay for the benefits.
Good luck!
As for power, you've always got UPS'.
Internet is the tricky one as all private hosts use redundancies, although I don't see why you couldn't sign up for regular internet and a cable connection.
I've seen a handful of ISPs that let you run a server (or more correctly, don't stop you), although I think most ISPs prohibit it to force you onto business packages (which you can get in your home too).
I know in the Toronto region there is Mountain Cable, which doesn't have anything in their Terms of Service preventing you, and have unlimited bandwidth (From speed tests I've seen, they're faster than Cogeco, and get an upload speeds in the 1+MB/s range). I think you've just got to settle with DynDNS, however from what I've read this is reccomended for static IP's too. If you did have two ISPs then this would solve the IP problem when you switch.
I'm actually considering going with Mountain Cable anyway as they're cheaper than Cogeco. Although their lack of information on their website freaks me out; I mean their service area map isn't even online. (Edit: They do have a service area map for their internet, however they apparently provide service outside of the area)