The Avaton and Atom processors are pretty good for small servers, and if you want some more power, I3's are pretty handy for the job, and support ECC ram, even though you probably wont need it unless you have classified documents on your media server (joking.). 8gb is pretty good but you can just as easily use 4 as well.
I don't have much knowledge so someone else may have to jump in, but i just use Plex for my media server, and its really nice. But if you're using something like freeBSD, i have no idea how to set that up.
I build Media Servers all the time. If this is the rig that just serves up the media but does not actually display it, then the following is an example of an excellent build that I have used several times.
The AMD chip is perfect for the work load. It is a Quad core with ample cache and performance for the multi-threaded applications aka server workload. Could have gone AM1 and still gotten Quad core but the FM2+ chips are more beefy.
8GB of RAM minimum
SSD for OS only
3 Drives for RAID 5 and a fourth for RAID 6 or RAID 0+1. Each drive is designed for 24/7 access. 64mb cache per drive.
Cheap GPU. no need for anything special.
DVDRW for any DVDs you want to rip to the server.
Install OpenSuse server or CentOS server or Fedora 21 Server with BTRFS for filesystem. Alternatively run a Debian or Ubuntu server with EXT4 for Filesystem. Install Kodi Media Center for the Media Server. Kodi is the new version of XBMC (What plex is built from). I used to run Plex but ditched it for Kodi.
I have a media server build that is running the e8500 at 3.0ghz with several 3tb drives in it and I dont have a problem with it. The mother board that I am using is the MSI laa775 combo so I could use ddr3 ram. I did use a dual nic card to hook to my router so that I wouldnt have to worry about loading up the lanes to my server. A lag 775 chip works great.
Here is an alternate build using the AM1 chipset. This build has a dedicated RAID card and an additional drive for RAID 10. Total Capacity = 4TB in RAID 10.
PCPartPicker part list: http://pcpartpicker.com/p/GdT24D
Price breakdown by merchant: http://pcpartpicker.com/p/GdT24D/by_merchant/
I had a E7700 running a media sever with 4GB memory it had no issues decoding multiple 1080P streams. I've used Linux and windows as the OS either work fine depending on how you intend to serve the media. I use Kodi for the front end on whatever machine is playing the media and PLEX purely for the sever client.
Depending on needs I recommend a Qnap or a Synology as you can set up PLEX server off them and they decode 1080P just fine for 1 or 2 streams while running Sick beard or Sab etc in the background.
If you need to run a VM or more than a media server with 1 or 2 streams LinuxMaster9s build is really good. A little over powered if you're only using it to download and transcode media but being more powerful than needed is never a bad thing and opens options.
Actually, I think you are not decoding the stream before sending it. If the client machine is running kodi, there is no need to transcode the stream on server side since the client would have the necessary codecs anyway. I usually run Kodi for the backend on the server and the frontend on the client.