I used to hangout in the old forums now we are in this new cool LVL1tech , so hi everyone.
I need the opinion of the networking wizards here to help me chose what to buy for my pfsense project .
I will put the links to the website that I found some old servers ( I believe will need to have as big as storage as I can afford , because we have download qouta and using pfsense caching will help me to avoid the extra $$ as far as i know that how it works for youtube and downloading steam games etc… because the other computers already have downloaded them etc so it wont download them again from the internet but isntead it will be taken from the router cache right ? (pfsense cache ) i think works something like that , now i have many computers like around 5 that will access it at the same time to back up videos also on other hand around 10 will copy games etc… )
so first thing came to my mind is SFP+ is a must since it will copy huge files .
PRices are in AED ( coz i live in dubai no download limit but once we transfer to 2nd location we will have qouta limit )
am trying to keep it under 800$ so hopefully looking into used workstations/ servers will help . some of the links are brand new. thanks in advance
Links :
250AED Server rack ( look like 10 U )
1,100 AED
1,500 AED
1,500 AED
1,450 AED
very good 3,000 AED
2,600 AED
might be the best but dont think it have gpu
2,700 AED same as the above but with extra ram and gpu matrox is included
1,450 AED
1,850 AED
1,800 AED windows server 2012 original come with it , scroll down will see the buying price is 1,800
1,800 AED
forgot to mention very important factor we need to render on the network so using NAS is a must
I will let others comment on the hardware requirements. I would just like to add that as far as I know anything with an SSL encryption is not going to be useful to cache. So that rules out youtube videos (link to 10 months old forum post and a 2 year old forum post). It seems some of the problem is that youtube constantly change their code. I don't know about steam games though.
Squid caching doesn't work for steam games also, the problem is that Steam has hundreds of CDN servers and you can't guarantee that you'll download form the same server. It does work if you download from the same machine minutes after the initial download but what use is that?
It will as long as they are not using HTTPs. For that you would need to Man-In-The-Middle your connection. Browsers do a lot of caching these days, so your millage will vary. I suggest you drop a squid installation on VM somwhere and do some testing first before you drop money on equipment.
The Stream issues can be easily remedied by using a shared drive for games.
In theory yes, but video content sites also use CDN servers so you may run into the same issues as caching Steam games and obviously as @Zumps pointed out, if those sites are also using SSL it still won't work.
thanks for the link . I am reading it , ok for the steam games will share them on the NAS ( in pfsense ) for the cache I still think will save tons in the long run because its not only youtube and videos we use , so having a cache will be good in general its not the only reason am going for pfsense , we still need to edit videos and download huge files so NAS will be good and having ability to control my router like in pfsense I beleive is better than spending lot of money on 2 devices a good router + nas seperate . while a building a pfsense with all these features with ecc ram will be good for all these read and writes going to happen specially that some of the guys will be video editing form these files ( even tho i recommended that they copy the files first before they edit it ) but sometimes they do it anyway .
so still after youtube and steam not working with cache it is a must to have cache ( btw forgot to mention very important factor we need to render on the network so using NAS is a must )
browsers do caching on the same machine but not on the network. right? lets say i am reading this post so another person on same network decided to go read it , he wont need to load it off the internet he will just get it from the buffer ( unless this site use HTTPS right ? ) so it will still be very useful
I disabled squid caching on my pfSense box not long ago for the same reason. I didn't notice any difference and in some cases squid can even negatively impact performance as the traffic has to go through the squid proxy whether it is cached or not which will add latency.
I am pretty sure it is discouraged to run both. I am not aware of any extensions for pfsense to make it run as a NAS if you want data redundancy and such, but you might want to check it out. What I do know for sure is that it is definitely not recommended to run pfsense as a virtual machine, in your situation next to a nas on the same machine.
I agree with @Zumps, it isn't recommended to run critical infrastructure such as router/network storage on the same machine encase something goes horribly wrong, especially if it is on bare metal. Could possibly get away with running the NAS on Linux and pfSense in a VM but most people will tell you that it is better to run both on separate machines for security/reliability reasons. If it is any consolation pfSense is very efficient and doesn't need a powerful machine to run so you get get away with a cheap dual-core. I have mine on an Athlon 5350 which isn't a very remarkable chip in terms of performance and it doesn't even break a sweat.
hmm so you mean it is bad idea coz if something goes wrong it will damage my router + my nas ?
coz I am not thinking to make a vm machine I was going to use Pfsense and the built in Feature for NAS. ( I assumed it have one , coz i remember watching a video about that back in tekSyndicate days )
btw I got multiple projects I should have made that clear XD
1st is my own home thing 2nd is my work for the company I work for 3rd project = me learning stuff XD so being able to learn maybe linux server stuff or windows server stuff is good or at least build my own noob router/nas
so for my own home thing I thought perfect idea to make a machine all in one , sort of media pc + router + nas+server then i decided to make media pc into nvidia shield pro andriod TV but keep the idea 3in1 router/nas/server
so now you making me not to do it :/ , so the general idea is am making it for my own use at home and I am crazy guy who like to store everything on the internet , right now am handicapped by my 12TB full so need a backup for them and need a extention like at least another 12TB to store more . also I got many ppl in the house who would love to be able to stream movies etc.. from my collection etc..
on otherside I need similar setup for work coz we work with animation stuff video editing and VR development stuff , I got a super fast switch with SFP+ in it so will try to get a server and (we have a rack 22u ) put a sfp+ card in it to be able to handle the huge network need , we have a cyber cafe that will also utilize our server/nas to copy games and share them instead of download huge amount , ( we also having 2nd branch oppening soon and it will be in a place where our qouta is limited so I thought should already build copy paste system that will be caching data to save qouta )
that is the full story :D thanks for help guys , u are being very helpful . i dont know who else to ask , am supposed to be the most Tech savvy in my company/social network so.. XD
I can't remember seeing such a video, as far as I am aware pfSense doesn't have a built in NAS feature, you'll have to modify it yourself to do that function.
Also I found some resources on the same subject while doing a quick Google search, one of them was from this forum:
The TL:DR for pfSense+NAS is that it'll only work if you are using VMs but it isn't advertised for security purposes as pfSense is marketed as a firewall and is critical security infrastructure, It is the first line of defence on the network and rolling a router+data storage on the same machine is a nightmare for security as your data is directly at risk if pfSense is compromised.
Trust me when I say first line of defence, my firewall blocks tens of thousands of connections from Asia alone.
Prettymuch this, OP. Because so many sites are SSL now, Unless you do MITM SSL (which squid can be configured to do and deploying your own trusted SSL is easy if you have a technology like ldap/active directory making shared auth/config easy), Squid won't do much for you. Steam and other services that might be using a lot of bandwidth on your network provide their own mechanisms for caching. Almost all of which involve an override in DNS.
I'd suggest separate PFSsense/network storage boxes, though. If you're on windows you can use WSUS to cache updates. Or on windows 10 the obscure checkbox "get updates from other computers" works fine and uses a checksum to verify the updates that came from a peer are legit.
now I will have to build 2 systems , 1 will be a freenas and 2nd will be pfsense . pfsense will run on very low specs I can use my current extra 2 psu's and Gpu to make a pfsense ( xfx XT600w two of it and gtx 680 also a geforce2MX 64MB hh from a friend who used to have a pfsense )
XD omg wendell did reply to this post. well am doing research and learning more now because half of the Terms you used I dont know them ( MITM SSL + ldap ) , so thanks senpai I got more materials to read&learn now.
(about the windows thing Yes I thought of that yesterday at work)
I mixed them up i think =.=! freenas video with pfsense one i guess.
ok going as everyone suggest seperate builds. then for my own home maybe something with ready built Nas from Qnap or thecus etc..?
and for the office one of these old servers . Btw what you think about the z620 from HP here found it used for 2500 AED = 680$ and can negotiate price to drop maybe it is worth it ? ( like wendell says dumpster diving hhh )
I'm no expert on enterprise gear as most of the stuff I am using is consumer hardware with enterprise/server software, mostly because of noise and space requirements as I don't want a rackmount server in the room going full belt while I sleep.
I used to hangout in the old forums now we are in this new cool LVL1tech , so hi everyone.