Network design for 50+ people

So I have recently come into a situation where I have to build a network for around 50+ people and it needs to be able to grow with time. All I know is how to build over kill home setups. I know little about enterprise gear out side of my University classes where I did a lot with old Cisco gear. and my own work with Ubiquity edge routers.

It’s not a big net work just need reliable wifi, storage, and firewall. Right now I have been left with:

  • Netgear R8500
  • Dlink SMart Switch
  • 2XAP Trendnet TEW 821 DAP
  • 2 HP printer
  • 35 direct connections to the switch

The wifi drops people randomly, and I do have a budget for new gear if needed. The big thing I need right now is stable wifi. I did a heat map and the coverage is good. I also scanned the bands and the wifi sits and the same channels as everything else around. So I was going to move them so there’s less over lap with the other wifi networks in the area.

Any suggestions on what I should do to make the best network posable?

This one’s a little harder to identify, but consider the number of user to antenna ratio. Consider that only 1 device can use an access point’s antenna at a time, at the packet level. I wouldn’t expect there to be a huge problem if there are multiple access points for 50 users, but this can entirely depend on the lay out. If most of those 50 users are talking to an access point with 2 antennae, you could encounter network issues.

My advice:

  • If you have a lot of wireless clients, focus on the wireless. It may be worth while getting the business to pony up for a Ubiquity setup, if that’s what you’re familiar with. At first glance, it looks like you’ve got a couple of nice access points that are tied into an enthusiast access point. I believe Ubiquity would handle the handoff to other access points better. But you may know more about that than I.
  • You have a lot of users for 1 admin, but you’re still a small business. In this case (and so goddamned many other cases) smart switches are stupid. It’s nice to be able to log into them for troubleshooting purposes, but by and large you just want straight, dumb switching. If you start getting into QoS and VLANs and other crap for 50 users, you’re setting yourself up for an network that has more management than is actually worth while. It’s just more stuff to break.

You confirmed what I was thinking. I was just looking in the network case and we have a weird case where we have a cloud router? provided by our ISP that I’m supposed to hook up, by they want me to route all local traffic through the local not work. So I’m thinking to keep most of the existing stuff and swap the router and switch out for a for Ubiquity Unify gate way and switch.

If it were up to me, I’d be considering either Ubiquiti or Mikrotik (leaning towards Mikrotik) for WiFi.

For switching - possibly edgeswitch lite 48 to start with, a pair would be nice.

A pair of pfSense VMs (on separate physical boxes) for routing seems to be the default choice (I’d go with Linux but that’s just me).

Also, 50 people in a work setting is big enough to start thinking about radius (using free radius) and 802.1x (both wired on Ubiquiti and wireless).

You’re likely going to need to have a way to organize documentation (cloud is the easy mode there), and admin scripts (gitea).

You’re going to need off-site backups, and procedures for testing restores, and a plan for testing if restores work.

Once you have those things in place, network and services on it can more or less easily grow from 50->1000… bearing in mind that new local services will need extra effort.

OK, lets say I go with a setup like what you said. What would be a good router? I’m thinking an Ubiquiti Networks ERPRO-8, or the Gateway pro? Maby a Cisco one but I’m waiting for my rep to get back to me.

Neither.

Ideally you’d have 10Gig links from your switch-(es) to your servers hosting your pfSense and other stuff, and an ISP VLAN for your internet that you can extend tagged to your servers.

For virtualization, either ESXi or proxmox would work at your scale.

IMHO, Cisco is still to expensive at your scale. You’d typically start by introducing their switches before you start using their routers and firewalls. You can probably serve 50 people + their guests with just a pair of
regular 48 port L2/L2+ switches, as long as their WiFi is decent and you do decent QoS via pfSense to maintain low latencies they’ll all be happy. I’d definitely go with 802.1x ,and then I’d start thinking about redundant connectivity for admin before thinking about investing money in a high end Cisco or juniper router.

Thanks for the information. I’m thinking of this setup now:

  • Edge Router Infinity
  • EdgeSwitch 48port X 2
  • Connected with SFP at 10Gbps
  • Cat6 1Gbps to all clients
  • Unify AP AC HD X 2

Software

  • PFsence Firewall
  • LibreNMS for monitoring

I’m still thinking of an offsite backup option.
And in working on upgrading the storage in the future but not right now.

Don’t get the edgerouter infinity , use pfSense instead.
(Save $1300 for later)

Don’t get two uap ac hd, get 4x or 5x uap ac pro (or lite, and then swap them out entirely). For same amount of money (or less) you’ll get better wifi.

If you don’t need to power too many desk phones, or cameras, get the edgeswitch es-48-lite (save $250 a piece).

The 1500 remaining budget you can keep as contingency or put towards storage… storage tends to usually be more expensive than people anticipate.

5 Likes

Just want to drop in and agree with @risk. Even for a novice like myself with not much experience, Pfsense + cheaper Unifi AP’s works really well. While my network isn’t nearly as big, the flexibility of both pfsense and unifi AP’s is great and they’ve got lots of support. The newer versions of the AC lite support proper PoE as well so no need to use the silly ubiquiti PoE variant and their silly adapters anymore. Just some food for thought

1 Like

@WilliamY Sounds like you have a pretty solid plan here. Excellent advice from @risk concerning the setup. Just have a couple questions that myself and throw in my 2 cents for what it’s worth.

Will you be on-site or working for this business directly? I come from a MSP background and I ask because pfSense is a great solution, when virtualized though it makes it difficult for the end user to powercycle equipment. Granted in a perfect world, you wouldn’t have to worry about this, but just a thought. If you are on-site, this is a non-issue. Perhaps pfSense bare metal?

Do you know what kind of budget you have and if that can be devoted to storage or other infrastructure needs? Depending on the size of the dlink switch you already have, I would buy one new switch instead of two, having the switch you already have as a backup. As the network stands now, you don’t need all the ports and if you get a switch with SFP 10Gb then you can always buy another down the line. I don’t know what you are running in terms of subnets or VLANs, so that could change the picture completely. Put the money saved toward other needs.

Again, just my 2 cents and based on the crap I run into all the time in the field.

The only stipulation I would give to going PFSense, is don’t put it on a VM.

Best practice puts your firewall on physical hardware, not on a VM.

Also, what speed connection do you have to your ISP?

2 Likes

Speed is 100Mbps Up/down

Yes, I work on site for the company. We don’t have an IT department or person other than me. Which I’m transitioning into the Netadmin roll. The budget is unknown but we deal with a lot of expensive stuff so I would guess around $10,000 max or around there.

I assume this is your WAN speed? If it’s the old switch speed, then toss it and get 2 new ones. Guess I’m just confused by what you are quoting.

This is a matter of preference for me, but I like to keep my routers and firewalls physical. If you need to update the host or reboot or it fails, then network and internet access should still be ok. Could have physical equipment and a VM backup if you are worried about cost? If you do a physical pfsense box, this would be fairly easy.

Just some back of the napkin layout and designs.

If you connection to the internet is only Bi Directional 100Mbit, then getting a couple of PFSense machines is going to be more than enough for the job.

Depending on how many people and how much data is being stored/pulled from your Storage system, gigabit should be fine. But 10G would be better.

if you only have 33 people in the office, I would suggest getting a pair of 48Port switches. One for the office, the other to act as backup and to house your servers, and Firewalls. If you can get a switch with a 10G backbone you could link them together via the 10G uplinks, and connect your storage server to a 10G link.

If you need to segment your office, a managed switch is more ideal with VLAN support.

agreed it’s best practice, but then again, I’m thinking about the OP in this particular situation, convenience should probably contribute more to his security than device/network isolation.

With OP currently being in over his head (IMHO, no offence intended) . I’m thinking the inside of his network is as safe as the outside right now.

Most likely they’re probably running wpa2-psk and staff would happily plug in a random flash drive to their laptops and connect phones to the same wifi and vlan where there’s a machine that’s not all patched up with a windows or samba fileshare and everyone’s work laptop has more than once been used at a nearest starbucks.

Document, deploy, document again and iterate, …

Any Suggestions on what hardware to use for pfsense firewall. Should I repurpose a computer for it or go with a Pfsense router by negating or another brand? I’m leaning works a Netgate but they seem to be sold out everywhere.

I wouldn’t get a netgate or the PFSense store version.

This is basically the same equipment.

further details on this comparison.

2 Likes

What is your business need for the network? What content are you shuffling around? What kind of performance do you need?

Like you say, it’s not a home network you’re building. Here are some buzzwords used in IT, that may be helpful when planning the network (they’re really aimed more at service development/SLA, but I get alot of help from them with whatever):

People - Who are the users and who will be resources in setting this up? How will the users adapt to any visible changes and is any training required?
Products - What equipment and technology do we need to complete the task?
Processes - (This ties in with ITIL stuff, but basically documenting, evaluating how it relates to and impacts other things in the organization, and planning the work so that it’s easy to fit into existing processes and to manage)
Partners - What external partners and suppliers are required or affected? Is there any reliance on a third party supplier?

Utility aspect:

  • Performance - what can you do to gain maximum performance, while…
  • Constraints - trying to minimize or mitigate limiting factors.

Warranty: (in your case, “reliability and robustness”)

  • Security - This one you should get plenty of tips from others. What I’d like to advise on, is making an IT Security Policy early on.
  • Continuity - planning for failures and minimizing downtime, but also looking at future needs (from a business/strategic standpoint).
  • Capacity - What are your needs? How do you scale if need increases? Map out bottlenecks and potential future tuning needs contra costs
  • Availability - Make your own mind based on the video. Closely tied in with continuity above.

Finally, you should use these words when going through the keywords: Value - Outcome - Costs - Risks
Try hard to think of them from your business’s standpoint, and not your own, technical one. Use the five W’s for each : )


I’m basing my advice on the following quote:

Start off by checking with your boss/whoever gave you the assignment how much he’s actually willing to put out, and what he’s really expecting the result (outcome/value) to be. You don’t want to under or over shoot too much with the first proposal - try instead to make your first estimate on his budget (even if it’s just on a hunch on his part), and then lay out any risks you see with that budget, as well as proposals for improvement (from a business perspective)
You won’t give him what he wants, if you’re not speaking the same language. (and yes, that means you have to adapt to his level of knowledge and insight if he’s the boss)

$10k is alot of money for 50 people, including work hours to set things up, unless you also need to install new cabling. You should begin by creating an inventory of the assets you have today, how they’re all connected, and if they can be reused.

Do you have an SBS or AD server in the office today? Those have alot of uses you can offload from the rest of the IT infrastructure (such as authentication and log/file storage), and may let you instead focus on building a stable and expandable network.

Depending on what you do at you office, your requirements will different capacity. If you shuffle around alot of big files, having a sling topology for the switches, or stacking them if they’re in the same cabinet, you can have a single/dual 10 GE link(s) between them, and let most traffic go over IP in the office. Connections to the router can then be 1GE (or less) without sacrificing much performance during file transfers in such a scenario.

I would advise against building a pfsense router, and instead going with something more ready-made. If you are transitioning into an IT management role and don’t have years of experience, building an entire network from scratch will be extremely time-consuming and unfulfilling because of all the gaps in knowledge, if you also have to manage configuration in detail. 10 GE routers are not cheap though, but using hosts and having the router primarily work as a gateway/firewall (all equipment connected via underlying managed switches), you can definitely make do with 1 GE, or even FastEthernet on the router, internally. Remember that there are direct-attach cables if you don’t have large distances between the switches (alot cheaper than optical transceivers).

For $10k, you could consider taking in a consultant design and configure the network. A decent prestudy shouldn’t take more than a couple of hours, and would give you alot of valuable information to build on - maybe enough to build the network on your own.

I would definitely consider taking in some expert advise (paid consultant), so you don’t make any changes that threaten your business.

My final advise is based on IT security: Be careful of what you share about the project outside the organization. Yes, that means even here. Try to scrub any descriptive questions from identifying your employer and your environment, so you don’t put yourself in a legal dilemma.

1 Like

50 users
100Mbps
Needs reliable WiFi

You’re fucked. Production network over wireless is bullshit. And that’s only the beginning.
I’d suggest changing the scope of the operation rather than trying to make this nightmare a reality.