My router is currently two floors down from my main rig. I have an ethernet cable fished through the walls up to the top floor of my house. let me tell you that was a nightmare to achieve. I don't want to do that again. So my question is, is it possible to run two computers off of that one Ethernet cable I ran upstairs? I don't think a splitter would work as that would cause lag issues( I think)
Is this what a network switch is for in this situation?
Yeah, you can use a switch for that. If you really want to go cheap on it and you don't need gigabit you can get something like this http://www.ebay.com.au/itm/2pcs-RJ45-Splitter-1to2-Network-Ethernet-Connecter-Adapter-/140571003917?pt=AU_CablesConnectors&hash=item20baaf4c0d on both ends and connect two computers, with 100mbps only 4 wires are used instead of the 8 that gigabit uses, so you can use both sets of 4 wires. But considering you should be able to pick up a switch pretty cheap that would be the best option.
Years ago I was right where are now, four workstations in my office and only one Ethernet cable running to the room. Granted this was before the widespread availability of gigabyte switches, but you get the idea. A 10/100 switch should suite your needs beautifully.
Your question was answered but I figured I might add some hardware options.
Since you probably don't need anything ridiculous, you can get a nice 5 port gigabit TP-Link or netgear switch for under £20. Gigabit might even be overkill but there super cheap already so why not.
Keep in mind that 10/100 Mbit ethernet only uses 4 wires, so a splitter can connect two devices using 4 wires each, but beyond 100Mbit (also known as Gigabyte ethernet) you need a switch. I would go for switch for futureproofing and for connecting more than two devices.
If your ISP provides > 100Mbits, then switch is pretty much the only option.