I think I already know that this isn't possible, however I need to picks someone brain on this. I watched the pfsense video ages ago and since then I have been itching to build my own custom router. The only thing I can't find and its probably because they dont exist is a pcie NIC card that uses a cable input instead of Ethernet. As my provider uses DOCSIS® 3 tech(https://en.wikipedia.org/wiki/DOCSIS) from the cabinet to my apartment.
I know I can place the provided router into modem mode and then use Ethernet to the new router.
Actually this question brought me to the idea of trying this kind of thing myself. But I have the advantage of a VDSL connection and I already found a card that could work with that. If I go for it I will post it in the forum and leave a link here.
You'll want to make sure that A) pfsense has the drivers for it and B) it's possible to configure it in pfsense (should be if it's pppoe or something like that).
Actually I found it listed just now. I'll buy the damn thing and try to make it work. I have my network setup perfectly running and I have a system i can try stuff on separately. So it is just an experiment. But there are not a whole lot of solutions like this out there so my guess is that it will be supported fairly quickly.
The card I am looking at is the Draytek VigorNIC 132 by the way.
Well you said a home made router (which is possible), but you really meant a home made modem + router combo right? You can still have a custom router if you want (which I recommend) since the provider provided combos are all backdoored! (not kidding) So it's advisable to just use the modem with your own router (and a home made one is best!)
Use hostapd with a wifi adapter compatible with AP mode, and a dhcp server to assign IPs to your clients! :) Simple! iptables to route traffic! Bam! :D
I agree with everyone else just use the modem and have the custom box be the roiter/firewall. Also it is a lot easier to have a separate WiFi access point as well. PFsense's wireless drivers can be flakey.