Sorry for the very late reply, but yes, it would use https:// and it would show as secure in your browser. I’m pretty sure your router doesn’t have that functionality built in, but newer routers should. Just be sure to double check before buying a router because not all consumer routers will have that.
I’m not sure what the process is for making this happen on newer routers (my router doesn’t support https), but it could be as easy as checking a box, or it might be a bit more involved, like generating, signing and importing certificates.