Server network segmentation

I wanted to redesign our local server network which at the moment is just a normal /24 network. Due to more and more people working on the servers itself i want to limit the access in the internal network itself by segmentation and firewall rules and acl´s.
Does anybody has expierence with this kind of work and can give me some advice maybe?
At the moment I would just split them up by the appropiate groupe for e.g. Exchange and AD server in one segment, the Database server in one segement and so on. Is their maybe a better way or solution?