My project for this Devember is CivSim, a colony builder / management type game.
This is a project that I have done some experiments with in the past to test some ideas on, but never really went any farther with it. The overall goal was to have something similar to Dwarf Fortress but with Gnomoria’s UI/UX. I chose to use this Devember to re-visit this project, get a fresh start, and hopefully come up with a decent working proof of concept by the end of December.
This is obviously a large undertaking so would like to establish some goals to shoot for. Something that is a challenge but also attainable by the end of December.
Goals:
Procedural World Generation
at least 1 Citizen for the Colony, with basic needs (Food / Water)
Basic Job System (Mining, Hauling, Foraging, Farming, Workshops)
Basic Crafting System
I decided to do this as a sort of Vlog, so have been recording each time I have been working on it. Below is a link for the 0.0001% that want to watch that.
Most of the initial work has been focused on the World Generation. It is very basic in its current state, but is putting out some decent results. Will eventually need to add some additional steps to make it generate something more interesting. For the time being it at least gives some data to start working with to feed to the other systems.
The remainder of the work was getting the Graphical and Input systems in place. Giving the ability to visualize the generated data and to start being able to interact with it.
Question.
Are you rendering all blocks even if they are invisible? I don’t think it would be difficult to write a simple code for rendering only the top most visible layer instead of all the blocks.
Currently all the layers above the current layer are set to shadows only. This is more of a design choice than anything since it does look better. On this small test map (4 (8 x 8 tiles) x 4 (8 x 8 tiles)) per layer, this does not cause a noticeable performance hit. If later on when I scale the map up, and if it starts impacting performance, it can be toggled completely off.
Everything below the current layer is basically being rendered. There are some optimizations being made to the meshes, but overall it is more costly to build and re-build the meshes as the layers are scrolled through. Unity3D’s rendering engine is fairly smart, it does Occlusion Culling and Frustum Culling automatically behind the scenes, so most of that stuff is not getting drawn anyways.
Most of this period was used to start implementing the Job system. Currently can assign Jobs to Mine, Dig Hole, Remove Floor, Build Wall, Build Floor, Build Ramp, Remove Ramp, Ramp Up, and Ramp Down. The basic logic is in place for the Jobs to be worked by a citizen of the colony. For now the Jobs will be “worked” automatically and sequentially each frame to continue moving along.
Expanded the World / Region systems to have ramps, which will allow for future path finding to traverse between layers.
Some basic UI elements had to be added to be able to select a Job to issue.
Added some additional functionality to the Input system to allow for dragging with the mouse. Can use the middle mouse button to move the camera around. Can use the left mouse button to drag out a selection region to assign Jobs.
Added a selection indicator to the Graphical system to show where Jobs are being assigned and whether it is a valid tile to do so. Also added ramp meshes for displaying ramps.
Looks great! I do have some comments about the mesh-generation though…
It looks like you’re not doing any sort of face-merging(e.g. combining a flat area into a single quad). You should be doing that though, it will drastically improve performance with larger worlds. If you’re not already, a simple block-culling algorithm also wouldn’t hurt, and is super simple to implement(Only add a block to the mesh if at least one side is “uncovered”).
It looks like you’re not doing any sort of face-merging
That is correct, it is on the TODO list though. Currently trying to avoid too many early optimizations as things are still changing quite a bit. The Graphical system is separated enough from everything that it should be fairly easy to tweak and optimize later on without bothering the other systems.
block-culling
This is something that is being done to a certain extent. I am using real time lighting, so the top and bottom face are always “uncovered”, otherwise lighting will not get blocked correctly. The side faces are not getting built when covered by a neighbor tile. As I mentioned above, Unity3D does a lot of behind the scenes optimizations for the drawing cycle, so chunks and layers that are completely covered are not getting drawn anyways.
I am still tossing around the idea of ditching the built in real time lighting and going with a vertex lit system. This would save a bunch of draw calls and frame time for dealing with the shadows as well as being able to further reduce mesh sizes. The down side would be dealing with dynamic light sources moving around would require more re-builds of the meshes.
Spent this period working on the Path Finding system and getting the Entity / Citizen classes started.
Been wanting to create a Path Finding module for several years now. Typically I have just written path finding specifically for whatever task that needed it. This was a good opportunity to finally get a generic path finding module built that I can reuse in other projects. Has a basic Graph class that allows setting up nodes and edges, and a weighted Graph for edges that have varying costs to move across. Implements Breadth First Search for doing standard flood fill type searches, Dijkstra for weighted graph searches, and A* for getting paths to a specific goal.
Created a base Entity class that will be used for Citizens and eventually Enemies / Animals as well. It will contain all the standard things that an entity will require, such as moving around, needs / desires, inventory, etc. The Citizen class will have all the functionality needed specifically for citizens, such as current job.
Stopped using the automatic job working code and now have the Citizen work jobs. When the citizen is idle it will check if there are any jobs pending. If there are jobs pending then it will do a Dijkstra search to find the closest one and assign the job to the citizen. It will then find an A* path to the job and start moving to the location. Upon arrival it will do some work until complete and then start over again if there are more jobs pending.
This period was spent working on getting Items implemented. This required some additional classes and some re-working of how things currently behaved.
First was an ObjectMaterial class. This class allows for defining materials that everything in the world will be made of and the properties of that material. Example: Dirt, Sand, Clay, Stone, Copper, Steel…
Next was an ItemType class. This class allows for defining what an Item is. For now there are two ItemTypes, a ‘Clump’ and a ‘Chunk’.
Finally getting to the actual Item class, it will be composed of one or more combinations of ItemType and ObjectMaterial. Example: Dirt Clump, is an Item that has an ItemType of ‘Clump’ and an ObjectMaterial of ‘Dirt’. Stone Chunk, is an Item that has an ItemType of ‘Chunk’ and an ObjectMaterial of ‘Stone’.
The terrain of the world is now made of Items. Dirt, Sand, and Snow are of ItemType ‘Clump’ and Stone is of ItemType ‘Chunk’. When the Terrain is removed through a Job it will produce the Item(s) it is made of on the ground.
Next step was to start having Jobs that require Items to be completed, such as building a Floor, Wall, or Ramp. Right now the default and only option is to build Soil Terrain, which is terrain that uses any ‘Clump’ Item. So, for those Jobs to be worked, the Citizen will go gather the required Items and deliver them back to the Job until all requirements are met before working the Job.
I am likely going to re-work the Item system a bit since I am not 100% happy with the way it is currently set up, need to re-think the the data structure and methods for handling them.
This period was spent working on the UI and refining the Item related classes.
Decided to start with getting the Build UI in place since it will need to interact with the Item system. Figured it would help me see what would be needed for the Item classes.
The Build UI will check the selected Job TileType and fill the left side with the available options. After selecting an option from the left, the right side will fill in with requirements. The user can then select specific Items or just select the default ‘Any’ option which will use any item that matches the ItemType requirement. The Job will now be assigned to make the TileType with the specific Item(s).
Added a Tile Info UI to be able to look at the Tile data during runtime. Currently shows the Tile position, Bottom and Top TileType, and any Items that are currently on the ground.
Needed to add some additional things to the Item class to accommodate Items made up of other Items and the ability for Jobs to reserve Items. Also added some additional helper functions dealing with Items.
To test the Item reserving process I added some additional Citizens. The reservation of the Items seems to work correctly. the Citizens have issues with the Region changing though. If a Citizen is using a Tile that is getting removed or no longer accessible it can lock them up and/or cause strange fun things to happen.
The next few days will likely be spent on a long over due, clean up / organizing / commenting, before going any farther.
The majority of this period was spent cleaning up and organizing things. The remaining time was spent getting other systems started and in place for the next push forward.
Noticed a few spots were things were clearly broken, but by random dumb luck still worked as intended, so got those fixed to work correctly.
Got some more UI work done. Starting to get more place holders in place for future functionality.
Started to implement a time system. Previously things were just running each frame. Now a Tick event occurs where everything is calculated and updated. The amount of time for a Tick event to occur is variable, so this allowed for a speed toggle. There is now a Pause, Play, 2x, and 4x speed option.
Also started working on the Plant class, which will be used for Trees, wild vegetation, farming, etc.
Began getting the Citizen ‘Needs’ logic in place. They currently have stats for Hunger, Thirst, and Fatigue. Those stats will increase each Tick by a small amount. Moving or working a Job will also increase those stats by a greater amount. Currently when they reach a threshold it just triggers a dummy function that resets the stat back to 0.
Officially going into Devember now, so time to really push for those remaining goals!
Spent this period working on Plants, starting on Furniture, and Citizen needs.
Started off with getting the remaining logic in place for Plants to work and be displayed. There are currently Pine and Oak Trees, as well as Strawberry and Cotton Plants. After that it opened up some additional Jobs that could be performed. I added those new Job types into a Agriculture section. Can now Fell Trees, Cut Clippings, Forage, and Plant Seeds/Clippings. The Planting new seeds and clippings is not fully implemented yet. Will be adding some more logic to Plants later on to account for growth stages and also for setting up Farm plots which will do the planting and harvesting automatically.
Wanted to get Furniture started so can have all the basic Citizen needs accounted for by adding in a Bed. Currently I added this as a simple build option to build a bed, the generated Job will search for any Log and any Fiber and build a Bed. Later, when Workshops get implemented, it will be a more involved process of crafting a bed frame and a mattress, then crafting into a complete bed to be stored or placed somewhere. Also at the moment there is no occupancy to the Bed so Citizens will just get into a big pile on the Bed to sleep.
With some berries as a food source, water on the map, and beds, was able to start adding in the Citizens needs more completely. Each tick the Citizens will gain more Thirst, Hunger, and Fatigue until they reach a threshold and will then proceed to search for an Item to reduce that stat. For sleeping, the Citizen will try to find a Bed to sleep in, if it can not find one or it takes too long (Fatigue gets too high), they will just pass out on the ground. The benefit of sleeping in a Bed is that Fatigue will reduce by a much larger amount per a tick. Eventually food, drinks, and sleeping surfaces will also affect happiness.
Also added in a notification system for the Citizens to indicate when they are thirsty, hungry, traveling to a bed, and sleeping. It will display an icon above them to show what stat needs to be met.
Spent this period adding more to the Plant class and also getting Designations started.
Added more to the Plant class to start handling growth stages. The Plants will currently just go from a SEED state to a HARVEST state after 500 game ticks, but the logic is in place to go from SEED, SEEDLING, SPROUT, MATURE, PRE_HARVEST, HARVEST, CLIPPED, and DEAD. The Plant Seed Job will now work and a Citizen will get a seed and go plant it. The Forage and Cut Clipping Jobs will now only work on plants in the HARVEST state. The Fell Trees job will now work on Trees with a state of at least MATURE.
Started to get the Designation system in place which will allow for designating area as Stockpiles, Farms, Dormitories, Personal Quarters, Dining Halls, Guard / Patrol areas, etc. First thing added was Farm since everything was already in place to allow for it. Can now designate an area for a Farm and it will generate jobs to plant seeds when a Tile has nothing growing in it, and generate a jobs to forage when ready to be harvested, then starting the cycle over again to be replanted. Currently the Farm will default to use Strawberry Seeds, will need to make an UI for managing the Designations next.
Spent this period working on the Designation system and UI for them.
First thing wanted to get in place was an UI for the Farm designation. There is currently a space for setting the Name, selecting what Plant to grow, stats of the grow plots, and options for turning various tasks on and off. The toggles for managing the tasks include Foraging, Cutting Clippings, Felling Trees, and a general Suspend toggle.
Next was adding in another Designation, the Stockpile. This was a little more involved since needed to add in an ability to separate the ground items from stockpile items and still have the items be associated with the Tile they are in. Also needed to add in some additional logic for when Citizens search for items to also look in Stockpiles and not just the ground.
Spent a little time improving the Tile Info UI as well. Mostly just making it look better, but also will allow for clicking things in the info box to go to that objects management screen. For now this only works with Designations.
Also wanted to find a way to use the Citizen Happiness, so made it a speed modifier for when they move. Currently just (HAPPINESS - 50) * 0.002, so it will either add or subtract 0.1 from their speed of 0.2.
Just a quick recap of goals since there is only about 1 week of Devember left.
Procedural World Generation - Can still be improved.
at least 1 Citizen for the Colony, with basic needs (Food / Water / Happiness / Sleep)
Basic Job System ( Mining, Hauling, Foraging, Farming, Workshops) Basic Crafting System
Ended up crossing out basic crafting system since I have no idea what the thought process behind that was. The crafting will essentially be done in workshops so that was a bit redundant.
Last thing to do is get Workshops in place and hopefully have enough time left to throw a good spit shine on it.
Used this opportunity to take a good hard look at the issues that I have been leaving in my wake up to this point. The Item and Object Material system was never really where I wanted it to be and just kept working with it the way it was to make it work. So wanted to build the Workshop class to start working Correctly™ in regards to the Item and Object Material systems. Started by thinking about how it will be loaded in later on rather than hard coded in and that sorted out a few issues. The eventual structure of the WorkshopType is cleaner than the previous ItemTypes and I should be able to apply a similar logic going backwards to help clean things up in the other classes.
The Workshops were built to be separate to the Job system, and they produce a new type called a Task. Overall the Job and Task systems are very similar, but the Task is unique to the Workshop and allows for some dedicated logic to be done that the Job system would not allow easily.
Added in the ability to build a Crude Workshop, currently it uses any log and any clump to be built. This will be updated later to allow selecting a specific material for the ItemType and also to use a chunk (stone) rather than a clump (dirt).
The Workshop management UI has a list of available Tasks that can be issued. Selecting one will show the required ItemTypes and allow for selecting a specific Object Material for those types. Can Add the Task to the pending Task queue. Any Citizen that does not have anything else to do will find an empty Workshop with pending Tasks. They will then gather the required Items for the Task and then produce the Item. Currently the produced Item does not get placed since the 2 news Items do not currently exist. The other elements on the management UI are just place holders for now and will need to be implemented.
With the end of Devember at hand, can say that the goals are complete for the most part. I will continue to work on this and likely will be focusing on cleaning things up for the immediate future and attempt to get a basic demo posted in the next week or 2.
The Video Blog portion will likely be transitioned into a weekly progress update rather than directly documenting progress.
Put together a quick page on my site to host the project. Put the current version on there with a Linux, Windows, and Mac build of the game. (Was only able to test on Linux).
Gee golly, been 5 months already since last update.
Been tinkering around and pushing forward with this project when I have had time. Been mostly cleaning up, optimizing, and generally despaghettifying things. As a result there is really not too much to show off, mainly the reason for a lack of updates up to this point.
The biggest thing that has changed on the back end of things has been an almost complete re-write of how the ObjectMaterials, Item/ItemTypes, and TileTypes work. I had already reworked them twice during devember, but they still had some issues working with each other efficiently. Since these are really the most basic elements of how the world works I needed to make sure they were fixed before moving too much farther with anything. Think I have finally got them close to where they need to be, but still tinkering around with them to make sure got all situations handled properly.
The other major thing to be getting reworked was the lighting system. I had mentioned fairly early on in devember that the lighting system should probably be done with per vertex lighting instead. At that point it would have been a major delay towards reaching the end goal, so left it that way for the time being. The system was relying on real time lights/shadows to make the tunnels and covered areas dark. For the most part it worked, but it added a very large overhead of additional draw calls per a frame including non visible chunks since they still needed to cast shadows.
Now with the per vertex lighting system the amount of draw calls has been reduced significantly. The main down side with this method is with dynamic lights since they will require rebuilding the mesh data for areas where a light has changed. Essentially it was a trade off of reduced draw calls for increase in rebuild calls. Overall it is still more efficient and personally I like the way it looks.
Hopefully will be able to put out a new updated demo sometime in the next month or 2. For now, here is a video of the the new lighting system in action.
Procedural World Generation - 
Can still be improved.
Basic Job System (
