[Devember2021] Forum Server

Hi my name is Dawid, I’ve been a fan of Level1 news for quite some time so I decided to participate in devember this year. Although I started working on the devember project about a week ago, I haven’t gotten much of it done. I’m doing cs degree right now and I have a lot of projects due by the end of November which will take priority (eventually :slight_smile: ).

There is a niche in light-weight forum software where not many options exist. The go-to opensource forum is Discourse, but that’s 3MB of javascript bloat just to load up the page. Many alternatives that are more lightweight often have issues like

  • Being very old
  • In-extensible
  • Lacking modern features or conveniences

There may be some great software out there that I haven’t seen, but honestly, I don’t care. I want to make this anyway for my personal use. I’m not a great programmer by any means, but I think I can make a somewhat competent forum server.

Features

What I would like to implement

  • Easily extensible static forum. Basically a backend that injects forum posts and comments into requested html files. Like a server-side static side generator that renders pages with new posts. See ycombinator’s hacker news.
  • Create threads
  • Post comments on threads
  • Search threads by category
  • Login and register ( no email, that would take too long to get right)
  • Login with github

Nice to have

  • Management console. I am awful at react (and any kind of frontend work) so this is just something I would like to have.

Nice to have, but very unlikely to finish

  • Enable multiple backend instances to write to a single “writable” instance and have all the others be read only.

Of course, the ultimate goal is to get something done. I would like to get a tangible final product that I could use in production. I’m going to need to setup a forum around January, so if all goes well I’ll be able to use my own software.

  • The backend is written in go
  • Mangement console will be a react single page app
  • Postgresql for persistent storage.

Link to github repo: GitHub - Dawid33/forum

3 Likes

Progress Update 0:

I’ve decided on the general functionality of the backend. It will server files from a directory like a regular http server. The frontend web developer will need to create a list of html files for whom requests will need to be handled directly by the backend. These files have tags in them with special attributes that the backend parses and uses to inject data into them. The following is an example:

If you visit the homepage directly, the backend will look for tags in index.html with certain attributes called “query” or “file”. If there’s a file attribute, it expects another attribute that denotes a filename. The html file with that name is then copied into the tag. That was fairly simple to do.

The query attribute will take in many different parameters that I will need to write documentation for. An example is as follows :

<ol query type="threads" category="default" threadid title userid threadid >
	<li class="post">
		<a href="thread?id=%s"><span>%s</span></a>
		<br/>
		<span class="post-subtext">400 points by %s 2 hours age | hide | <a href="thread?id=%s">198 comments</a></span>
	</li>
</ol>

This queries all threads from the default category. It uses the content in the div to generate html for each thread. The attributes with no values ( aka equal to an empty string ) denote in which order to sub in data into the tag, like a printf statement. In fact the go code for this is literally the Sprintf function that subs in the values in place of each %s. All the information that is queried is retrieved from a postgres database. The add comment button misleadingly adds new post to the database with a POST request, which is redirected into a GET.

What it look like right now with garbage test data :

I don’t know if using custom attributes like this is a good or bad idea, but I guess I’ll see with time. I’ll need to put in some error checking to make sure none of the attributes collide with official attributes.

Making this for the comments will be basically the same process. What will be difficult is adding an option for nested comments using this system. That’s probably what Ill be working on next week.

P.S Yes the website looks like hacker news, so what :slight_smile:

Marvel at my CSS skills… that I copied from random CSS example.
This is as good as its gonna get for a long long time.

I don’t want the total file size to exceed 50KB. 100 KB would be an arbitrary limit I’d like to stay under.
I will setup some kind of pagination that will work with the HTML attribute system.

If someone is looking for a light weight forum software wouldnt they care more about a highly efficient backend that does not soak gigabytes of RAM? To you know safe some bucks on hosting.

Rather than an excessively low frontend target of 50kb. By the looks of it you are using react as far as I can fell that alone is above 100kb or not?

But idk maybe it will be great for some.

If someone is looking for a light weight forum software wouldnt they care more about a highly efficient backend that does not soak gigabytes of RAM? To you know safe some bucks on hosting.

Yes you are absolutely right, I certainly don’t want a backend that uses up huge amounts of memory. I will implement caching of certain pages that will update with every database query. I’m not anywhere near that point yet. I’m not worried about performance either, I just started the project after all.

The website needs to lightweight for the user first and foremost, the backend’s performance is secondary ( for now at least)

Rather than an excessively low frontend target of 50kb. By the looks of it you are using react as far as I can fell that alone is above 100kb or not?

I’m certainly not using react for the forum itself, I’ll use it for the internal management console which will only be accessible to admins/moderators. I doubt I will get any of it done though, its not a priority.

The purpose of this is that when you request a web page, all the data you will want to view is already on it. You don’t need to load any part of the page with javascript. This means that any part of the forum can be viewed with no js, which is an important feature. The 100KB limit for html files is arbitrary, I just want to keep the page size small. Realistically It could be more, but that would be “less cool”.

Thanks of the reply
Dawid

Yeah, I guess it´s good to try. I´ve never been particularly worried about it to that extend. For stuff like SEO, deep linking and previewing content of links, SPAs are kind of meh. I get that. But performance wise I don´t think it is a gigantic problem today.

There has certainly been some developement into that direction I´ve noticed. Frameworks like https://svelte.dev/ that try to kind of get you “the best of both”. As in provide ways to precompile and work without any JS at all, but at the same time give you the ability to still do dynamic content like you would have in something like vue. Never looked that much into it yet. Maybe will at some point.

I see, svelte looks really nice. Their approach seems different to mine though, as far as I can tell. I haven’t looked into these thing too much either, honestly I just wanna have some fun writing code.

But performance wise I don´t think it is a gigantic problem today.

You’re most likely right, after all most of the web today is filled with chunky webapps. They must be doing something right. That said, optimizing something for the sake of awesome optimization can be great, like https://esbuild.github.io/ . As far as I know esbuild started out as a passion project and continues to be one to this day. Besides, not everyone has gigabit internet, just this page takes me 1 - 1.5 seconds to load.

I will take a look at svelte later on though, see if I can rip off their ideas :slight_smile:

Markdown support in comment section. The box to the right is a live preview of the output. Like on discourse. Obviously very basic right now.

This website is extremely insecure. SQL injection, XSS, file system traversal you name it, the website is probably vulnerable to it. That will take some time to iron out haha.

if you were to restructure the names of your CSS output / layout, to use the same names / structure as those used on the L1techs forum, simplifed of coarse (because thats one of your targets), then later on others can easily drop in off-the-shelf custom CSS - I think the L1tech forum software is called Discorse (but dont quote me on that - I think its mentioned somewhere, cant find the copyright atm)

EDIT: its called Discourse according to the webpage source

At least that way you can get away with not having to worry about actual designing a themes convention, even if you use a heavily reduced set of definitions

EDIT: alternatively there are plenty of Wordpress themes