Dynamic_Gravity's Devember 2018 Project

Previously

Last year, I wanted to see what it was like to build a basic CRUD app with a MEVN was like.

This year, I’ve spent a lot of time continuing learning Vuejs, but I think its time to learn something a bit more focused on performance and a heightened mobile experience.

I’ve always had this dream to make my own forum software, so I’m going to try and make that happen. That’s also what my project last year was loosely based on. This project will borrow a lot of the things Discourse does well–more on that later–, and my goal is to do this mostly from the ground up.

That means, the only thing I am going to not do from scratch is the JavaScript framework and the icons. Everything else will be just me. No bloat here.

Contract

I, Dynamic_Gravity, will participate to the next Devember. My Devember will be to design and implement my own forum software. I promise I will program for my Devember for at least an hour, every day of the next December. I will also write a daily public devlog and will make the produced code publicly available on the internet. No matter what, I will keep my promise.

Architecture

Client

Resource Selected
JavaScript Framework Mithril
CSS Pure.css + Custom
Font(s) Raleway, System Default Fallback
Icons Fontawsome v5

Server

Resource Selected
Database MariaDB
Backend Nodejs(GQL or REST)

Deployment & Scalability

Strategies

  • Source
  • Docker (single, multiple)

Note: HA is not in current project scope.


Repo

7 Likes

Day 0.1

Phase: Planning


Decided to work on the potential database schema. I thought about what data I would absolutely need to record. I don’t want to get bogged down by other features so what you see below is what I would consider the bare-bones. This will likely change in the future but for now, this is my starting block. I did my best to make sure I was in third normal-form as well.

Overview

+-----------------+
| Tables_in_Forum |
+-----------------+
| Categories      |
| Replies         |
| Threads         |
| Users           |
+-----------------+

We have a total of four tables so far. A table to hold the different categories, users, threads, and replies.

Categories

+----------------------+--------------+------+-----+---------+----------------+
| Field                | Type         | Null | Key | Default | Extra          |
+----------------------+--------------+------+-----+---------+----------------+
| category_id          | int(11)      | NO   | PRI | NULL    | auto_increment |
| category_name        | varchar(255) | NO   |     | NULL    |                |
| category_description | varchar(255) | NO   |     | NULL    |                |
+----------------------+--------------+------+-----+---------+----------------+

Each Category will be uniquely identifiable by its category_id. The meaning full part that people will search by though is the category name. However, if a category name is every changed or removed it would break a lot of existing records and then each reference to that name would need to be manually altered to the new name. If the database gets hundreds of thousands of posts then that process could take hours for a simple change. This is a quality of life thing.

Users

+---------------+--------------+------+-----+-------------------+----------------+
| Field         | Type         | Null | Key | Default           | Extra          |
+---------------+--------------+------+-----+-------------------+----------------+
| user_id       | int(11)      | NO   | PRI | NULL              | auto_increment |
| user_handle   | varchar(255) | YES  |     | NULL              |                |
| user_password | varchar(255) | NO   |     | NULL              |                |
| user_signup   | datetime     | YES  |     | CURRENT_TIMESTAMP |                |
+---------------+--------------+------+-----+-------------------+----------------+

A user is pretty simple. All that’s really needed is a way to uniquely identify them and their desired handle plus salted & hashed password. The only other meaningful bit is to track when the user signed up for things to determine the age of the account and such.

Threads

+-----------------+--------------+------+-----+-------------------+----------------+
| Field           | Type         | Null | Key | Default           | Extra          |
+-----------------+--------------+------+-----+-------------------+----------------+
| thread_id       | int(11)      | NO   | PRI | NULL              | auto_increment |
| thread_created  | datetime     | YES  |     | CURRENT_TIMESTAMP |                |
| thread_title    | varchar(255) | NO   |     | NULL              |                |
| thread_author   | int(11)      | NO   | MUL | NULL              |                |
| thread_category | int(11)      | NO   | MUL | NULL              |                |
+-----------------+--------------+------+-----+-------------------+----------------+

A thread table only exists to link multiple users to multiple replies; referred to as an intermediary table. Since there is a many-to-many relationship. Many users can have many different replies in this table.

However, there is a one-to-many relationship between users to a thread. I.E a user can have many threads but each thread can only relate to one user.

Other meaningful bits are the thread title so users could search for something specific and when the thread was created for chronological searching.

Replies

+---------------+----------+------+-----+-------------------+----------------+
| Field         | Type     | Null | Key | Default           | Extra          |
+---------------+----------+------+-----+-------------------+----------------+
| reply_id      | int(11)  | NO   | PRI | NULL              | auto_increment |
| reply_body    | text     | NO   |     | NULL              |                |
| reply_author  | int(11)  | NO   | MUL | NULL              |                |
| reply_created | datetime | YES  |     | CURRENT_TIMESTAMP |                |
| reply_thread  | int(11)  | NO   | MUL | NULL              |                |
+---------------+----------+------+-----+-------------------+----------------+

Replies are interesting as they link back to both an author and a thread. This is a one-to-one relationship though.

A reply can only relate to a single user, in a single thread.

By design, a reply can be large. Up to 64K–over 65,535–characters. I figured this was a reasonable ceiling for any single post.


Flow

So after a new user joins, in a forum where the categories have been defined he will go to create a new thread. In the thread, he will define its title and its contents. Then thread will be created with the specified title, and the contents will become the first post in the newly created thread.


Source Code

I am working on cobbling a repository together, so, for now, the source will be provided below.

I am using MariaDB, so if you have that installed and want to check this out on your machine you can bootstrap the database with the source file below.

schema.sql
CREATE DATABASE IF NOT EXISTS Forum;

USE Forum;

CREATE TABLE IF NOT EXISTS Users (
  user_id INT NOT NULL PRIMARY KEY AUTO_INCREMENT,
  user_handle VARCHAR(255) DEFAULT NULL,
  user_password VARCHAR(255) NOT NULL,
  user_signup DATETIME DEFAULT NOW()
);

CREATE TABLE IF NOT EXISTS Categories (
  category_id INT NOT NULL PRIMARY KEY AUTO_INCREMENT,
  category_name VARCHAR(255) NOT NULL,
  category_description VARCHAR(255) NOT NULL
);

CREATE TABLE IF NOT EXISTS Threads (
  thread_id INT NOT NULL PRIMARY KEY AUTO_INCREMENT,
  thread_created DATETIME DEFAULT NOW(),
  thread_title VARCHAR(255) NOT NULL,
  thread_author INT NOT NULL,
  thread_category INT NOT NULL,
  CONSTRAINT `fk_thread_author`
    FOREIGN KEY (thread_author) REFERENCES Users (user_id)
    ON DELETE CASCADE
    ON UPDATE RESTRICT,
  CONSTRAINT `fk_thread_category`
    FOREIGN KEY (thread_category) REFERENCES Categories (category_id)
    ON DELETE CASCADE
    ON UPDATE RESTRICT
);

CREATE TABLE IF NOT EXISTS Replies (
  reply_id INT NOT NULL PRIMARY KEY AUTO_INCREMENT,
  reply_body TEXT NOT NULL,
  reply_author INT NOT NULL,
  reply_created DATETIME DEFAULT NOW(),
  reply_thread INT NOT NULL,
  CONSTRAINT `fk_reply_author`
    FOREIGN KEY (reply_author) REFERENCES Users (user_id)
    ON DELETE CASCADE
    ON UPDATE RESTRICT,
  CONSTRAINT `fk_reply_thread`
    FOREIGN KEY (reply_thread) REFERENCES Threads (thread_id)
    ON DELETE CASCADE
    ON UPDATE RESTRICT
);
2 Likes

Ambitious, admirable goal: :ballot_box_with_check:
Lightweight, performance based code: :ballot_box_with_check:
Not using React/REDUX: :ballot_box_with_check:
Database Normalization practices: :ballot_box_with_check:
AdminDev excited to follow along: :ballot_box_with_check: :ballot_box_with_check: :ballot_box_with_check:

2 Likes

Thanks! I’m excited as well. :slight_smile:

1 Like

How was your experience with Vue? I did little outside the basics, but I much preferred it to React.

It’s hard to write bad looking code in Vue.

Also, it feels like the cherry-picked features from angular and react. Template files written in html with sprinkled handlebar syntax just feel so much more natural to me.

And then the directives , similar to anglular ng-*, are a nice touch as well. So much easier IMO.

There’s there’s the optional Vuex, and Vue Router for code splitting and centralised data store (REDUX). So it can be lean or as full-featured as it needs to be. This is tremendous for scaling projects.

The CLI 3 tool doesn’t need you to eject to make any custom changes, and vue ui is fucking gorgeous.

1 Like

Just a note from my fuzzy memory…

Look into search indexing, I vaguely remember it effected the structure of my tables. That may be a performance factor when many users are looking up a single reply table.

2 Likes

I will definitely revisit, appreciate the in-depth response.

I went Vanilla -> Meteor -> Vue -> React -> HTML + CSS :sweat_smile:

Meteor was nice when I used it as well, but I felt like it tried to do too much, and I felt like I was fighting the framework to get it to do what I wanted. Haven’t felt that with Vue.

Regarding react though, I do feel like that has much more finely tuned state control though. So if you really need such control than it would be the better choice.

1 Like

Yeah, the replies table will definitely be the one that gets hammered the hardest of the bunch.

I will be looking into how to make it more performant when I need to cross that bridge. Thanks for your bits of wisdom. :slight_smile:

Damn, I seem to be having a bear of a time getting heroku to work.

Build Log


-----> Node.js app detected

       

-----> Creating runtime environment

       

       NPM_CONFIG_LOGLEVEL=error

       NODE_ENV=production

       NODE_MODULES_CACHE=true

       NODE_VERBOSE=false

       

-----> Installing binaries

       engines.node (package.json):  10.13.x

       engines.npm (package.json):   6.4.x

       

       Resolving node version 10.13.x...

       Downloading and installing node 10.13.0...

       Bootstrapping npm 6.4.x (replacing 6.4.1)...

       npm 6.4.x installed

       

-----> Restoring cache

       - node_modules

       

-----> Building dependencies

       Installing node modules (package.json + package-lock)

       added 2 packages from 2 contributors and audited 6657 packages in 5.23s

       found 0 vulnerabilities

       

       Running heroku-postbuild

       

       > [email protected] heroku-postbuild /tmp/build_8a2e1bce6a139533b1b0d7b16e71d931

       > npm run build

       

       

       > [email protected] build /tmp/build_8a2e1bce6a139533b1b0d7b16e71d931

       > webpack --mode=production

       

       Hash: c7479425d288149e1091

       Version: webpack 4.26.1

       Time: 1698ms

       Built at: 11/27/2018 10:37:14 PM

        Asset      Size  Chunks             Chunk Names

       app.js  28.3 KiB       0  [emitted]  main

       Entrypoint main = app.js

       [1] (webpack)/buildin/global.js 472 bytes {0} [built]

       [2] ./src/main.js 181 bytes {0} [built]

       [6] ./src/views/UserList.js 274 bytes {0} [built]

       [7] ./src/models/User.js 290 bytes {0} [built]

           + 4 hidden modules

       

-----> Caching build

       - node_modules

       

-----> Pruning devDependencies

       removed 433 packages and audited 156 packages in 4.613s

       found 0 vulnerabilities

       

       

-----> Build succeeded!

-----> Discovering process types

       Procfile declares types -> web

-----> Compressing...

       Done: 18.8M

-----> Launching...

       Released v8

       https://limitless-spire-88948.herokuapp.com/ deployed to Heroku


@AnotherDev or @SgtAwesomesauce think you could help me out?

What’s in here? Does it have app.js as the entry/launch point?

It launches server.js

Scroll up, I added my repo.

It’s strange because when I’, running it locally it works just fine. But once deployed it 404’s.

Is this through Heroku CLI or the web?

Getting on the train, brb will test it later.

The duck says /dist is not versioned

It is being deployed to heroku through gitlab.

I got that as well but it doesn’t help me.

I can see that webpack is building the assets and storing them in dist once I push a build though.

If I open up bash on the dyno I can see the files there.

I think I found the problem. Fucking CSP.

Fixed it.

It was a combination of CSP and me being a dummy. Works now.

2 Likes