It’s the prepared commands are known ahead of time, then why not just skip the sqlite step and store a JSON file on the server, and when they ask for commands it just responds with the JSON file, or store as a JSON structure on the server backend? That way you don’t need the overhead of connecting to a database.
My question is: what’s the point of the database?
As this is a chat bot for Twitch, it has to be very customizable and allow for data to be created from the user, such as commands.
For example, if the bot is running in a streamer’s chatroom, they can type ‘!addcom !test This is a test.’ This adds a new command to the database.
Using a database, and a smal one such as Sqlite3 made much more sense than using JSON knowing that new data would be added often.
TL;DR only some commands are prepared ahead of time.
The bot also has a points system. At every x interval determined by the streamers, it ‘rewards’ all viewers in a given chatroom with points as a form of community building/engagement. This is done by using Twitch’s API. Using a database seemed far more effective for this use case then using JSON, in fact, that sounds horrible to me.
Hope that makes sense.
Actually, is this case i prolly would’ve just used a .env file with the key-value pairs for custom things, otherwise provide a default set of stuff, stored as an object in memory on the server.
Would be even easier to tinker with since there’s no database dependency. Only downside I can think of is that it would need to be restated every time a new command was added to the env file.
But hey, if you’ve got something that works by all means stick with it. Just thought I’d give my $0.02 
Interesting: would you use.env files for the use case of a points system as well?
Now that I recall, another reason I picked SQLite over JSON was dealing with a situation when the bot crashes. If the bot crashes while writing a JSON file, it could cause serious issues, whereas crashing with a DB would be more recoverable and less damaging.
Nah, I’d probably just store that in a JSON file on the server as well. Although it would be crude, but its not like its doing anything special besides keeping track.
Has the bot actually crashed when trying something like that or are you just being overly paranoid?
If you have plans to do a rich sort of features which require a database down the line then yeah use that.
Like for example, I’m making a forum software so I need a database
So If I was you, and using Nodejs, I’d just read the JSON file in when I start my server, and create a function that would accept an object key-value pair, and then when I need to make changes:
Keep in mind this is for someone that is running this bot on a VPS or local server.
If you wanted HA or load distribution, then it would be best to use database stored somewhere else and just have multiple ‘workers’ hit the database.
Sorry for the all posts!
No, it’s been very stable with how I use the bot. So I would say I’m being paranoid.
I talked with some devs on the Twitch Developer Forums and they went with database solutions, but usually using something like Postgres. This was early on in my development.
I’m writing this with the idea that non-technical users would use the bot, so I’d rather avoid data loss even if it should never happen.
The overhead hasn’t been an issue so far: after I implemented transactions properly, it’s been able to distribute points for tens of thousands of viewers very fast even on a slow HDD.
But I see where you’re coming from. I’d have to rewrite a lot of my bot to not use the DB, but I’ll reconsider on future projects.
No worries on all the posts it’s great and super helpful, thank you!
P.S: Here’s an actual payload example:
[{"commandname":"!Kappa","commandresponse":"4Head","commandpermission":""},{"commandname":"!pepega","commandresponse":"","commandpermission":""},{"commandname":"!pepepga","commandresponse":"Yooooo LUL","commandpermission":""},{"commandname":"yooo","commandresponse":"4Head","commandpermission":""},{"commandname":"!4Head","commandresponse":"Kappa","commandpermission":""},{"commandname":"!XD","commandresponse":"LUL","commandpermission":""}]
Most of what I’ve done today has been in the last few posts, but here’s a summary.
Now that I fixed my Golang API to send properly structured data, I’ve been attempting to bring in some GUI elements. For example, I’ve tried making a button to show / hide the data, but to no avail. I have a JS component that calls the API, and when my button calls it it’s not receiving the data properly, similar to my issue before.
I also tweaked my Golang solution for sending the proper API data, so doing some back-end stuff again was fun.
Next step is to continue working on basic GUI elements to interact with the API data.
Today felt fantastic.
I have decided to switch to React for my front-end framework. I really liked Mithril, but for a noobie like me, the amount of resources online was not working for me.
For whatever reason, the way React’s create-react-app package is structured has clicked with me.
I finally have a basic GUI working, below is a screenshot.
All of this data is gathered through the back-end API.
Next is to start implementing buttons to facilitate when commands, points etc are shown. After that, start manipulating data from the GUI?
P.S: I’ve pushed all of my changes to this point to the repo, here it is: https://github.com/murnux/HappyBot
Progress
Very happy to say that progress remained good today. I added the ability to see your current title and game, and to display the stream is offline when it is. (Because Twitch blocks you from getting title / game data when the stream is offline). This also has given me the opportunity to learn how to have multiple states per React component.
The other thing is a tabs system to clean up output and display one big piece of data from the bot at a time, below is what it looks like.
Very pleased so far.
Today i focused on writing POST handlers for the golang server and for user input on React that sends a post request to the server.
This has resulted in the functionality to create, delete, and edit the various data the chat bot uses such as commands.
So in React we have a form in HTML, when clicked will be handled by a function. Here’s a delete function:
handleDelete(event) {
event.preventDefault();
fetch("http://localhost:4000/api/delcom", {
method: "post",
mode: "no-cors",
headers: {
'Accept': 'application/json',
'Content-Type': 'application/json'
},
//make sure to serialize your JSON body
body: JSON.stringify({
commandname: this.state.comname,
})
})
.then( () => {
this.componentDidMount();
});
}
And now the GUI looks like this:
It actually works pretty well, but I do have a problem with it. Every time one of these delete / create functions are called, I’m ‘forcing’ the table to be updated which means it calls the Golang API again and again each time something is edited, created etc.
My issue with this is because, each time the API is called, the Golang programs eats up around 500 KB of memory each API call which adds up. I’m not sure if this is normal, or if something is wrong in my code.
Nevertheless, it continues to come along.
Hey all, not one of my more productive days. My mind wasn’t quite in it today.
What I did try to do was to figure out how to make React render the JSON as a key, value pair when the JSON property names are different for each item so it doesn’t have to be hard coded, but no luck so far.
I’m also having an issue where any numbers I input in a form are being sent to my Golang program and eventually the DB as the number 0, and never what I actually input. I’m finding no good resources online so I’m gonna be working on that tomorrow.
I also added a nice animation for the tabs to make it look a little less harsh.
Hey all, today went pretty well. I’ve done a bit of a remake of the GUI (and I’m sure I’ll do more in the future), the bottom screenshot shows where I’m at. Honestly, I’m not yet sure what I want the GUI to look like yet, but it feels better than it used to.
Right now I’m facing a design issue: whenever someone makes a new command or any such item from the GUI for the bot, how do I make the bot itself (the Golang bit) to update its data on demand? Most of this data is in the form of a map, so if I just set it on a timer, the memory usage could rack up fast.
Something to think about!
Some important fixes
Today was successful for both the back-end and front-end parts of the bot.
On the back-end, I’ve fixed a long-standing memory usage issue that I haven’t figured out for quite a while. It wasn’t a total memory leak, but each API call would increase the memory usage without the OS ever releasing it. Same happened when I reassigned the value of hashmaps, increasing even when the length was exactly the same.
Turns out it was me being silly and putting a bunch of database open connections to SQLite3, and they were never closing properly. Now they close properly, and I made the database variable a global one so the entire program has access.
On the front-end, the home page now lets you update your twitch title and current game. Obviously it calls the Twitch api to accomplish this.
As seen on the nav bar, I’m starting to add all I wanted in regards to creating, editing and deleting database data without the need of sqlitebrowser.
Tomorrow will be focused on thinking of content for the home page that may be useful among other things.
On another note, I went ahead and did a production build of the golang back-end and the react front-end. It’s all running under 10 MB of ram, so that’s pretty awesome.
Hey all, mostly front-end work today. I’ve been tried to get a vertical nav bar working within react, which I’ve finally done today, so that’s the biggest change you’ll see.
One thing you can’t see is that each page renders with a nice little fade animation, makes it easier on the eyes.
My biggest question is the color scheme I want to go with. I love my dark themes so that’s probably what I’ll go with, but not sure if I like the blue / grey look. I think the blue is fine, but need to think about the grey elements.
I think the dark blue/grey looks nice. 
Maybe a bit low contrast between the dark blue and dark grey. If you used the grey border on the left too it probably would look more consistent, but it depends on what you want to do.
Nothing major new today, just some touch ups to the GUI. I’ve messed around with the background navigation menu color pictured at the end.
The biggest thing was implementing searchable and paginated tables, handy for when data from the DB gets very large. Sadly, using this table breaks the fade animation I had before. I suppose this is what happens when you use different third-party modules together.
Tomorrow will probably be similar, in addition to adding functionality to edit current data instead of just creating / deleting them.
Edit: Thought i’d share, I hit a error on the back-end I’m all too familiar with: index out of range. Because so much of this program is based around string splitting (for the sake of parsing user messages), the bot may expect multiple words with spaces in between when they don’t, so the bot panics and crashes.
This just happened with one of the mini games, roulette to be exact. A user types in !roulette 50 to bet 50 points. But when I typed in just !roulette the bot crashed, because it was expecting a second element. All it takes is a simple check on the length of the array: if len(optionsSplit) < 2 { BotSendMsg("@" + username + ", please bet your points. Example: !roulette 50") }
Today was mostly back end work. I’m adding edit functionality, so the biggest GUI changes were making the appropriate forms for that pictured below.
For the edit functionality, I had to figure out how to let people keep a certain value what they had below. So if they want to change a command !command, but wanna keep their response This is a response but change the permission, how can the user easily keep their previous value?
I thought that letting users keep the inputs blank for what they want to keep the same would be nice. So in in the golang code, when an item is edited, it goes through code like this:
if com.CommandName == "" {
for k := range coms {
if k == com.TargetCom {
newName = k
}
}
} else {
newName = com.CommandName
}
com.CommandName is checking whether that specific input field was empty, so the JSON would be an empty string and where ‘TargetCom’ is the command the user typed that they want to change. The newName variable is passed to the database. This same code is re-used for every column in the database.
Not sure if it’s the cleanest solution, but it does work.
Yeah, you just change the piece of code (key/value) you need. Although, I wouldn’t commit this file your repo’s. It’s usually untracked.
However, with .env files once you make a change you need to reload the program.
Think of .env files as CLI arguments, just that they are parsed in a different way that is more convenient, but simple to edit. The simplicity makes up for the lack of robustness.
