[DEVEMBER 2021][COMPLETE] Loyalty Scheme in a Box for Small Business

I was asked how possible this was a year or so ago so I sketched out some requirements, but then the need went away. Decided it’d make an interesting venture for a Devember project (feel free to disagree :sweat_smile:)


Definition of Done:

  • UI for customers to log in and track their points
  • Admin view for business folks to see stats
  • Ability to add/subtract points
    • Thinking of exposing an API for this to enable plugins for e-commerce (shopify, square, etsy, etc)
  • Packer/Terraform to build and deploy given an API key (Linode to begin with)

Extras

  • Client/admin mobile app
  • Additional deployment targets (AWS, Azure, GCP, DO, Vultr)
  • E-commerce plugins
  • Some ability to print physical rewards cards for the technologically illiterate customer?

Technologies

  • Docker containers for local environments
  • Packer/BottleRocket/Ansible for immutable builds
  • Terraform for IAC deployments
  • Probably Vue for frontend because frontend is worstend :man_shrugging:
  • Golang backend (I’ve written something like this using a Python FastAPI backend before which is nice but I’d like to improve my Go as part of this (and Python is thicc))

This is also as yet un-named so if you’re feeling creative I’m taking suggestions. I can offer README credits and a virtual pat on the back as rewards.

3 Likes

Try FealTY. So far, you have good SEO, as it points towards MeriamWebster and Co. No name-alike brands yet.

Three syllables = best length

The TY at the end is for the “Thank You” for being loyal. Also helps with the spelling.

1 Like

Update 1:

  • Named, repo created
  • Created skeleton of the API backend using Fiber
  • Imported some old Mongo code to get the DB stuff off the ground quickly
  • Currently works at a very basic level, I like Fiber so far
    image
1 Like

Update 2:

Quick one, more to follow this afternoon.

Completed:

  • Get single account
  • Get all accounts
  • Create new account
  • HTML template to view all accounts in a browser
1 Like

Update 3:

Slightly stalled because I had to submit a patch to the best MongoDB GitHub Action so that they offer auth - Feature: Add --auth Functionality by biodrone · Pull Request #26 · supercharge/mongodb-github-action · GitHub
I used someone else’s fork in the meantime but apparently they’ll be creating a new release of this tomorrow so I should be able to switch back over properly soon.

In the meantime, I’ve completed:

  • Tests
  • CI pipeline (including auto semantic versioning based on commit titles)

Currently I’m using the HTML rendering capabilities of Fiber to provide a web-browsable frontend but I think that may fall down in terms of user friendliness when I add auth… Might try and figure out a way to render a login page, then have one api route with www-form login which that redirects to. Hopefully that engine will be able to handle that! :crossed_fingers:

1 Like

Update 4:

Been playing with Packer and Terraform all week, really an enjoyable part of the project. I knew that would be the case but that’s just because I like automating things :man_shrugging:
Currently, Packer is building the required image and storing it at Linode as a private image, then passing the image ID to Terraform, which brings that up. Arguably, Terraform is overkill for just bringing up an image, but it will allow scaling up of things like the DB node if load requires it.

Completed:

  • Packer and Terraform scaffolding, base configs, etc
  • Database node Packer and Terraform pipeline

1 Like

Update 5:

It’s ALIIIIIIVE :zombie: Been a while between updated but I’ve been busy writing more YAML than is advisable :sob:

Now builds images and deploys successfully entirely from GitHub actions CI/CD workflow. Particularly proud of the way that packer passes it’s manifest files to Terraform and then they’re parsed for the most recent image build. Also props to Linode’s Packer provider for giving images sensible names, this is way more of a pain with AWS AMIs!

Completed:

  • Finalised Packer and Terraform builds/deployments for backend and DB
  • Included a domain and A record configureable from env vars because it’s nice to be able to have that IP dynamically update with new builds
  • State is stored in Linode Object Storage (any S3 compatible backend will work) so the TF part of the pipeline is idempotent, packer will always rebuild images but there’s not much to be done about that
  • Traffic runs through a Linode VLAN for those sweet, sweet budget savings

Next up to add:

  • Look into a NodeBalancer for frontend traffic
  • Add a workflow with a delete button that can be manually run so that the infra can be deleted easily
1 Like

Update 6:

Mini update because I’m currently writing docs and needed a break so that my brain doesn’t go to sleep.

Finished the Infrastructure Destroy function, that is now just actionable whenever via the GitHub Actions workflow. Works pretty nicely imo :man_shrugging: Brings to mind that I need some sort of DB backup function, maybe just cloning it into Object Storage will work fine as a backup (2nd order, the Linodes themselves are backed up by default now)

Update 6.5:

Leyman docs now complete, proper technical API docs tbd (hopefully tomorrow :crossed_fingers: )

Update 7:

Decided to add API auth instead of docs because… reasons? Either way, basic auth is now done, will take an admin password from env and use a Mongo DB collection to hold additional users. Would like to use somethnig like Vault for this obvs but it’s not really within the initial scope of this project.

Update 8:

Okay so I decided that because docs are the worst, I’d do fun things instead. I can hear the script kiddies cursing me from many Russain basements but I think I’ll survive.

Anyway, I finished the Authentication and managed to use the Go html templating engine to render a super basic authenticated management UI which forces the browser to set a basic auth header so this works both with browser and API routes. Re-using the same auth mechanism for both without tonnes of extra code was nice :ok_hand:

Currently working on making the Update and Delete bits of the UI work (because they’re forms they have to have a different endpoint to send data to because x-www-form-urlencoded != json :man_shrugging: I might look at handling one content delivery mechanism being blank in the code at some point but for now this is the quickest way to properly finish this bit off.

I think this is technically finished now, the API side of things works, is deployable purely via GitHub Actions, etc, etc but I want to finish this bit before I’m happy with it.

Okay, we’re done!
:tada: v1 release is now live on GitHub :tada:

Latest changes:

  • plebian docs up to date
  • added a loadnodebalancer
  • Terraform now automatically grabs a cert from Let’sEncrypt
  • possiblity to specify multiple app nodes

High level overview of what the hell this wound up being:

Say you own a small business that sells some kinda product with a small footprint (here in the UK we have a bunch of markets with small traders who were in mind for this) and you want to reward your users for their purchases? Enter FealTY, the self-hosted rewards scheme where you control all of the data and don’t have to know any of that darned COBOL to set anything up. Create 2 accounts, set 4 variables and boom - your very own system.*

*GDPR regulations are not my responsiblity

The more technical breakdown is that this is:

  • a pretty simple Go app
  • a MongoDB backend (because familiarity)
  • a HTML frontend entirely rendered in the Fiber Golang HTML engine (one of the cooler things I played with in this project, learned quite a lot from that)
  • Packer to build all of the dependencies into an app image and a db image
  • Terraform to deploy both of the above images, as well as a Linode VLAN for inter-layer traffic
  • Terraform to also deploy supporting infra including
    • Nodebalancer for traffic distribution and SSL termination
    • Domain & subdomain with an A record rewards mapped to Nodebalancers IP
    • Object Storage to store Terraform’s state, making the whole project deployable (and destroyable, and re-deployable) from GitHub actions

Key Decisions

Language: Go
Reason: Want to learn more Go for system tooling, also it compilesa nice lil binary and you don’t need to build anything like a python interpreter to run it

Underlying Framework: Fiber
Reason: I’ve seen a lot of hype around this and having used raw Go Mux in the past I wanted to see what the benefits of this were. Conclusion: I like it a lot, would use again.

Platform: Linode
Reason: Because it’s the best, of course. And $100 of free play money can’t be overlooked, building images from pipelines gets expensive because of first hour billing :sweat_smile:
I specifically didn’t go with containers because the idea is kind of if you get bigger then you can scale the number of app nodes horizontally behind the loadbalancer which doesn’t really lend itself to a containised workload. The ideal scenario would be a K8s cluster in this day and age but when the selling point is that it’s cheaper than a $50pcm subscription to a SaaS platform, that’s hard to do for the price point…

Currently the admin UI is the main point that users would be controlling the user creation/search/modification of accounts from, although there is a basically self-documenting REST API detailed in the main.go file that the UI is taking advantage of.

The main point of this was to show off some cool Packer/Terraform build possibilities and how cool it can be to build immutable infra that someone else can make use of to deploy their own version :ok_hand:

Follow the GitHub README and go to https://rewards.$DOMAIN/api/v1/accounts/admin and enjoy the Web 1.0 goodness :joy:

The repo, just popping it here for posterity

2 Likes

I also forgot to mention a curious discovery I made whilst introducing firewalling to the app node. You have to have a public interface on the node, even though the nodebalancer uses a private IP to communicate with the node…
Hence why the node has a public interface and a firewall to only allow traffic on the relevant port, the app auth already protects that port.

1 Like