Hello world 2:The Electric boogaloo …and now we draw the rest of the owl
Recipe Website Preamble:
Problem Statement: a group messaging that is alt discord(text)(I don’t know who they are going to sell to) and me and the boi’s want a place that is off a big platform, but there’s a problem I don’t want to explain what IRC is and matrix seems ify, plus I am software engineer thingy so why don’t I just make my own.
This is really just a study into kube and go with external given public deadline for that pride hit if failure ensues.
This will be more of a Infra and less code, but if I get the infra a good spot(It never will be) I can get some good feature creep going.
Minimum Viable Product
“Users can login and see and post message to the group”
Registered User can login
Registered User can Logout
Registered User can Edit User Information
Registered User Once logged in
Can See List of "Servers" they are IN
Can "Enter" "Servers" they are part of
Once in "Server" They can post messages attached to their User
Read Message History of the "Server"
“Time Allowing”
Invite Users via Email
Invited Users can register
Have "Server" "Admin" group
Can Invite New Users to platform
Add register Users that are part of that server to Admin Group
Can Edit Information about the "Server"
Messages Have Timestamps
TLDR; Text Discord, but different, but really just a disguised study into Kube and go
Okay I have gave up on due this with just kube, and getting some help with from Helm.
I am switching from linode domains handling to cloudflare cause I can’t get the DNS01 to work correctly with the web hook function of cert-manager. There is this Linode Community Post from sub one year. Two personal from that post give two chart Git OneGit Two that try and deal with this. I can’t get them to work for some reason or another(I assume I don’t wtf am doing the more I work on this). Cloudflare is a supported DNS01 provider of cert-manager. That transfer is in progress now so i just have that working when that is done.
[85%] helm keycloak/postgres
[55%] ingress with tls
[-20%] actual development
Some Finds along the way
Lens a k8 “ide” it just a kube dashboad with some nice options, but I have run into an issue with custom definitions
Oliver Coding Blog so found this blog post that outlines at lot of what I am trying to do for the most part
Note: the keycloak chart values are little out of date. I will push mine once I get them closer finalized
Thoughts
should have some more time to work on this as this month winds down due to things.
For some reason I really don’t enjoy Helm, Id rather just work with k8 yaml no charts. I know I properly wrong, but just find helm annoying for some reason.
I had limited interactions with SMTP really only inputting provided cred to use one that was already set up for some bot related work professionally to integrate with an API
Now working through all the keycloak features when I see this.
But something i did not plan for, and email sever idk why. It would used for users to validate an email. Granted this i not one of the item i have listed as necessary goals, but non the less I will waste only today and try and figure how to add some SMTP to my cluster.
So I go to looking, and i find some open source ones
I find two that the internet seems to like: IRedMail & Postal
IRedMail
doesn’t seem to be production ready or at least the docker version.
Seems to be more ready for containerization and gives a nod to linode, but when I search on linode for any info on it there isn’t any it seems. Regardless I think will try with this seems a lil more ready plus I already have one redhat item going.
Okay after spending some time looking around at the state of the postal. I have found a few items.
Postal Github and it creators and maintainer are making updates and so on, and are focused on containerization.
They don’t have a public docker hub from what I can tell, which is odd but I see why so you have to host the image your self if you want use stock version.
There is a helm version of this that is of this that was created about 2ish years ago by a Linkyard with consideration to cert-manger and ingress options which is good.
There is an issue tho with it being 2ish years old the kube api objects are out of date(apiVersion: apps/v1beta1) which can updated.
Also I am not positive but looks like readiness probes wern’t a thing at that time so initcontainers busybox containers are used to wait for the two required backend to be spun up. There also a few other things if left in would bother me so if I do end up going with would require a good amount of changes
There also seems to be another option to get this working and its a customized docker version of postal with some spam and anti virius running on alpine instead of using the given Ubuntu. Github tiredofit/docker-postal
Its in docker and on the hub which is nice save me some time plus it comes with some addon option out the box
yeah that’s is good amount of kube objects and also the those pvc are all in different volumes which i guess would work, but that seems wrong. I had to look into this while writing this,(5 mins later). It seems you can do both so hmm
So what the plan for SMTP
I am going to use both linkyard/postal-kubernetes & tiredofit/docker-postal I am going to mash them together into something that can work on kube. They will just plan kube yaml for no helm. Once I get to a place with all these object I may consider making it into working helm but maybe
I have started on this I got the all the env variables files in config Mappings and respective secrets in to separate files (50+)
There is still a good amount of work to do on converting this into something that works well.
Edit
I will not be doing this Postal is kinda of mess.
The tiredofit/docker-postal uses his own custom alpine image which is eh, but I tired to remove the s6 thing in that wont stop the container even it fails from what I can tell and even it do build its hug for alpine idk
but that image will failed then I check the git hub issue board 40+ open issue. I do think i could messaround it get it work with the base image, but have spent about a day on this. SOoooooo
Preamble: I have play around with keycloak before but I really don’t understand all the in’s and out. I want to learn more about it so I know all the options and constants when planning out the user structures and how the roles would interact with data. I also need some info how plan out the api’s I need to create to interact with it.
Okay I found a book on keycloak that came out this year. It happen to be on O’reilly so that subscription is coming in handy
Why is a book needed, cause the documentation on how to best operate with keycloak is some limited info this I was able to stumble through to get some interaction with the keycloak but know that could be better.
so the book has some in good info and explains a decent amount more than the basic information . The book has an associated github with some tools for working with openid to get a better understanding of this standard(and the other) Link to Openid Playground
This book seem pretty good for what I need have really only one or two hiccups. Looks like the keycloak used for this book is 3 major versions behind even tho it was released this year. It seems v11 was release July 2020(current release is 15). so eh.
There is a golang section which was cool, but it just the standard req/response handlers so I will be looking to gorilla options for this after this
ch8
read time about 8 hours I am not the fasted reader plus I did most of the excises
Post Book time
Okay I did like this book for the most part(or at least for parts I found relevant which was 75%). It did good job in comparison to what is available on the web on how keycloak works and how to get it do what you want. It did go over some design points which i found useful in addition different approaches to problems with some realism through in(i.e don’t make your own Oauth framework). The goal of reading this book was get a better understanding of what I can and need to plan for when designing this app which it did. There is also some relevant information on how to just do some things I had a hard time figuring out my self. tldr book 7/10 would read if getting started with the keycloak.
Unrelated
I am getting back into this after sometime due to getting promoted to customer,social event, and a bender should be able to work on this full time starting 12/6. I am not sure when this contest endings(I assume end of year), but I will be continuing afterwards anyway.
Reply
@paulwratt Yeah i did see that I’m not sure how long that will take to get that thur there CSRs so i am going to fore go it for now, and put in it for it later, cause I’d like to have any emails be hosted by myself(aka Linode) than use of the paid options. I was looking at there TOS for some paid email providers and they are a lil spooky.
Next Item(s)
I am working on plan and design of the app it self now. I got the ol pen and paper out trying to stop my self from making things to complex.
So I am going to let keycloak act as centralized authorization server (crazy i know, but there are other options.) so this will allow me the most flexibility when creating roles and access, Cause I know I am going to change it a good number of time during this initial creation phase.
I am just shooting for the MVP right now( I’ve had to stop my self like 4 times already for trying to plan for the future)
So that means
One “Sever”(Discord term(actually in the docs its called a guild))
One message channel
Only one type of User (So can read message history and post)
At least to start i am just going to have these groups
I am going to lean on keycloak to handle what user have access to what servers(discord term) for now I am not postive this best way, but speed is the name of the game right now I need to get some Proof of Concept completed sooner than later.
Just some basic test users with assigned happy path roles
its noon 30. work to cont.
Openid Connect Framework
Okay the book and from last post did have some example code for using go-oidc but it did not have PKCE which seems very important. so i need to find one that does.
There seems to be two options that le google throws back
I will do the oidc integration with the code later
For now I am starting on the code for the the application now. I have started with some basic apis and middle ware for the server data type. Just gonna start basic stuff and I will build outward. I am going to integrate the keycloak SSO later. I am having trouble getting it work due to lack of understanding on my part( I have already lost like three days on that, I just need to move forward)
By the time you read this should be on github for this(Github Link).
Note: I did remove the infra yaml from the hub for now. I put it back later once I have the sites project structure more thought out. I know I do want everything in one repo
Short term Action items
Services
[ ] Get J SON validation integrated
[ ] Get Channels data structure complete
[ ] Get Channels Service Completed
[ ] Get Message data structure complete
[ ] Get Message service Completed
Frontend
[ ] Flesh out the UI I made the other day in NextJS with bare minimum servers,channels, and message items (just need to display what server/channel you are in plus the associated messages)
[ ] Connect the frontend and backend and wiggle them together for now
Containerization
[ ] Containerize the services
[ ] Server
[ ] Channels
[ ] Messages
[ ] Containerize Next JS frontend
[ ] Thrown them on the cluster
[ ] Add them to the ha proxy
[ ] Check the freaking DNS
Longer Term Action Items
[ ] Open API ( Swagger documentation)
I think i mite do this sooner than later I think It would help me with doing validation plus code gen for the client for the frontend
[ ] Integrate oidc (some how)
[ ] Add a database (currently using in code data store)
So I have the basic REST crud operations for the for the server service. For the Create, Update, and delete aren’t necessary for the MVP, but I thought since i was here and for practice. For the future I will have to the create, delete, and update interface with keycloak to add/remove/update the necessary roles, but that is for a later time I really just need the GET right now and will remove them from the client.
Client → Runner → Mux → Handler → Validation → Server CRUD function
Open 2.0 (Swagger) Meta + Docs
So looking at what needed to be done I needed a client for the frontend so I figure I’d do two things at one documentation and get some of that yummy code gen. so I did that for this primary start.
$ server-api/handlers/docs.go
// Package handlers classification of Server API
//
// Documentation for Server API
//
// Schemes: http
// BasePath: /
// Version: 0.0.1
//
// Consumes:
// - application/json
//
// services:
// - application/json
//
// swagger:meta
package handlers
import "server-api/data"
// Generic error message returned as a string
// swagger:response errorResponse
type errorResponseWrapper struct {
// Description of the error
// in: body Body GenericError
...
Serve both the Redocs and the swagger on the base directory
Don’t think I’ll keep the yaml on forever but I think it will help with code gen once it gets on the cluster mite make it easier.
Next.js Frontend
Alright I am getting into frontend and straight up out of my element.
So I followed the Next.js quick-start guide. I have some experience with js but with bots so eh. I followed the guide to get two pages up and still need to table or something for the get call for the table.
This package I am going to try and use for next js code gen (Next Swagger doc) needs the json instead of yaml changed the make and runner to do that and host that file with the yaml. Quick addation just have the go runner server that file and have go swagger create a json version in the make file.
Switch to typescript project type for Next js Frontend
For rest of the 12th I just was trying to learn how to best use Next js and the service I got the codegen to gen some code, but need to learn how Next js wants things. God I am bad at the frontend thing. went with openapi-codegen-typescript cause I just wasn’t all the positive witch one to use in swagger editor.
After many hours I have come the understanding that I have no understanding lol.
Ditching Next.js
I thought Next js would be easier, but I was really struggling with it for some reason It just wasn’t clicking for some reason. SO we are going to react. There is more and better documentation for it( yes I know next js has react, let’s just chalk it up to me being dumb and move on)
Gonna also try out this AXIOS which seems to be a lil simpler than the openapi-codegen-typescript and the swagger editor massive file.
Not sure if the updates where I really don’t do anything but fail and learn are useful, but I figure they mite be to someone. Cause I only knew a few of these things beforehand, but professionally I only work with java and services this full stack thing has bloody large amount of moving parts and most them I’ve never work with.
211214
Went with the Just the React and Axios, and I was able to get that to work, but the frontend is gonna need more work as time goes on, but I have completed my goal of wiring the front and back end together. The only change needed to the back end was to enable CORS with gorilla toolkit so that was straight forward
[x] Connect the frontend and backend and wiggle them together for now (Ongoing)
Longer Term Action Items
[x] Open API ( Swagger documentation) (Ongoing)
…
There all on going as add more objects they will needed to added, but there is a pattern I can follow moving forward so the big brain thinking can be used later
Onward to Databases
So i have( bad one) link to the frontend so let try and get the preliminary step to get the backend(database up). SO have pick a type wither relational or document style, but I will see that go ORM support and move from there.
So I do have some worries about doing these integrations before data is near finalization which would leave to rework, but i think standing it up before i start can lead me to making better decisions on what i’ll need for the data, plus data structure is never finalized.
Note I plan to turn on ssl mode(link) on later when this gets on the cluster
Using GORM for database
Documentation for this and the package it self makes this rather easy. It is extra dependency, but for now its find. I like ORMs but I understand why some don’t and agree for the most part
Automigration is complete for the server struct
.ENV files and being smooth brain
Took me a few (*cough hours) to get the env files to work ending up using the Viper package. Works rather well. There are some random files that need to be cleaned up from some other packages trial and error, but I will make a push by the time this goes up so if you want take a gander
Will cont to work on filling out rest of the CRUD and project files directory layout will be the next items. After those items I will get to start getting to the channels once that
Get all Servers which will probably be the most used and really the only one that will needed the most at this time, there others are there for future use.
This was stuff I have done a lot of time in other langues, nothing special. I wont do an update like for the rest of the api, but i figure it best to do one.
Channels
Started the work for adding the channels struct to the Server Struct and figuring out the mux, validation,gorm, logging, and error handling when dealing with a nest rest object.
So In my head its straight forward, cause adding the channel obj to the Server is straight forward but integrating with the existing middleware, and mux is throwing me for a loop.
0415
Native Postman runs like Sh*t
the native app runs like trash it eats up RAM and CPU. Not really related to this project, but I though it was interesting that is not just a me problem I notice while leaving it open. Link to Postman Performace Issues (Github)
I am running short on time for this, because at a minimum I have the following things to complete to meet the MVP. Be Straight up I am not going to be able to finish the MVP with amount of time I have left. I will try for the first two just to see a message on the frontend.
Even after #devemeber2021 is over, I’d still be interested in ongoing updates, even if it ends at “functionally complete” and not “damn this is a fine piece of coding and design”.
Yeah, the fails are also noteworthy, and will help some other people in how they feel about their own failures, especially if its with the same tool or framework. Maybe even one of their devs will come across the post and think “hey maybe we should have a look at this from a noob perspective”, and fix it - meh, you can always live in hope (or phantasyland).
.env: this is one of the most common web probes I see in my firewall endevours - I protect this file (non-exhistant on my setup) by passing back a 4.5Gb DVD ISO image. For fav.ico I 404 it unless there is a HTTP_REFERER.
Anyway, keep up the hard work, it would be nice to see this completed.
GORM leads it self to putting you in a circular import loop by design and I have come to learn that from trying to add this temp user struct due to them being many to many
Which doesn’t make to much since to me for some reason
Autonomous structure passing
Which mite work but not positive and seems kinda hacky
Mess with the project structure some more.
SO I am going to move all the things into there packages(channels, servers,messages, and users)
Does this fix the problem Nope, but I will also stop calling the object it self and just call ID of the associated item.( I already played with this in the Channel)
I did know about circular imports, but just didn’t think about it until I had had already done it. The GORM documentation did help, but its still on me but here:
Okay here(above) do you see it? They are in the same file. I just filtered that out in my head.
Thought I just share that. I would like to move away from GORM at some point cause it’s not the fastest performance wise, but until I get closer to a finalized data model i am going to lean on it to save some time
220101
Now What
So I am going to try and break down each of the services that I have created to SQL schemes/Queries with the necessary item so that don’t have go into the database for any reason.
Looking at that code you 'll see items that lock me into postgres,(pgx , & “information_schema.table”) which removes one of the advantages of using pure SQL and that it can be database Agnostic. I may end hating this later, but whatever send it.
I am trying find a way to not have to maintain both the model for json and the sql schema together in the same struct so when I need to make changes or additions i can in one place, but there doesn’t seem to be one that can do that with references to other tables
Here is sqlx, but I still have to maintain the schema separate,
GORM did that, but doesn’t seem to a viable option for micro services or at least at my understanding for what I want to do with.
Next things
Reading another Book
If I am going to do the microservices things I guess I should read their bible things. Cause I think I missing some information on this topic. Going to Cherry pick some chapters that seem relevant to see if any insight can be gained
With this I have exactly what I want, but I also have know what I want so a double edge sword.
This is not what I going with, but show some things, but is still very wrong
Red: UUID this was easier than I thought to get working, but I will need the extension to be enabled for each database it used in. Just another thing to error check for and implement.
Blue: Postgres allows for arrays, but after reading up on them they do append when updating they update the whole array which is a waste, so I think going with inter-between table for many-to-many at least.
Yellow: just being able to do a reference. I think have be able to alter a table with running a query while still using GORM, but doing so makes me just not want to use GORM or any ORM(go-pg did the same thing).
#devemeber2021
So I failed, but I will continue to work on this item for some time, but I need to find a new job so that may take of available brain availability
I will keep going with this. It was always a study first and it lined up with #devemeber2021. Granted I was hoping to get farther than getting to temporary user data model to implement messages and realizing my database interface conflicted with microservices aspirations.
I have no fantasies that anyone would come to this for lessons learned lol, but it helped me get my ideas in some form organization.
For the .env on the back end I will remove write access or if I am really worried about I will pass them as a argument and keep them in a kube secret
For the .env on the Frontend I will look into your SSFW, but Frontend seems so far at this point.
there is not much in SSFW that can offer you a solution. The best I can do is to say, pull up the “known_urls.txt” an search for “.env” to show the sorts of locations are probed.
EDIT: it should be possible to 404 any HTTP attempt to get /.env via webserver configuration, while still being available to the web app (because its a local service/process accessing it)
As for the rest of your project, even your 1st couple of posts pointed to it being (possibly) hard to complete.
On the Postgres lock-in, there should be a solution to that, I cant say what that solution might be, but my previous experience says its possible, but it might be one of those things where you need to allocate some time to “go down a deep dark hole” (so to speak).
Anyway, I think you did alot, learned alot, and exposed alot (to us) - all well worthwhile.
(sorry, meant November)
