I’ve been kicking around this idea since Jan 2020. As the Linux world moves closer to using Wayland for desktops by default, there’s been little if anything done to address the need for Wayland-based thin clients (thin clients are still used in some university computer labs and the like).
The Wayland protocol is quite I/O agnostic, it’s just a buffer and message passing protocol. The plan is to create a split client/server which acts as a Wayland server on a remote machine (and can run another nested Wayland server in client mode), and acts as either a Wayland client on a local machine, or uses the kernel frame buffer. Everything will be built around a single remote-client assumption (simplifies the server half), and minimizing CPU resources on the client (it’s THIN). Bandwidth is not a major consideration as these generally run with dedicated 1gbps or better lines, but encoding the output is a possibility (lossless h.264 should work reasonably well a large portions of the frames are unlikely to change). The remote machine will provide the buffer for the remote Wayland client to draw.
Anyway, I think step 1 is writing out a vocabulary for the pieces in play, as it is a bit confusing. I’ll aim to upload a .dia file once I have that worked out.
Second will be naming the project something better than Wayland Thin Client.
Third is figuring out if wlroots or libwayland will work, or if starting from scratch will be easier.
Alright, I think my hours about up… Let me see if I can describe the topology in English well enough to avoid the diagram (which doesn’t really help…).
Wafer
Working name for the project, and for the client and server halves considered together.
Mainframe
A powerful multi-core machine with a GPU supporting the GBM protocol, and gigabit or better ethernet.
For testing, will be my ryzen 3800x machine.
Thin Client
A low powered linux machine with USB input and 1 display output, also with gigabit ethernet.
For testing, will be one or more Raspberry Pis.
Wayland Client Compositor
A wayland compositor (kde/gnome/weston) running on the Mainframe and outputting to the Thin Wayland Compositor.
Thin Wayland Compositor
The server half of the Wayland ThinClient. Runs on the Mainframe, exposes the Wayland Server protocol via a unix socket on Mainframe.
Expects a single Wayland Client Compositor
Thin Wayland Client
The client half of the Wayland ThinClient, runs on the Thin Client and outputs to either a kernel frame buffer, or to another Wayland Compositor.
Client Application
An actual application a user wants to run (firefox, libreoffice, et cetera).
This makes it rather obvious that I need a better name for the project, as differentiating between the Thin Wayland Client and the project name is… not obvious.
Sounds like a great project. I don’t understand all the complexities but I think you’ve done a good job laying out the components.
Might be bit on the nose but you could name it Wafer. It was the W from Wayland and the meaning of thin. You can break it down to Wafer Mainframe/Server and Wafer Client etc. for each component.
Not to mention it’s a “wafer thin” client I doubt anyone will suggest anything better than that, and lord knows I’ll not think of anything better (my projects tend to be named things like “Proxy Server” and so on).
Hey, we’re moving closer to heat death of the universe too .
Honestly, when your usage pattern fits with Wayland’s model, it’s pretty fricken awesome, it’s just the edge cases (which account for probably 75% of real-world use cases still).
I used to be hardcore wired about wayland. It was super cool in concept. Then I realized that designing a software stack specifically that nothing can connect to is kinda dumb?
At the same rate I want XtoGO but wayland for my servers.
That nothing can connect to without the compositor’s (and user’s) explicit permission is smart. FTFY.
It took far longer than it should have to get standard extensions protocols for screen sharing and input redirection, but now that the Wayland world has mostly resolved around KDE, Gnome, and WLRoots, it’s actually feasible to get extensions written and widely supported. And it’s nice when my compositor pops up a window saying “application X wants to do Y with Z”.
Alright, first status update, a little bit late (but when your three year old needs attention, what can you do ).
wlroots is not going to work, certainly not easily. There are problems, one annoying, one major.
The annoying problem is it’s written in C99, complete with array specifications that are invalid in C++. I could switch to using C99 (not my first choice, but doable), or I could translate or otherwise abstract away the offending headers (most of the wlroots headers are fine, I can probably skip the matrix-helpers). Wrapping the actual data structures with their associated functions into proper classes would be somewhat time consuming, there isn’t a solid pattern to them to let me hit the header with a macro and be done with it, but that’s not the end of the world.
The much larger problem is wlroots is fairly opinionated. When you call the initialization function wlr_backend_autocreate, it autoselect the appropriate backend, both which graphics card should supply buffers, and how the compositor output itself should be displayed. I could probably dig in to that function and duplicate the needed parts of it while not creating a compositor output, and there is an optional parameter to create your own renderer that might work, but it doesn’t look like it will actually make things easier for me.
Fortunately, someone else already had the idea of C++ wrappers around libwayland, called waylandpp, it’s a much thinner wrapper than wlroots, but given that I just need to proxy events to and read buffers from a remote compositor, that should be fine. Will give it a spin tomorrow.
Have you seen project Greenfield? It’s a “cloud desktop compositor,” as in, you just open a browser tab, go to a URL and your desktop is running away on some server or something. Maybe contribute to that project instead?
Way to try to take the wind out of someone’s sails. I have been following the Greenfield project since its first public release, but it is totally unsuitable for use as a Thin Client.
First, it requires running a web browser on the Thin Client. Web browsers are heavy, and no, making it into an electron app doesn’t solve that issue (just running a local X server or local Wayland compositor is a stretch on some of these machines).
Second, if you ever try actually using a GTK application with the broadway backend, you’ll find it doesn’t work well. Shortcuts don’t propagate properly, and that’s for a single application. Add in needing to switch between applications and good luck.
Ok, there are some good points made here. Indeed, browsers are heavy. And so could Xorg or a local Wayland compositor be considered. However, the test bench for the Thin Client itself
I have run Firefox on a RPi2 on JWM+Compton. It was a little slow starting it up and rendering webpages, but it was an ok experience, in absence of any other PC. So I believe running Greenfield on it would be doable. Even more so if using slightly less resource hungry browsers, like Midori or Falkon. It could probably even be doable on the original RPi model B. I don’t believe we will find Thin Clients much weaker than the original Pi (all Thin Clients I encountered were running Intel Atoms, but I can’t say I have a lot of experience with them).
The problem with running any program and finding it doesn’t work well revolves around 1 problem: Greenfield is pre-alpha software. It’s normal to find unexpected behavior.
Now that I mentioned “the bad part”, let’s move to the “good part.” I mentioned Greenfield in my first comment, because I wanted to attract attention to it. Fragmentation should happen as rarely as possible IMO. I think I kinda understand why you want to develop your own thing and I wish you good luck with it. I’m not trying to
Personally, I just find it wrong for programs to be running in JavaScript. So having an alternative written in C* would definitely be welcome. If it could also work or be adapted to work like X2Go, it would be even better. And if you manage to KISS, maybe development could move on faster than Greenfield (with some help from other programmers on L1F, wink, wink, nudge, nudge).
And I haven’t noticed it was a devember post. Was it tagged from the beginning? If so, sorry, didn’t see. Ryan said there will be some terrible people out here, you have been warned. Welcome to the forums, tho’! Genuinely hope you will stick around after devember, it’s a nice place for tech enthusiasts (there are also other developers here who find help and stuff here, one who comes to mind being Gnif, who made Looking Glass, a program which was featured a lot on the main Youtube channel). Just lurk deeper.
Hope you can make Wafer and make it good. Best of luck!
Digging deeper, I don’t think any of the existing solutions will check all the boxes for doing this easily. I’ve also been at this far too long tonight (when did it get to be 1:30AM?). I have several ideas on how to proceed, but I don’t think I can articulate them well enough at the moment. I’ll catch up on the thread and explain what I’m thinking once I’ve slept (with luck, I’ll get to it before the kiddo awakens in too few hours).
Falkon probably can run Greenfield (Midori is a little less certain, it’s html5 support is incomplete and Greenfield probably uses some semi-advanced html5 stuff). Still, compared to a purpose-built C/++ application, running a fairly memory hungry application to host your kinda memory hungry application is going to be worse.
I have a first gen (almost first batch even) Pi that I’ll test it with. If I can get it working on a ZeroW, that’s even better (I do have a Pi4 as well, but if it works on the original version it’ll work on anything).
I didn’t say the experience with Greenfield is bad, I said using a GTK3 application with Broadway is bad. Bad is too strong a term really, as it does exactly what it promises, and the Broadway implementation is mature. It’s just that, say I want to shift-right-click on something in the GTK3 application, my browser will interpret that as a force-context-menu override and display the stock right click menu. Maybe you can get around that using webgl full screen, but if you have multiple windows, you need to override Alt-Tab, which your local compositor is not going to allow. This problem is also present in RDP/VNC solutions, and is a limitation of the “access another computer virtually while still using my local desktop” paradigm. That doesn’t make these solutions bad, just not really suitable for the “I didn’t realize I was logged in to a remote machine” use-case.
I apologize that I implied poor intentions on your part, that was unjustified. I agree that unnecessary fragmentation often ends up with more projects languishing, but I think it’s necessary in this case. I could probably reuse the compositor implementation from Greenfield, as it’s in typescript and runs on nodejs. (And there’s the catch, given that a server is likely to have up to several dozen connected users, even with node’s JIT, that’s going to be resource hungry).
And yes, it was tagged devember from the start, I signed up for a forum account because it sounded like a fun thing to do. No worries though .
I didn’t even realize that Shift thing. Yeah, depending on you WM / DE, it would be a terrible experience. Some VNC clients have the “send key / key-combo” buttons, which break the immersion, but make the inputs work.
I completely agree with making an immersive thin client, I don’t think we have anything like this. And I don’t recommend reusing Greenfield either (I don’t have a lot of coding experience, but I can see my colleagues project, which also uses nodejs, it’s a hog of a program - aside from other JS programs which are hogs as well).
With a night’s sleep (and plenty of coffee, I really am getting to old for the all-night-coding binges), I think I have some idea of how to proceed.
The kind folks in #swaywm on freenode pointed me at a better introduction to the Wayland protocol than the docs hosted on freedesktop.org (hence why I stayed up far too late reading). If you want to understand the Wayland protocol, or how libwayland (the reference implementation) works, https://wayland-book.com/ is where to start.
As I wrote last night, I don’t think any of the existing solutions will work well. In part because they’re all written in pointer-heavy C (and for this project to succeed, I need to enjoy it). And in part because they have certain assumptions baked in (like having a display connected) which might not hold true. I’d likely end up reaching into non-public internals, or at the very least, having to read lots of those non-public internal. Having spent a fair bit of time trying to read the libwayland source code last night (usually if the freedesktop.org documentation for a project sucks, the source code is clear enough to use), I can say that libwayland was written by people far smarter than I.
So here are the three options still on the table.
Option 1: Make a wlroots backend
Instead of trying to make a custom passthrough compositor, I could just add a backend for wlroots that handles a remote display. This could even use LD_PRELOAD tricks to let wlroots based compositors display over a network transparently. The major downside is that wlroots isn’t really designed for pluggable extensions, so I can’t simply populate a struct of function pointers and call wlr_register_backend or anything simple like that… I’d also probably end up writing it in C, as that would make it easier to get included in wlroots upstream. This would basically be adding network transparency to wlroots, and would then rely on nesting gnome/kde inside wlroots to support other compositors. It may be the simplest approach. It would also pretty much preclude nesting the client half inside a Wayland compositor on the other end.
Option 2: Implement the core Wayland protocol in C++
So far, no one has implemented Wayland in modern C++. There’s a C++11 wrapper around libwayland, but that comes with the same limitations of libwayland, and only has the common and client code, no server wrapper. Writing a “wayland scanner” that would generate the C++ interface would be pretty simple, then it’s just a matter of implementing the pieces of it needed for tying everything together. Even if I don’t get Wafer itself in a useful state before life pulls me other directions, the C++ implementation is likely to be useful for someone at some point. Like option1, it is transparent to the end-user applications.
Option 3: Implement Wayland over TCP
The core wayland protocol includes an interface for shared memory. It is the only interface for bulk data transfer included in the core protocol. It would, however, be possible to write an extension interface that requires clients (in this case the Wayland Client Compositor) to allocate their own local buffers and stream the buffers over TCP. This is the most technically complicated approach, as it would require patching existing clients (at the least wlroots) to use this extension. It’s also the most technically “correct”, as it would allow a client to know it’s being accessed remotely, possibly changing how the client behaves (for example, letting the client query available bandwidth or similar). Any client not patched is likely to fail, as the server would not expose the wl_shm interface. It would have the advantage that the Thin Wayland Compositor drops out of the mix, so that offsets the complexity somewhat…
I am disinclined toward option 1, as I really don’t enjoy working in C. Also, while theoretically straightforward, it’s not really got a future. In the best case, the new backend gets upstreamed and wlroots-based compositors benefit from it. It won’t really help other compositors, it won’t let you nest a wayland window in a local wayland compositor, it won’t allow a future extension for network-aware wayland clients.
Options 2 and 3 would share a fair amount of implementation details, so can be developed semi in parallel until one or the other becomes the obvious choice. I’m actually thinking I may implement a full pass-through compositor on the mainframe side, and then use the theoretical TCP-wayland extension to communicate to the thin-client side. In the future, wlroots, or gtk, or any other Wayland client could then implement the TCP-wayland extension and skip the mainframe compositor.
NodeJS is very much where Python was circa 2002. It ships “batteries included”, so the odds are good you can skip all the low-level details and write your simple thing with minimal programmer time. The downside is it achieves that flexibility in large part by just throwing memory and cycles at the problem. It’s a very good thing for us that it exists, as it allows easier prototyping, which means faster development, but I wouldn’t use it for a well-defined problem outside its wheelhouse.
