[Pointers] X11 Servers, connecting to existing connections -hacking

Software & Operating Systems Linux
1

So I'm a dairy farmer, and a couple years ago we bought a pair of Lely A4 Astronaut milking robots. They work great and they milk cows pretty well but there are a few things that they don't let you do out of the box.

Now for the cool part. They run Linux! I'm not exactly sure what's going on behind the scenes but according to their service techs the robot has a built in tablet running like Android 2 or something awful and ancient. Here's a screenshot. http://imgur.com/a/6CmiQ

The robots are connected to a LAN. I fired up a port scanner and found an open VNC port, but unfortunately it is view only:. All is not lost however, Port 6000 is open! (and also unsecured) That's how I got this screenshot, Using
xwd -root -screen -silent -display 10.4.1.102:0 > screenshot.xwd

What I am asking you guys for help on, is taking over that X session remotely, so I can click those buttons on the screen from, well, not in front of the screen.
I've spent about 5 hours reading but X11 terminology is really starting to wear by brain out, and it seems that 99.9% of examples deal with either SSH tunnelling (not an option) or using Xnest and XDMCP to start a new login session remotely, which is also not what I want to do.

I think it seems like Xnest is along the lines of what I want, but the critical piece here is to connect to an existing "Client -Server session" ? Is that correct terminology?

TL;DR: Port 6000 is open, I can grab frames from it but my goals are 1) stream it live to my PC. 2) use mouse&keyboard for input

Any input you guys have would be greatly appreciated. Will provide robot or cow pics on request

Andrew

2

Prolly not what you want but i'll throw in a comment anyway: (can you ssh into port 6000?) easy way to forward X through i.e. ssh, you find out which $DISPLAY your local machine is connected to

$ echo $DISPLAY

Assuming it's :0.0

You open your $DISPLAY to your local network, better be behind a firewall.
(On your local machine:)
$ xhost +local

Then on your remote machine you're i.e. SSHing into

'# export $DISPLAY=:0

Open up the GUI app on the remote machine and the window will show on yohr local machine.

Close your wide-open $DISPLAY with
$ xhost -

Again, prolly not what you want but this is just a headsup
EDIT: not sure if it works over a network on second thought, I just use that to access GUIs in containers.

1 Like
3

I have no command-line access on the robot. The only input device is a touchscreen, of which I have placed a screenshot in the OP. SSH port is available but I have no login credentials. Port 6000 however, is already open to the LAN, and you may already assume xhost +local has been run (not on my laptop, on the robot) because I can grab frames with xwd -root -screen -silent -display 10.4.1.102:0 > screenshot.xwd

I can also send mouse-move, click and keystroke commands with xdotool. This does give me a rudimentary remote-access but it is incredibly clunky becaue you have to grab a frame, move the mouse, then click. Each one of these operations takes about 10 seconds. I have tried sending Alt+F2 x t e r m but apparently that is not a valid keyboard shortcut on this display system.

1 Like