Return to

Robot hacking, leads to many questions about X11


Ok so I have a pair of Lely a4 milking robots, about 4 years old. Here’s a video if you want to see what they do
This thread is the story of how I persistently prodded the system until it (sort of) delivered the functionality I wanted, and a request to you, dear viewer, for any advise you might have on achieving the last 5%

When we bought these robots I was disappointed that it didn’t come with tools to let you do stuff remotely, like clear errors, reboot the robot etc. So naturally I installed Kali linux, plugged my laptop into the robot LAN, and fired up NMAP.

Starting Nmap 7.40 ( ) at 2019-02-25 12:08 
Eastern Standard Time                               
Nmap scan report for                                                                                   
Host is up (0.0019s latency).                                                                                   
Not shown: 996 closed ports                                                                                     
PORT     STATE SERVICE                                                                                          
22/tcp   open  ssh                                                                                              
5900/tcp open  vnc                                                                                              
6000/tcp open  X11                                                                                              
8080/tcp open  http-proxy                                                                                       
MAC Address: 00:12:FF:01:A4:0C (Lely Industries N.V.)                                                           
Device type: general purpose                                                                                    
Running: Linux 2.6.X|3.X                                                                                        
OS CPE: cpe:/o:linux:linux_kernel:2.6 
OS details: Linux 2.6.32 - 3.10                                                                                 
Network Distance: 1 hop                                                                                         

So this is interesting… It has a VNC port open. Fire up VNC:

Very nice! Unsecured VNC! It shows the same stuff as the touchscreen on the robot.
On the other hand. Bummer! It seems to be view only :frowning: I can’t do stuff, just watch.

But what’s that? An open X11 port on 6000?
This is where I start to get out of my depth. I really don’t understand the X11 client/server model. Is it backwards, is it forwards? Do clients listen with open ports? Do servers display stuff on screens? I have no idea.

What I DO know is if i type:
export DISPLAY=

Then the xterm window shows up on the robot screen :laughing:

So clearly this port 6000 is also unsecured!
At this point I spent days trying to understand what was going on. I can capture the frames from the robot screen using xwd(which is pretty cool) and I eventually discovered a pretty neat utility called ‘xdotool’ which allows me to send keystrokes, mouse movement and mouse click over the network to port 6000. We’re in like Flynn! If I “xdotool click 1” while the mouse is over the button in the bottom right, the actions menu comes up, and I can initiate a cleaning cycle, put the robot into or out of operation, or divert that cow’s milk to a holding pail to be fed to calves (red buttons on the right)

If you’ve made it this far, thanks for reading! And if you know how X11 works please enlighten me as to how this functionality is possible, and if there is a slightly less ‘bodged’ way to do this.
Ideally what I’d like to do is ‘connnect to the x session’ if such a thing is possible. Exactly like how you would VNC into a machine. It kind of seems to me this is exactly what X was designed to do?

Bonus points:
You may have noticed there’s also an ssh port open. About that:
It appears to accept password logins, and I have tried all the obvious ones. I’m 99% sure it is running a hella old version of openssh that probably has documented vulnerabilities but again that’s out of my wheelhouse unfortunately. It has been a while since I tried the ssh but I seem to recall having to allow an encryption method that isn’t even supported normally anymore.

Bonus bonus:
There’s a single open USB port on the thing, so naturally I plugged in a keyboard and pressed Ctrl+alt+F3
And yes! it gets me a login screen, but again I have no idea the credentials :frowning:

Any feedback would be greatly appreciated!
Calf tax:




see if xtogo sees it thats all I got.

also smooch a baby moo nose for me I love cows <3



You guessed right, it’s doing what it’s supposed to do. Xorg acts as a server for remote management. Taken from man page:

Xorg listens on port 6000+n, where n is the display number. This
connection type is usually disabled by default, but may be enabled
with the -listen option (see the Xserver(1) man page for details).

Seems like it’s using host-based access control, hence no login needed. Most likely configured to accept all connections from the LAN, or who knows, the whole internet.



Is there a way of “Streaming” mouse inputs and clicks from my machine to the robot, rather than entering them discretely from the terminal?



Can’t say why it’s not forwarding your inputs, tho best practice would be to forward X through ssh anyway. Having Xserver ports open is terrible from a security standpoint. It’s quite interesting that the manufacturer left those ports accessible like that, whatever the reasons might be. There’s better ways to remote machines, it’s not 1995 anymore.
Maybe you could call them and ask what’s up with that?

While you’re at it you could also ask, but you’re most likely not gonna get, creds for the box to ssh into with - ofc you could locally access the machine and change passwords and configure ssh by different methods but really you shouldn’t mess with the system at all since it’s not a testing environment.



It is very strange that they leave the ports open, I think it is a holdover from testing/dev but I cant be 100% sure. These are 1/4 of a million dollar robots. Typical of companies like this, their response to any technical questions like that are “We aren’t going to tell you” Despite the fact that it’s my hardware. (Stallman intensifies)
I agree I’d like to get terminal access just to see what the heck the system looks like on the software side.

I have been told by a service tech that this touchscreen is an ARM tablet running android 2. You’ll recognize the GUI on it from the screenshots above. This tabley has 2 plugs on it. One proprietary waterproof connector with a bunch of conductors, approximately 12 or so, and a USB port. The fancy cable goes to this board

Which has what I think is an ARM SOC in the middle, and then it has a whole bunch of CANBUS connectors and logic outputs (for running relays to open and close valves etc) The network connects just above that ARM chip

I can’t tell whether the screen is a separate system running its own os just do do the display, or if that other connector is a proprietary waterproof VGA + power, for example. I do have physical access to the machine obviously, but there’s no removable storage devices

I do have local access to the device but I have no idea how I’d go about changing the passwords without any command line access, or any harddrives I could dig into and change config files
There is a DB9 port that I believe could be used as a serial console, but I don’t have requisite hardware to plug into it, and I’m afraid it might be 5V or something. I don’t want to fry the motherboard. Also there’s still a pretty good probability I would still have to log in on the serial right?



x2x should be able to do what you wanted in regards to control. It’s typically used over SSH, and allows you to share your keyboard and mouse pointer with another X server(You don’t need to use it over SSH though!). I think it requires the xtest extension, but that’s fairly default. Other than that, you could possibly start a VNC server on the X server(Even from another machine!) with normal mouse/keyboard support(You want a VNC server for a running X Server, not a VNC Server with a built-in X Server).

Also, you should be able to check the SSH version, by simply:
nc -vvv 22
Or you can just connect with SSH and the verbose output, also tells what other authentication methods the server supports, among a lot of other things. If you report that here, someone might be able to look at CVE’s for those versions, and might be able to provide you with a working exploit.

Besides, X was never a secure protocol. it should be fairly easy to exploit. Never done that though.



Oh, this image looks very promising! There are 2 connectors commonly used for serial access on the board. The one in the lower-right is just straight up a standard serial port. The connector in the top could be a RJ-45, possibly ethernet, or maybe a serial port like on a switch! I think with that kind of access it should be very possible to get a shell running on that thing!

One quick note though, I don’t know if poking around in the software is permitted by software licenses, or if there are other legal or warranty hazards you need to look at.
I think the chance of a hardware fault because of this is very small, but again, I don’t know enough about this to be sure. Proceed with care!



Bruh kevin mitnick straight up stole sunOS in the 80’s and screwed around with it however he wanted.

You bought them they’re yours / your companies.

1 Like


Thanks for the interest and the reply!
That Ethernet port is the robot network(normally plugged in) It was unplugged for the picture because we had a switch burn out so I took the picture mid swap.
I saw the serial port on the board. I have been thinking of picking up a USB to serial cable to try it out and see what I get, but for now I dont have one.
In the meantime, here is some selected output from nmap -A -vvv:
Kinda interesting that lely has their own MAC

22/tcp open ssh syn-ack ttl 64 Dropbear sshd 0.52 (protocol 2.0)
| ssh-hostkey:
| 1040 be:44:a8:36:71:ec:1e:b9:df:28:23:d3:c9:eb:b6:8a (RSA)

MAC Address: 00:12:FF:01:A8:11 (Lely Industries N.V.)
Device type: general purpose
Running: Linux 2.6.X|3.X
OS CPE: cpe:/o:linux:linux_kernel:2.6 cpe:/o:linux:linux_kernel:3
OS details: Linux 2.6.32 - 3.10

So it seems to be running dropbear 0.52. There is a CVE which allows username enumeration
so that’s kinda interesting, but even then I’d have to bruteforce passwords



So today I was back in a city with an adequate computer store. Picked up a USB-RS232 cable (and a null-modem adaptor in case I need it) Will try on the weekend and see what happens when I plug it in. Might have to do a reboot and see if the bootloader does any serial output.
Now for the reason I did a bump without any serious progress updates…
Is there any way to do an automated sort of ‘scan’ or ‘sweep’ of serial port settings? I’ll try the most common 9600 8n1 and 115200 8n1, and maybe I’m overthinking it, but aren’t there huge numbers of possible combinations? I’m worried about the possibility that I get the settings wrong and get no output, even if the robot is outputting actively.



Update. Connected to serial 38400 baud. Upon bootup the robot outputs a bunch of diagnostic crap. Unfortunately I don’t think any of it is interesting for the purpose of getting to the command line.