Return to Level1Techs.com

Unusual networking/security conundrum

Well perhaps not so unusual I suppose…

We have many CNC machines that all run off WinXP/2000. The cost to upgrade them to W10 is prohibitive. We have projects that cannot be “outside” of mainstream/typical enterprise security. These machines need access to files created by the programming department. The IT dept does not want the older OS’s to have access to anything. So they are physically connected but blocked completely. The IT dept also does not want to do much of anything with these machines because of the age of the OS.

I am thinking an intermediary server, specs can be VERY low, nothing bleeding edge here. Something like a small W10 box with no more/less than a quad core and 16bg of ram. W10 would host a VM running Debian, and using IOMMU (from what I understand), I should be able to dedicate an NIC to the Debian VM. The VM would plug into a completely separate router that also connects to all the shop floor machines.

So… The IT dept can set up a shared folder on the W10 machine so the programmers can upload their files there. The Debian VM would also use this shared folder and give the shop floor machines access to these files. Keep in mind the Deb VM would not be “physically” connected to anything but the stand alone router that the machines are also connected to, no internet or any outside connection, so this solves the security issue. The Windows machine that the programmers upload files to will be completely integrated into the current security system so no issues with an obsolete OS.

These files (G-Code) are never really more than 30mb MAX and they only get accessed like once ever hour to 2hours or so, by perhaps 4-7 employees (bandwidth will never be an issue on a 1gb connection). Needless to say, the hardware should be relatively inexpensive, hell some of the machines only have a 100mb NIC.

I would like other experts thoughts on this setup, opinions otherwise, and/or recommendations on hardware for ease of IOMMU setup. Do you see any caveats in the setup? Pitfalls to avoid?

1 Like

As I understood you, you need a “minimal NAS” with dual NICs (one port per network).

Something like a Shuttle Edge EN01j4 with the PoE PSE Module installed should do. Mount on DIN-rail and forget it exists.
Or Shuttle Box-PC BPCWL02

That lil box will support windows 10 and 2 separate NICs on different IOMMUs?

Is there a specific reason you need a VM running on there?

Edit: Specs wise, it should.

I need a highly stable connection to WinXP/2000 for sharing files. The VM, with it’s own NIC, running Linux will provide that. This is just the first solution that came to mind. Because Debian communicates with XP/2000 better than W10. The VM also provides separation from the current network, so the Machines do not connect to the existing network and get blacklisted from communications.

Highly stable is a given (Industry PC after all).

My thinking would be to have something like the above provide an SMB-share and then have one NIC of the PC face towards the engineers and the other NIC towards the shop floor.

The IT dept will not setup security on JUST a linux machine though. The “big brother” software that they use does not have a valid linux install, OR they refuse to bother with linux at all. Sorry I didn’t explain this from the beginning. By giving them a W10 environment they get to feel like they are in control of something and by having a linux VM, I get to configure the shares to the machines, nice and simple.

I’m already going to be on their naughty list, because I am going over the IT departments heads to get this done, I have to include them somehow and I have to give them a system that they understand how to configure. They have already said that connecting the machines is “impossible” with their current security software.

2 Likes

@ir_efrem Have you informed anybody from your management team about the situation? IT should not be the deciding factor if a whole section of the company can work or not. That is a management decision. It is up to management to take the advice of IT. When IT says that XP is EOL for sometime and a security risk, management has to decide what the cost of upgrading these machines would be compared to running them on a EOL platform with potential risk of ransomware if connected to the rest of the company.

Your IT department should be the ones coming up with the solution as well to securely connect the XP machines to a isolated part of your corporate network where files can be dropped.

How are the files being put on the CNC machines now? Or is this a new process?

The idea of running a Windows 10 machine with a shared folder to a VM on that machine is plausible and could work. However you say the IT dept likes to lock things down, that same logical would apply to the Windows box as well. If they are worth anything they are not going to allow Hyper-V to be enabled and then run a VM which they have no visibility into. Going over their heads will just cause contention.

As @MazeFrame suggested using two nics with one on the corporate lan so devs can drop files and the other into the standalone shop router so the CNC machines can have access to the shared folder should work just fine. I have done this in the medical industry, with a old XP machine that needed to be connected to a it’s standalone machine network, while also being connected to our EMR system.

So instead of running a vm on windows 10, have your IT department source a machine with two nics or just a buy a secondary PCIe nic card and install Windows 10 LTSC. This will more than stable enough to run a small file share, however IT will need to enable smb1 communication on the Win10 box. This should satisfy your IT departments needs to be in control of the box and you will have a stable way to get files to your CNC machines. Once again, inform your management of the situation and let them make the final decision. If they shoot the idea down, post back here. There are many different ways to accomplish what you want, while maintaining security and business continuity.

3 Likes

Alternative to consider:

Onto the network with xp/2000 machines, add a Debian on bare metal machine, one network port to xp/2000 network, other onto internet (directly or via corporate, doesn’t matter). Upload files over https, or ftps, or sftp over internet, share with xp/2000 network using samba.

Nothing for IT department to manage other than allowing programming department access to 1 extra network ip:port number that’s using TLS or SSH. No big brother security, it’s not a computer it’s an unmanaged device on a separate network that they can firewall off from everything, except 1 ip:port towards it.

Actual hardware could be a raspberry pi in terms of processing power needed, but because you want something not absolute crap, make sure it’s something with no moving parts and reasonably enclosed in a some kind of metal case that doubles as a heatsink, use optane or similar high dwpd flash storage with pwp (bit pricier but not prohibitively so… could cost you $100-$150 total).

A $50 nanopi r2s that’s industrial device looking-ish and has two network ports would do the trick just fine, as long as you store gcode on some usb3 storage. As an added bonus, you could probably battery power it, or attach wifi or lte dongle or something. You could get two, other as a spare and for testing.

IMO @OriginalDotte knocks it out of the park. I would stick to Win10 assuming the IT department does any or a combination of network discovery scans, patch management and hardening requirements. The latter would be a problem for SMB v1 being enabled so as @OriginalDotte mentions, a shot caller needs to be involved if any deviation to benchmarks, policy or standards are being made.

Using an OS already on the network (so I’m assuming there is a test and evaluation method for them, GPOs setup, routine scanning, patch management etc is already in place) IMO is more desirable over introducing more OS’s and threat vectors into the network to pivot from such as a NAS and/or *nix OS. And its just more burdensome for that team to stay on top of (new T&E documentation, patches to stay on top of etc).

If you do not want SMB v1 to be present on the corporate WAN/LAN, then maybe bothering with a VM with one NIC in pass-through that has share access to the host OS share but then presents a SMB v1 share to the “encapsulated” CNC network might be worthwhile? I’m not sure.

Sounds like a fun project, but yeah, buy-in from the powers that be is important. Make them think it was their idea.

2 Likes

The IT dept has said it’s impossible, end of story, no more discussion.

Now let me explain a couple things about our current office politics. We are a daughter company, quite literally. A father “gave” his daughter a company to make goods for his company. She is a figure head, and dad still runs the company, plus in my area there is an additional tax write off for female owned businesses. Not being misogynistic but we are a pieced together company that really has little sway and is sort of the “joke” of the company (as a whole)… Our IT dept, is actually in the parent company, we don’t have our own. They do not answer to anyone in my building. In a perfect world everyone would do what is “normal” and right, I don’t live in that world at my company. If our CEO (who has asked me what we can do) says just do it anyway, well I am going to give a whirl and see what I can accomplish.

Files are put on the CNC machines via sneaker net which is wearing out the USB ports. I cannot be certain if anyone if the IT dept would even care about Hyper-V being enabled, but I can only guess here. So here I am… At least 2 of the main machines do NOT have an upgrade path to W10, the others do, but this is a minimum of $15,000 per machine. For 6 machines, well, I hope you see the problem. I need a machine in the middle that they can “secure” and a way to make it so the CNC (XP/2000) machines can get files off those. I am not entirely certain if my idea is completely viable but I have been tasked to do “something.”

Edit: Files are already shared across the network using SMB btw.

I like the idea, but I have to consider the number of steps to get files where they need to be and make it a browsable location so the CNC programmers can literally browse to the file location from the CAD/CAM software to save files. This is a non negotiable requirement.

Constraints that I do not like are in place and I have to try and “fit” a solution into place. This is why my original idea still seems the strongest “in my mind.”

1 machine, 2 NICs, 1 VM with a shared folder from the host, then just share that drive via Linux. I would do an XP virtual machine, but I haven’t found a way to share a “shared VM folder” in XP. In Linux it’s pretty seamless, because you just set it up that way, nice and simple (in my head anyway).

Edit: I was kind of thinking a 1U mount so it fits well into the existing rack TBH, but even a desktop case just sitting there will be fine. our server closet is actually a 10x10 room, so space is not a constraint. I would prob end up getting a 1U router though, but that’s besides the point because a router is the simplest piece of gear in this whole setup, hell if it’s cheaper I can just use a switch and plug a cheap ass router into it. I’ll see when I get to that point.

This was my first idea but they are dead set against it, because “W10 does not share files with XP/2000 very well, we don’t want to constantly mess with this.”

Funny thing is, there is already W2K server up and running, doing tasks for an entirely different set of machines that is walled off from the outside world but other machines have access to it. They are being lazy about our options and they only want ALL W10 machines from here on out so they don’t have to deal with additional systems in place. I understand this but sometimes you got to do what is needed. The machines that we have that do not have a “simple” upgrade path to W10 would cost in the order of $100k+ to completely upgrade the hardware/software to be cross compatible with the old machine. So you may as well spend the extra and purchase a new machine. Sad thing is that industry (not the our machines anyway) does not follow our “need to secure and upgrade” many new machines that would replace what we have, well, they still use XP as well, because it works so well for these type of controller systems.

So what would you fine folks do?

Unfortunately you have a precarious situation. Which I can sympathize with as the US medical industry has the same issues with old operating systems and the enormous cost to upgrade them which leads to no upgrades and IT struggling to secure and connect them. As a new MRI machine can easily cost 5 million.

If you can get a Windows machine with two nics, then the idea of using the win10 box as a standalone machine with a shared folder is a viable one. However if you do not have local admin rights on the machine, you will not be able to install anything. That includes Hyper-V or VirtualBox or any basic software for that matter. If you do get local admin access, you are still left with how to get the files to the shared folder. Usually that is done through standard NTFS permissions set by the IT department through security groups. Probably how your current SMB shares are setup.

If the IT dept wont setup security groups then somebody, most likely yourself will be monitoring some corporate folder where the files are currently shared and coping them to the shared folder you have created which the CNC machines can view.

@risk idea is good. Should work just fine, you will have total control. However the same thing applies that someone will need to monitor wherever the files are being dropped and copy them to the other linux machine by some protocol.

If you are dead set about going down the whole linux VM or bare metal linux, can you fit managing that machine into your current job duties? It will be your responsibility to keep it patched and fix anything that breaks. Also you will be the sole owner of this process, so if you take vacation how will the files get to their destination or if something goes wrong who fixes it? If this new process goes into production and you aren’t there to fix it how does the business continue? Definitely not trying to discourage you, just want to make sure you avoid some of the pitfalls I have fallen into.

One final thing is to CYA for whatever you decide to do. You said your CEO has tasked you with this issue. I would gather my ideas, present them to the CEO and have him sign off in writing to on a plan. What you don’t want is for this project to some how lead to into a ransomware event because the linux machine wasn’t patched and @ir_efrem was responsible he set it up and no higher authorities knew about it.

All very good points. I am hoping that the IT guys will be onboard with the idea when they find out that they don’t have to set it up. One less thing for them to do… I absolutely have to have their co-operation or it wont move forward at all. They have to set up the shared folders so the programmers can access it. I think we can avoid a ransomware situation, all they have to do is cut the system off from the outside world, in which case it would have been easier to just use a W10 setup for the whole project…

I am not aware enough of how these types of attacks happen to know how this would happen anyway. It would be a windows share to start with, so if a person has “hacked” their way past the firewall and onto the windows machine already, well… how does the Linux VM even play a part in this? Does Hyper-V make this specific system even more susceptible to outside attacks that can make it past all the security in place now? This isn’t going to be a computer that is used for checking email and browsing, nothing else will even be installed on it and no one will ever touch it. Just windows, the VM, and the security software. I truly desire to understand this a tad better so I can weigh my options here.

Edit: As far as that goes, the VM also wont have any access to the corp LAN at all, so a “hacker” or malicious software would need some high level control over the Windows Machine and know what do actually do to the Linux VM to “perform said dastardly deed.” If the breech has given access to the windows machine, the problem is already there because those files are all that are on the machine. Since this will have a limited account on Windows, there will also only be so many things a security breech would give the malicious person/software access to. Would Hyper-V just break the entire network and give someone high level access to all that is there?

From a security perspective, what you’re building is similar to a “bastion host”. This pair of bastions (VM+win10 host the VM is running in) is protecting both your winxp/2000 machines and your corp network/win10 machines.

Your idea to do it with VMs could work technically, but it’s super complex in a way. It presumes there’s a way to share files between a VM and host, which exposes you to an attack vector.

Further, you can assume whoever has root on win10 absolutely owns your winxp/win2000 machines (sooner or later) and the attack vector will be through your precious IT managed win10 programmer boxes that humans use to do work / read email / communicate on teams, etc…

Next evolution of the idea:

use a pair of bastions. One in front of your winxp/win2000 boxes, one in front of your win10 network. And agree on a protocol how they can share data.
If you make win10 be a host with a file share , how will your Linux box access those files - and vice versa.
To read (or list or write, or have other access in general) files from a Windows share managed by active directory, the samba client needs a kerberos principal that can be used to get tickets from the AD controller.

In principle this is easy peasy, but I doubt your IT dept is knowledgeable enough.

If you host a Samba share on your industrial bastion, all you need is to open port 445 and allow any user coming from the win10 bastion to write to it unauthenticated - you trust the network. The workflow there would be to have a robocopy cron running on win10 host to copy files from win10 share to Linux share.

Your IT dept might be ok with this, I wouldn’t be. Majority of windows attacks have historically involved scheduled tasks on Windows - it’s a very useful widget.
Secondly, you’re making a read/write dumping ground on your Linux box that can go unaudited to your Windows XP/2000 boxes … Bad idea.

Next evolution: two hosts, on your Linux / industrial host, split the win10 writable section of the filesystem from a filesystem that winxp/win2000 boxes manage.

This is better, because you can nuke any file that’s not valid gcode from the share before any of the winxp/win2000 hosts are exposed to it.

It means win10 is exposed to potentially rogue samba host.

Next evolution: find a protocol win10 IT department might be happy with to allow the win10 to use to copy files out.
Industry standard is sftp/ftps (it’s how most banks exchange files, they call them “ftp drops”, they stick encrypted zip files in there and then email each other passwords). You’ll need software to periodically copy win10 gcode to winxp/2000/Linux bastion.

Next idea: run an sftp drop over an LTE stick / internet if need be, and have individuals from your programming dept upload files directly - avoiding any festering win10 file shares that are much harder to setup as write only drop off points.

Does your programming dept have access to the internet? If yes, this is probably what you should do. Let them log in through individual public keys.

Next evolution: put the public key on the win10 file share host and have it copy things over to your drop off.

At this point we’re spinning in circles.

final evolution

Make an SFTP drop on Linux, and let 1 IT managed windows 10 account/machine/user with access to a network share somewhere (can be anywhere on your network), upload files to the SFTP drop, it can be a VM instead of a physical host if they’re comfortable giving you RDP access, no physical hardware needed. It just needs access to the drop, not the rest of internet, and needs to run winscp to your SFTP drop, that’s it. They can monitor and do whatever with it. No physical hardware needed there.


Your Linux host can prune the sftp drop and only ingest valid looking gcode into a samba share for your industrial winxp/win2000 machines. That’s not your IT dept problem it responsibility, same as winxp/2000 machines aren’t. It doesn’t need access to any of your IT managed infrastructure, one machine on your IT managed infrastructure needs access to it, and they can monitor the hell out of that machine.


Hardware wise, any machine would do for your Linux box, it doesn’t need to sit on a shop floor but should probably be on the same VLAN, or be accessible by your winxp/win2000 boxes; it does need two network interfaces at least, but that’s up to you. It should have internet access (doesn’t need to be accessible), because you’ll need to pull updates once in a while, and there should be ssh access to it exposed from your corporate network for SFTP as well as you yourself to be able to manage it. But needs nothing else. You could even forbid it internet access and feed it updates through ssh while logged in.

3 Likes

Have you considered a raspberry pi running USB mass storage emulation over USB OTG?

I like that solution, but it would require more computers to manage and does not run windows anywhere. But if the Pi is 0wn3d, the computer attached to it is as well(it could even switch to HID and change BIOS settings!).
On the other hand, the 2K/XP machines would not require any network connections.

If running windows is not a hard requirement, did you consider just using a simple NAT router(you can get openWRT-supported “plastic” routers for <$25 easily for testing the setup), and block all traffic except to some filesharing somewhere? Some OpenWRT routers are powerful enough to host a simple SAMBA share themselves.

My full idea goes something like this, I think I laid it out well but here goes, just to remove ambiguity.

Setup W10 on physical hardware (perhaps LTSC if needed, we probably have extra keys for either). I make the user login a local admin. The IT dept already trusts me enough to have made me a local admin on my workstation. I then set up VMware and set up the shared directory. IT Dept sets up that same directory and maps it to the 3 CAD/CAM workstations that need to save gcode there and 3 shop floor workstations that need read access, they also install the fortinet software (which makes them happy). There are already rules in place for this share, so it should be copy/paste for them.

Now when I setup Debian, I set this same directory as a share for the machines that needs access to gcode. This is VERY simple to do. As discussed 2 separate NICs and the Linux VM only connects to a new router that only connects to the shop floor machines.

Please correct me if I am wrong. The attack vector that you speak of, seems to me, is already there. If some one gets into the system and has access to the current machine hosting the share, well they already have the Gcode and gaining access to a Linux VM will only give them the same access that they already have, to the Gcode. Since a physical change would be needed to in order for the Linux VM to access anything else, there is nothing more that could be done?

I want you fine folks to keep in mind that I HAVE to set this up so everything is completely transparent to my windows users. They open File Explorer in W10 and POOF, there’s the files, readable and writable. I do find it odd that anyone in the building can walk up to a workstation and delete all the files, but whatever, it’s already setup that way so I wont change it; it’s stupid that they bleat about security but have this flaw. This ease of access is a top of the list requirement. #2 on this list is ease of admin/security.

I have admin figured out though, because our building has a WiFi connection for visitors. It should be a simple manner to connect to it download Linux Updates and disconnect. A couple minutes a month will keep security patches in place and up to date. And yes this is also a separate attack vector, but nothing is 100% and this connection to the internet can be done on a random set of days at a random time at night, to help mitigate some of the potential threat. A script can be set to run, connect, update and DC. I can also check in on it and train at least another person on how to maintain the server. This should be a VERY simple and stable connection for users to get access to gcode on their machines, there is so very little to go wrong.