Small business Server advice ( Medical center)

Hardware Hub
1

Hi guys,

I am currently looking for advice on the best way to improve old servers for a radiology center I recently joined ( software and hardware wise) . I must point out that I am not a professional IT guy but I have always been tech savy ( I do run a small server in my house).

Present configuration of the client ( Software and hardware) :

Software :

The client has 2 major software that he purchased in order to run his business. These are common type of software used in the healthcare industry. The first software is a radiology information system (RIS) which allows the client to record all data about the patients ( personnal information, billing and radiology reports). The software uses mainly an SQL database. The software itself is not very resource intensive but the sql database can suck quit a bit of ram ( I am not with familiar with how SQL works but I think it uses as much as you give it)

The second one is a PACS system, which centralize all the X-ray images from the differents machines ( CT, US, Ultrasound…). This software uses also an SQL database but also stores the images locally.
The stored images are in a very specific format called DICOM which splits the images into very small files ( 1 KB or 512 Kb).

The business uses windows server as it is a requirement for both software to work

Hardware :

The company uses two system : one for the RIS software and another one the pacs sotfware.

RIS hardware : The first one is an old Dell Server ( Poweredge T110) (CPU : E3-1220 V2, RAM : 16 GB DDR 3 ECC, Two 256 TB drives in raid 1 and A 4tb usb HDD for software backup). The whole database size does not exceed 150 GB ( 10 years) )

Pacs Hardware : Dell PowerEdge T340 ( CPU : E-2124 4 cores, RAM : 16 GB DDR4 ECC, HDD : Two 1 TB in raid Raid 1 for OS install and Four 4 TB drivers setup in raid 5, the raid configuration is managed by the dell controller percH330).

The business is fairly small as the first server is used roughly by 10 People and the second one is mainly used by the doctors ( 4 to 6 People at most)

Clients : Around 20 computers ( Some of them are really old runing windows XP but perform light tasks ( generally as a portal for a machine) and some of them fairly recent, dell precision with core I3 processors used for printing and word processing within the ris software.

The whole business runs on a 1GBS network connection.

Problems or annoyances with the current configuration :

  • I am an accountant and my work heavily requires the data from the ris software. When I try to extract data from the software over a long time frame, it takes way too long. However the users working on data that is recent, do not experience any hickup as i think it is stored in RAM.

  • Sometimes the transfer speed to the PACS server from the machines slows down if multiple machine are sending data at the same time and we had issues of images not being sent.

  • Recently, The PACS server reached its had full capacity and i tried to expand the raid array but it is still reconstructing after 1 day.

  • Both servers are painfully slow within windows server, the interface is laggy and takes 30 sec to 1 min to launch a software.

Other Information :

  • The Pacs server receives around 6 to 9 GB worth of files everyday. As I previously mentioned, the individual file size ranges from 1KB all the way to 10 MB

-Recently, the client had some issues with its network cabling so we redid the whole thing in CAT 6. Some of the client were negociating at 100 Mb speeds. We also invested in some ubiquiti gear in order to improve the network.
I will be adding a 10 GBE connection to the PACS Server and one of the machines ( CT scans) which generate the heavier files ( Around 1 GB per exam).

  • Backup : The client uses USB drives for backup. However, the backup is managed by the RIS and PACS software and backups 6h after closing. It only backup images and SQL databases

  • The CPU load on both hover around 40 to 60% during heavy use and 2-10% during light use.

  • The RAM usage is around 12 to 13 GB.

Future configuration :

As I see it, I have two options, either change the current servers for a new one or keep the old ones ( Most likely as the owner just splurged on the T340 Poweredge in 2021 and does not want to invest anymore money). Either way, I want to migrate the storage to SSD at least for recent data and setup a proper backup server.

Questions :

Scenario in which we keep the old servers :

  • I was considering Virtualization with proxmox and run both software on the T340. Do you think It ll be able to handle the workload of both softwares with only 4 cores ?

  • Which bring to my next question, I want use ZFS but as I understand it is not natlvely supported by windows. I would either need a different server and setup a network share ( SMB or ISCSI) or virtualization. Do you have any information on how SQL behave with zfs over ISCSI in windows. I heard some things about mismatch sector configuration and asynchornous read and write.

  • In the event that I cannot use ZFS, I would have to stick with raid using the percH330 controller. During my research, I found out that dell servers are very picky about what type of SSD it uses. I was considering using standard consumer samsung EVO sata SSD. Do you have any experience with samsung SSD in dell servers ?

  • Backup : If I go virtualization, I can use proxmox built in VM backup and use R sync for the other data. But If keep windows server as the host OS, what do you guys advise me to do in order to create a full system backup that requires little downtime to restore as these two software are critical.

Would love to hear your thoughts.

Thank you in advance.

1 Like
2

Proxmox uses ZFS (windows can’t tell the difference, has nothing to do with ZFS), you’d just carve out ZFS volumes through the proxmox web UI, which are basically raid-ed block devices, which can underpin virtual disks for Windows. Windows only sees a SCSI or a virtio disk.

Don’t bother with Samsung Evo SSDs, at least use Samsung Pro or something else enterprise-y that doesn’t eat your data once it hits the rated write workload (Samsung Pro is better rated and only goes read-only once that happens).

Some Dell controllers support IT mode, make sure yours does, I think h330 does iirc - or at least it can be flashed to IT mode.

You should really consider a second proxmox machine, for when the first one is being repaired or upgraded.

You can stick VM snapshots and file based backups onto a TrueNAS box with just a bunch of HDDs.

It’s a shame it’s probably to expensive to make that software work efficiently.

3

Those servers are woefully outdated and underpowered. I’d suggest the following:

The above bullet-points contain links to Aliexpress stores. Depending on your location, prices for these items will vary.

For the NVMe drives mentioned above, you can have 1TB drives for as little as USD 50-55 each. I’d recommend getting 2TB drives at USD 120-130. In a RAID6 config, those 2TB drives allow for 4TB super-fast storage for the databases and running projects (i.e. patients) while you can reuse the old 4TB HDD’s for long term storage. Do note to safely store data on those drives elsewhere before redeploying those HDD’s!

The mainboard has 6 PCIe slots, allowing ample room for the NVMe adapter card, an HBA to manage the HDD’s and a 10Gb network card between the server and the switch.

Of course, the new server needs a new case and PSU, but provided the case has enough room to house the HDD’s, a pretty standard ATX case will do, although I do recommend a 600W PSU as minimum. As an accountant, you do the math :stuck_out_tongue:

As the OS I’d recommend TrueNAS Scale. It has the stability of Linux, as well as it’s frequent update schedule so security patches are coming quick, has ZFS support (but IMO ZFS is overhyped :roll_eyes: ) and allows for Win-OS containers/VM’s to run seamlessly so the proprietary software doesn’t even know it’s not running on native Win-OS anyway :stuck_out_tongue: Additional benefit: you can easily separate the Win-OS machines from the risks of the internet by disallowing connections to and from the web to those VM’s. Patient privacy is utmost important in an industry like yours.

For backup, repurpose the T340 the owner bought recently-ish with new used/refurbished 16TB drives. The old T110 might be repurposed as an off-site backup target for added redundancy, again with 16TB drives. Hint: make the initial backup at the office before deploying the T110 to its external location. It’ll save you days of copying time :wink:

HTH!

1 Like
4

That is some freaky shit going on.

First off - that hardware, today, is ancient and probably has a value of less than $1000. You could replace all of it with a budget machine and it would run both use cases just fine, in fact, with at least twice the speed, this would solve everything pure performance wise (though not storage wise):

However, a medical setup? That’s where things get a bit… Tricky. I do know there are some stringent requirements on these things working with minimal downtime. Therefore, I would, absolutely, contract Dell or other computer vendor in this situation for the hardware. So my first piece of advice is:

Order the hardware from a vendor (Dell, HP etc)

It is not that I do not think you will not do your job here to the outmost of your ability - you taking the courage and just discussing this here on this forum shows more dedication than many would be willing to give. It is that Dell can provide with 24 / 7 support all days of the week and when shit hits the fan at 3 am on a saturday morning and three people need their MRI scan now or the clinic will be in trouble… Calling Dell for a replacement PC within the hour is simply a life saver. Literally, in this case.

So, yes. You do need something with ECC support, because it is medical. Lives and health are never to be gambled with. And yes, you do need redundancy and reliability in this case.

So, how do we achieve this? @Dutch_Master outlined a good start, but I would go one step further.

Get a NAS

Yes, you heard me correctly. Order an actual file server that you can install the SQL servers on, and separate the data part from the desktop part. What NAS you might say, well, you could just keep using your existing 4 disk file server I suppose, but for the love of god at least install an SSD on it!

Put the medical software in a VM

You actually thought of this already, yes this is a good idea, yes there is almost guaranteed a way to make it happen even if you need HW passthrough, and this VM setup would achieve multiple things, but chief among them is the ability to just reinstall and / or replace any faulty computer within 15 minutes.

Run the VM on a cheap desktop

The last part of the plan. Like it says on the tin. Use low power desktops to run the actual medical software inside VMs. Since they connect to the server via network and SQL, very little thought need to be spent here. Have Dell come help you set this up. If you are lucky you might even have a custom Linux distro with the sole purpose of booting said VM in a RAMDisk. :slight_smile:

Hope this helps some.

1 Like
5

Before you go on a shopping spree I recommend getting familiar with the license terms of the professional software used.
Make sure the terms don’t restrict the hardware the software is allowed to run on.

3 Likes
6

Unfortunately there are far too many businesses like this. One or two 10 year old servers, a backup process that may not have been tested for years, and a shoestring IT budget.

If one of those servers died and wouldn’t turn back on right now, how screwed would your business be? I’ve seen this happen to a few small medical practises who haven’t kept their hardware up to date, or ceased paying for backup support. It sucks even more when they’re your customer, you’ve been telling them to buy a new server for years, and one day poof … there goes all the data.

So what I’m trying to say is, it sounds like your boss seriously under values IT. And this is probably the most important thing to tackle.

I was considering Virtualization with proxmox and run both software on the T340. Do you think It ll be able to handle the workload of both softwares with only 4 cores?

What is the CPU and memory usage like with the current workload? If it’s maxing out CPU, then it probably can’t handle more. Not to mention what happens to the workload while you’re reformatting the machine? It’s much safer to have a new machine to install this all on.

Just be warning licensing of windows server on VMs can be a little complex. And you’ll likely need to pay more money for running windows server as VMs, even if the server has a windows server license.

Also I’m a little confused, you’re an accountant, but you’re talking about the business as if it’s a client of an MSP. Did they come to you to update their IT, or are you employed at the business?

Hardware

As for actual hardware, if you need to buy something you’ll probably be looking second hand. I’d be looking at a Xeon E5-XXXXv4 variant. This gives you DDR4 memory, and a modern (enough) cpu.

Good luck to you.

1 Like
7

Hi,

Thank you for your response

I am aware that ZFS is transparent to windows. I guess I did not phrase my question correctly.

If I use Virtualization, Which is the best way to pass the ZFS pool to windows for my use case ? Virtio SCSI or Virtio Block ?
I am planning to setup three differnt pools :

1 Pool for VM OS (Windows Server) SSD based
1 Pool for SQL database and recent data Storage SSD based
1 Pool for Archive Images HDD Based

What do you think of its layout ? and how much redundancy i should use ( Raid z1 or raid Z2 ?)

Thank you so much for you input.

8

Hi,

Thank you so much for your response,

I am currently negociating with my boss to invest in a new server. However, If I am going to that, Isnt Proxomox a better option for virtualization ?

Thanks

9

Yes, it’s a viable alternative.

10

Thanks

11

Hi,

Thank you for your response,

I totally agree with you and the owner is fully aware of the risks of not investing it proper IT infrastructure. However, since he is a doctor and does not have time to deal with this and has been scammed so many times by IT professionals, he is a bit sceptical as to investing more money.

I am currently employed at the center, I just use the term client for simplification. This whole server infrastructure happened because I was able to solve some minor IT issues for them and my boss just tasked to look into what we need to improve the speed and reliability of the software they are using.

As for your question, the CPU utilization is around 40 to 60% during heavy use. So i guess, a new system is mandatory.

Could you please elaborate on why we would need to pay more more for a windows licence if we virtualized it ?

Thanks

12

Hi,

Thank you for your time and response,

Unfortunately, I live in a country ( Morocco) where dell does not provide ongoing support for their devices ( software wise) for small businesses. Most of their hardware is sold through third party seller. This dell server was acquired through the recommandation of the medical software company. But honestly, It just seems that they are working with the hardware seller to extort more money from their customers as they have very little knowledge about hardware, servers…

The setup you mentioned is intersting as you suggested mostly consumer grade component. I was actually thinking of building a server with consumer part. Do you have any recommandations for something a little more powerful than you suggested ( Mostly if should go with AMD or Intel and which plateform). I would like to retain some enterprise feature like ECC and preferably a motherboard with IPMI

Thanks

13

Ok, I have a better view of your use case then.

So given that Dell / HP / IT in general sounds pretty corrupt in your country (sounds like a golden business opportunity to build a quality Mom & Pop store that provides quality IT though), here is my approach in more detail:

Hardware: NAS

Ideally, I would like to recommend something like the Asustor Flashstor all m.2 SSD NAS, because let’s face it spinning rust is simply reaching end of life and I think two mirrored 8TB NVMe SSDs plus an OS drive should cover all your use cases.

However, the Flashstor does not support ECC and this is probably more expensive than you can afford right now in either case. The market will need a couple of years to bring out proper support for m.2 / E* form factors. In the meanwhile, all-NAS continues to be too janky for your use case. You want stability and tried and trusted here, with as little jank as possible.

So, let us build a proper traditional server, this time. Something like this would fit your bill:

Type Item Price
CPU AMD Ryzen 5 7600X €209,00
Motherboard ASRock Rack B650D4U 1GbE €406,97
RAM Kingston FURY Renegade Pro RDIMM 16GB DDR5 6000 MT/s CL32 €81,36
RAM Kingston FURY Renegade Pro RDIMM 16GB DDR5 6000 MT/s CL32 €81,36
Storage Kingston KC3000 512GB m.2 2280 PCIe 4.0 €65,04
Case Fractal Design Node 804 €112,89
Power Supply be quiet! Pure Power 11 400W €49,90
Total €1006,52

You could possibly find a cheaper setup than this, but not by much, and modern hardware is almost always better to buy with warranty and all intact.

Hardware: Clients

This one is by far the easiest part. Just get one or more of these (or better):

Software: Server

Any regular NAS solution will do, yes to ZFS and do consider your hardware options. Just make sure you install an SQL server here, too. This will be your backend and primary hot backup server.

Of course, backups, backups, backups. The 3-2-1 scheme still holds today.

Software: Clients

As a first step, install Windows on these and set up a local SQL and backups to the NAS, and you are golden.

The second step is to look into virtualisation, and immutable/image based distros like Yocto or NixOS.

Third step, run all of this in an 8GB RAM disk and load images from the NAS, then you won’t ever need to care about a faulty install ever again.

Is all of this easy to set up, no, it will take a while to get right and will require quite a bit of research, but it can be done and in steps, too.

Other

Make sure you invest enough money to keep a spare machine available for the NAS. Be aware that you are treading unexplored territory here and you should take a long look at IT practices in medical facilities in your country. This is going to be a long journey, but it will be worth it in the long run.

The main takeaway is to rely on cheap throwaway hardware with backups instead of a single fat server that must not fail - it is a whole lot easier with a setup that has replacement parts and spares. If you have two identical machines then if A goes down, you can easily do phorensics on A while hooking up B, then repair A and have a spare ready to go once machine B kicks the can.

How do you justify the expense? Easy. How much will it cost you to NOT have these systems working, and how frequent will it be that the systems will be down? If the answer

All of the above is my own opinions, they should not be taken as facts, just advice meant to help guide you to a good solution for your environment. Due diligence is required. I hope the above was helpful! :slight_smile:

14

@wertigon
Except that it wont even boot because you’ve matched incompatible parts and several chocies are either poor or makes little sense.

The ASRock has poor aftermarket support at best, AGESA 1.0.0.7c was released about half a year ago and there have been several iterations since.

Why stick a 105W TDP CPU in a tiny case on there when you can get a 65W with minor performance penalty and still have a lot of headroom?

RDIMM wont work…

@ismailben
There’s no domest software support(?) for any vendor? You just go to www.insertvendorhere.com and it’s done? You pay for testing, certification and that X works with Y and Z which can be a very expensive lesson otherwise among other things. All these companies do much more testing etc than what an induvidual can do and that comes with a price.

15

I think we already established OP is in a spot where aftermarket support does not matter? But feel free to give a better recommendation if you think the above one sucks. The B650D4U has two m.2 and three PCIe ports plus good reviews, which is the main reason I am recommending it. :slight_smile:

You are right, I checked briefly and the 7600 was more expensive but today it is cheaper. Either works just fine in that motherboard, though.

My bad, I was under the assumption that the B650D4U server motherboard would support Registered ECC RAM. Guess not, but that is an easy fix then:

New total is €974,58.

16

I wouldn’t want that in a medical center. You need to be secure in a situation like that. So i think the budget route is the wrong route.

You need at least a machine that encrypt the shit and all the consumer machine’s don’t have enough pci expres. to get enough hardisks in there to saturate a 10 gb network. Yeah it might for one user but not when there are more users.

https://www.intel.com/content/www/us/en/architecture-and-technology/intel-quick-assist-technology-overview.html

you vind this on xeon d with N in there name. And if you don’t need the quickasist The boards without a N are also a better pick for building a server like this.

On xeon scalebale 2nd gen its on the chipset c624 or c648 and some more. you can find those machine’s on motherboards.

on xeon e machine’s its mostly not availble but you can get them on add in cards.

Searching for those machine’s on the second hand market is a pain in the B,

In a medical centrum i wouldn’t like going for hotsswappelble disk. because you want to protect your data.

doctors shouldn’t use usb disk for backup.
Doctors need a system with proper security. .

If doctors think they can do this on a system of a 1000 euro’s, These doctors should check there brain.

I don’t care if somebody’s pictures of cats are getting stolen. People really hate it when there medical records are not proteced and end up on the street

17

Hi. I’m a radiologist. I would not take on a responsibility such as this as it is frankly not your job and you do not fill the technical requirements for taking in this job. Some thoughts:

  • if something goes horribly wrong (cybersecurity incident, data loss due to hardware failure/ransomware, etc) you will be caught in it and be the one to blame. You definitely do not want to be sued for this.
  • you are rewarding the bad behavior of the boss (not willing to spend properly on IT infrastructure and IT people).
  • you may be called for IT problems during the night due to being a shadow IT. You are not compensated for this.
  • There are a lot of smart people that can help you here but the knowledge you can gleam from here definitely does not replace a proper IT personel and new/better IT equiptment.
  • The owner being “scammed” by IT is non-sense, just because the owner is a doctor and dont really know how enterprise IT works, it doesnt mean you are being scammed. There needs to be proper and clear communication between the owner and IT. Enterprise class equiptment is expensive for a reason.
  • Anyone of us can spew out machine specifications that may seem “good enough” but a good IT can point you to a true cost effective solution without overspending on parts that you dont need and skimping on the parts that you need.

Remember that at worst. you are potentially going up against bad actors/hackers that want to exort money from the low hanging fruits, sadly it means us in the medical industry. The days of cheaping out on IT infrastucture is gone since the boom of cryptocurrency and rise of ransomware. Your boss really have to pay for real IT and real, up to date equiptment. While I do applaud your desire to improve the IT, you should not be the one responsible for this because, you are not IT.

The alternative is not pretty. And you might as well go back to actual paperworks. There is no shame in that (we still have paperworks here) The PACS systems can work completely offline unless the radiologists want to work at home, in which they shouldnt probably allowed remote access because proper cybersecurity is hard.

4 Likes
18

@regulareel is correct, liability is a big deal.

… but you could pay for “proper” hardware, all supported part numbers e.g. 50k once off for a pair of machines (will last for 3-5y) + 20k sysadmin, or you could diy maybe a better cluster for 10k-15k total.

Either way, you need to go for a pair of machines, don’t be stupid and put everything on one box, and don’t be stupid and not have offline backups.

Regardless of how much warranty you get - it’s not a replacement for working machines.

@wertigon is correct too, your needs are simple enough to run on a desktop class machine , *d4u ASRock boards are especially nice because you can remote into them. I don’t know / not sure, if you want to DIY them yourself into a case. I’d do it, I trust myself to do it well more than I trust Dell/HP whoever, but … that’s after hundreds of computers built in my lifetime and only if I had time to spend.

Supermicro / Gigabyte servers are another option, mostly assembled, you can maybe get them preconfigured from a vendor with whatever is the cheapest Milan era EPYC and 64G-128G ram each - probably good enough.

But in either case, you’re assuming a role of maintaining a thing, how sure are you that you want to take this on, instead of bring someone in on a contract?

1 Like
19

Hold it folks! You’re roasting the OP for a number of things that in the West, would be unwise or even unacceptable. But the OP is in Morocco, which in IT terms especially, is pretty much a 3rd World nation. What you take for granted as you’re in North America or (N/W) Europe is not so common south of the Strait of Gibraltar. So if the OP states IT stuff is corrupt in his country then yes, that’s because it is :roll_eyes: Unfortunate, but harsh reality for many nations on the planet. Just not the one you happen to live in. :globe_with_meridians:

(FYI: my country has a large contingent of Moroccans living here and I’ve worked with quite a number of them. With mixed experiences I’d have to say but that’s beyond this discussion. Period)

That said, there are valid points made by various contributors so I hope the OP will take these to heart and help with his decision-making processes. :green_heart:

3 Likes
20

This. At the end of the day, there are two aspects here, the technical aspect, and the sociological / political / legal aspect. Technically this is not a very hard problem to solve IMO and there is off-the-shelf hardware and software available, but policy wise it can easily become a minefield.

If it is a question between having a jank setup that works but is held together with duct tape and chewing gum, and not working at all, then I have full understanding for choosing the jank route. It will bring trouble down the line, and it will create a lot of problems for the poor soul that has to maintain this, but as a temporary solution while waiting for the clinic to generate enough money to do this properly? Yes, I can see why that route would be considered.

If Morocco is corrupt, perhaps you could try to reach out to an IT consultant in the EU, like France or Germany perhaps? The on-site help they could provide would probably be invaluable.

1 Like