Managing other processes with python?

Dev Zone Code
, ,
1

Hello there, was wondering if I could get some feedback on what the best approach to a particular use case I’ve got going on would be.

I’m starting to look into rewriting this little bash based Minecraft server manager I wrote years ago (and never finished) in python.

Background Info Not super important, but explains my thought process

The old implementation uses the screen program extensively to spin up server processes in a way that they are not child processes of the original script and also still able to be attached to if needed.
I’ve found this implementation with screen to be rather clunky, and that it conflicts a little with how the terminal works with minecraft servers (for example, I can’t scroll in the terminal. It instead seems to send up arrow key inputs).

Not to mention I can’t even begin to yet again wrap my head around the bash syntax. I don’t remember much about when I developed my original bash script but what I do remember was me and a buddy sitting in a discord call for two nights straight bashing out heads against a wall figuring out the syntax. I don’t want to deal with that again, hence my desire to try this over in python.

I’ve decided I do not want to rely on an external library to manage interacting with the java processes anymore, and that I want to get my hands a little dirty and manage reading their output and sending input to them myself.

However, I’m not comfortable with the idea of the java processes being children of the python process. My idea is to spin up the java servers as background processes, directing their console output to some sort of temporary file (either in the server’s main directory, or in a temporary file in /var/tmp). Then I can simply run the python script whenever I need to start/stop a server, read console output, or send console commands.

I want to approach it this way because I want to make it easy for me or my friends to ssh into my server, run a script, and just be able to perform whatever action is needed. No having to load a background process or screen session to get to an already running python script. To me this seems like the simplest and easiest way, though that could be from simply naivety.

What I’ve found so far and what led me to post here

I’ve found this stack overflow post detailing a lot of different methods to potentially achieve what I have in mind, though the answers cover many years and many different approaches. I was wondering if anyone here might be able to give me some guidance as to what rabbit hole of learning I should start with considering my use case. Since it seems there is several ways of skinning this cat and I’m not really sure where to start.

To Summarize

I’m stuck in some analysis paralysis unsure of how/where I should direct my efforts on this adventure. I’m familiar with python and java, but threading, sub-processes, etc. is a realm I’m wholly unfamiliar with, and would like some help getting pointed in the right direction or (if applicable) why my dreamed up approach is misguided.

Thank you in advance to anyone who might be willing to help me out (=

2 Likes
2

Sorry if this is total BS, I never played Minecraft or hosted a Minecraft server. Just some outside perspective.

When I read this I think you are trying to do two things at once. I would handle spinning the services up and down to systemd, it will also take care of the logging.

For controlling the running servers I found:

If this suites your needs you would have an elegant solution to control the servers.

You could still write a script to create some command line interface for controlling the different services, but you wouldn’t have to deal with subprocesses and parsing their output.

The other thing that comes right into my mind would be docker. Spinning up containers, attaching to them, reading the logs, etc.

3

Theres a lack of replies here. Let me see if I can help. Full disclosure this is only a suggestion and not meant to be your full solution or even the direction you should take. I am just going to spit ball something

So here is what I propose you do. Your idea of starting Java server processes as separate background processes instead of child processes is valid. This way, you can avoid complications with process termination and can manage them independently. You dont want one to crash and then the parent crashes. The child should be able to crash and the parent should be able to handle it. Just like IRL :wink:

Now you need a way to log the console output of java. I always hated that it was done poorly with minecraft servers for example. Redirecting the console output of the Java processes to temporary files is a good way to read logs in my opinion. You might also consider using a more structured logging framework for the servers themselves if you want more control over log management. An example would be taking the ogs and putting their standard output into a syslog server and collecting them withh some logging software. Have it process metrics and display them with grafana for example… ANyways thats scope creep. But you get my point.

Also… just like the output you can manage sending commands to the Java processes by writing input to their input stream, which you can manage in python rather trivially. I do have some suggested libraries that you can start with and heck I might as well throw down some example code.

The first library is named subprocess. Essentially it allows you to spawn new processes, connect to their input/output/error pipes, and obtain their return codes. It provides a lot of flexibility in how you create and manage those processes. OBVIOUSLY you will need the os lib for handling file paths and managing temporary files and If you want to create a responsive system while waiting for output from your servers, threading can help. This can be useful if you want to listen in on outputs from multiple servers simultaneously. I imagine from your background information that you might be doing so.

Example Code:

import subprocess
import os

def start_server(server_name):
    log_file = f'/var/tmp/{server_name}.log'
    with open(log_file, 'w') as f:
        process = subprocess.Popen(
            ['java', '-jar', 'server.jar'], 
            stdout=f, 
            stderr=subprocess.STDOUT, 
            cwd=f'/path/to/{server_name}',
            start_new_session=True  # Start a new session
        )
    return process  # Save the process ID if needed

Example output reading code:

def read_output(server_name):
    log_file = f'/var/tmp/{server_name}.log'
    with open(log_file, 'r') as f:
        # Move to the end of the file
        f.seek(0, os.SEEK_END)
        while True:
            line = f.readline()
            if not line:
                break  # Exit if no new lines
            print(line.strip())

You can also send commands by opening a new subprocess with stdin attached to your Java process if your server accepts input from standard input.

I hope you find this helpful tbch

Best of luck with your project!

4

@PhaseLockedLoop will kill me for the suggestion, but.

If you are using a standard Linux distribution, pretty much all the functionality is already built into the system manager, systemd - it can spin up processes, redirect stdin and stdout just fine.

On top of that, systemd has an extensive D-Bus API, which allows other processes to connect to it and ask it to do things.

That is my first idea here - don’t reinvent the wheel. There is already much more powerful thing on your system that is capable of doing what you want. You just need to talk to it.

1 Like
5

This is an option but make sure programs perform std output so systemD can log it lol

That too. I hate daemons but they work

I hate Python too but if someone wants to use it. Im not going to stop them