So I made a Script with Node, here is the script and some questions

Development Software Development
,
1

Some Explanaition

Currently at a mission to find out witch scripting language to use to build myself little custom commands that do something.
Many times when it’s simple shell is fine. But I’m not very comfortable using it for anything larger. When you have to google your head off to make simple if/else statements, loops and comparisons it’s not so very fun. For a while I thought I’m better off shell scripting, because then I can be better at terminal. But I kinda stopped believing that.

Python seems popular for scripting workloads. But I don’t know python yet. So I went to node.
I can partially see why people might prefer python over node for this as it’s not particularly helpful to have an async library for reading user inputs from the terminal within your often very synchronous script (since what I’m using this for is mostly do A task bottom down instead of me writing the 10 things I need to do).

But otherwise it’s pretty good. It’s way easier to make it platform independent compared to shell. And Im more used to it and don’t have to google everything.

So I said I wrote a script what does it do?

I still had some series around that I never bothered to rename in a way that I can put them into plex. Like say naruto. I’m not really planning on watching it again tbh, but whatever I had it might as well rename the 200 files or whatever there are.

The script works kinda like this
You have
‘Naruto 1 - Somename.mkv’ //Dang it I sure had a lot of time back then to type out all those episode names lul
…
‘Naruto 21 - Some other name.mkv’

Now I want it to be ‘Naruto s01e001 - the name.mkv’

So now thanks to my script I can
rename -y 'Naruto {ep} -{epname}.mkv' 'Naruto s01e{ep:3} -{epname}.mkv'

And that’s it 200 episodes renamed. (it’s all the same season now, but it was before and it’s ok with me)

Partially, I feel like someone here is as soon as they see the code work on their 1 line shell command that does what I did in a 343 lines node script…
Well I did not anticipate it to be this long either. But I also kinda don’t see how you can make it a one line script either. Unless you use some prebuilt program. Of course they probably exist. And maybe some shell command can do this too. But oh well… I would have downloaded some gui to do this for me if it was really that important to have this done I would have done this half a year ago when I installed plex. However, i think it’s neat and I might be able to use this for other things.

The damn script

Feel free to use it and tell me if you broke it. :slight_smile:

const fs = require('fs');

const readline = require('readline').createInterface({
    input: process.stdin,
    output: process.stdout
});

/**
 * Excapes a string for use in regular expressions, thank you stackoverflow.
 * @param {string} str 
 */
function escapeRegExp(str) {
    return str.replace(/[.*+?^${}()|[\]\\]/g, '\\$&');
}

class SegmentedPath {
    
    /**
     * 
     * @param {string} path 
     */
    constructor(path) {
        this.segmentPath(path);
        this.regex = this.buildRegex();
    }

    /**
     * Builds the SegmentedPath.
     * @param {string} path 
     */
    segmentPath(path) {
        this.segments = [];
        let str = '';

        [...path].forEach((val, i, arr) => {
            if(i == arr.length -1) {
                if (val == '}') {
                    if(str != '')
                        this.segments.push(new Placeholder(str));
                }
                else {
                    this.segments.push(str + val);
                }
            } else {
                if(val == '{' && str != '') {
                    this.segments.push(str);
                    str = '';
                } else if(val == '}' && str != '') {
                    this.segments.push(new Placeholder(str));
                    str = '';
                }
                else if(val != '}' && val != '{'){
                    str += val;
                }
            }
        });
    }

    /**
     * Builds the regular expression for matching paths in 'ls' to the SegmentedPath.
     */
    buildRegex() {
        let regex = '^';

        this.segments.forEach(val => {
            if(typeof(val) === 'string') {
                regex += escapeRegExp(val);
            } else if (val instanceof Placeholder) {
                regex += '.*';
            }
        });

        //TODO: maybe add alert if there is Placeholder at the very end more files may be selected than anticipated by the user.

        regex += '$';

        return new RegExp(regex);
    }

    /**
     * Mainly for checking if this worked. May be removed later.
     */
    toString() {
        let str = '';

        this.segments.forEach(cur => {
            str += (cur instanceof Placeholder ? cur.toString() : cur);
        });

        return str;
    }

    /**
     * Feeds all the valid paths into the SegmentedPath to determine Placeholder
     * values from.
     * @param {Array(string)} paths
     */
    feedPathValues(pathsArray) {
        this.pathCount = pathsArray.length;

        pathsArray.forEach((path, pathIndex) => {

            let pathCopy = path; //TODO: should probably change this path instead of the path array, but technically either works fine

            for(let i = 0; i < this.segments.length; i++) {
                if(typeof(this.segments[i]) === 'string') {
                    path = path.replace(new RegExp(`^${escapeRegExp(this.segments[i])}`), '');
                } else {
                    let val = '';
                    if (i == this.segments.length - 1) { //Placeholder at the end of the path
                        val = path;
                        path = '';
                    } else { //Placeholder somewhere within the path
                        const regex = new RegExp(`^${escapeRegExp(this.segments[i+1])}`);
                        while(regex.test(path) == false && path.length > 0) {
                            val += path[0];
                            path = path.substr(1);
                        }
                    }
                    this.segments[i].pushValue(val, pathIndex);
                }
            }

            if(path.length != 0) { //TODO: Toggle some flag to skip this path automatically
                console.log(`The leftover length of path is expected to be 0, but was ${path.length}.`);
                console.log('Proceed with caution.');
                console.log(`Especially for path: ${pathCopy}`);
            }
        });
    }

    /**
     * Copies the placeholder values from another SegmentedPath
     * @param {SegmentedPath} from 
     */
    copyPlaceholderValues(from) {
        from.segments.forEach(fromSeg => {
            if(fromSeg instanceof Placeholder) {
                this.segments.forEach(toSeg => {
                    if(toSeg instanceof Placeholder && toSeg.name == fromSeg.name) {
                        toSeg.valuesMap = fromSeg.valuesMap;
                    }
                });
            }
        });
        this.pathCount = from.pathCount;
    }

    getPaths() {
        let arr = Array(this.pathCount).fill('');

        Array.prototype.appendPlaceholder = function(placeholder) {
            for(let i = 0; i < arr.length; i++) {
                if(i in placeholder.valuesMap)
                    arr[i] += placeholder.process(placeholder.valuesMap[i]);
            }
        };

        this.segments.forEach(seg => {
            if(typeof(seg) == 'string') 
                arr.forEach((cur, i, arr) => arr[i] = cur += seg);
            else
                arr.appendPlaceholder(seg);
        });

        return arr;
    }

    moveToInteractive(to, saveMode) {
        to.copyPlaceholderValues(this);
        let pathsFrom = this.getPaths();
        let pathsTo = to.getPaths();
        
        let i = 0;

        if(saveMode == false)
            console.log("Save Mode is disabled.");

        const question = 'Continue? [y/n]';

        console.log(`Moving file ${pathsFrom[i]} to ${pathsTo[i]}`);

        let onInput = (input) => {
            if(input && input == 'y') {
                fs.renameSync(pathsFrom[i], pathsTo[i]);
                //execSync(`mv '${pathsFrom[i]}' '${pathsTo[i]}'`);
                console.log(`   moved '${pathsFrom[i]}' to '${pathsTo[i]}'`);
            } else {
                console.log('   file skipped');
            } 
            
            i++;

            if (i < this.pathCount) {
                console.log(`Moving file '${pathsFrom[i]}' to '${pathsTo[i]}'`);
                if(saveMode) readline.question(question, onInput);
                else onInput('y');
            }
            else readline.close();
        }

        if(saveMode) readline.question(question, onInput);
        else onInput('y');
    }
}

class Placeholder {
    
    /**
     * Constructs a placeholder. 
     * @param {string} name 
     */
    constructor(name) {
        let split = name.split(':');
        
        this.name = split[0];
        this.valuesMap = {};

        if(split.length > 1)
            this.length = split[1];
    }

    process(val, length) {
        let len = length === undefined ? this.length : length;

        if(len == undefined)
            return val;

        let out = val + "";

        while (out.length < len)
            out = "0" + out;

        while (out.length > len)
            out = out.substr(1);

        return out;
    }

    pushValue(val, index) {
        this.valuesMap[index] = val;
    }

    toString() {
        return `{${this.name}${this.length === undefined ? '' : ':' + this.length}}`;
    }
}

//*********************
//*** Pretty Output ***
//*********************

function print_usage() {
    console.log('**************************');
    console.log('*** Usage of rename.js ***');
    console.log('**************************');
    console.log();
    console.log('Example usage:');
    console.log(`node rename.js 'Somepath to Files s{0}e{1}.mkv' 'New s{0}e{1:2}.mkv'`);
    console.log();
    console.log('On the left side placeholders are defined and given a number, or text {episode} would be totally valid. No duplicates allowed though.');
    console.log('On the right side the placeholders can be used and with {episode:2} it will make sure it has at least 2 digits so 1 gets 01.');
    console.log('Placeholders can also be shortened so formatting 003 with :2 will result in 03.');
    console.log();
    console.log('flags:');
    console.log('-y : Just move the files without asking for confirmation.');
    console.log();
}

//*************************
//*** The actual Script *** 
//*************************

let arg1 = process.argv[2];
let arg2 = process.argv[3];
let arg3 = process.argv[4];
let saveMode = true;

if(arg1 == '--help' || arg1 == '-h') {
    print_usage();
    process.exit(0);
}

if(arg1 == '-y') {
    saveMode = false;
    arg1 = arg2;
    arg2 = arg3;
}

if(arg1 === undefined || arg2 === undefined) {
    print_usage();
    process.exit(1);
}

console.log(`Argument 1 [path]: '${arg1}'`);
console.log(`Argument 2 [newPath]: '${arg2}'`);

let fromPath = new SegmentedPath(arg1);
let toPath = new SegmentedPath(arg2);
let toMove = [];
let fromPathStr = fromPath.toString();
let toPathStr = toPath.toString();
let errors = 0;

if(arg1 != fromPathStr) {
    console.log('Something went wrong parsing your path variable. It no longer matches your input.');
    console.log(`Your input [arg1]: ${arg1}`);
    console.log(`Was parsed to: ${fromPathStr}`);
    errors++;
}

if(arg2 != toPathStr) {
    console.log('Something went wrong parsing your toPath variable. It no longer matches your input.');
    console.log(`Your input [arg2]: ${arg2}`);
    console.log(`Was parsed to: ${toPathStr}`);
    errors++;
}

if(errors > 0) {
    console.log(`Seems like ${errors} critical error${errors == 1 ? ' has' : 's have'} occured. Exiting program.`);
    process.exit(errors); 
}

console.log(`\n Looking for files to move matching: ${fromPath.regex}\n`);

let files = fs.readdirSync('.');

files.forEach(cur => {
    if(fromPath.regex.test(cur)) {
        toMove.push(cur);
    }
});

if(toMove.length == 0) {
    console.log("No files are matching the pattern. Got nothing to do...");
    process.exit(1);
}

console.log(`Found ${toMove.length} file(s) to move.`);
if(toMove.length < 10) console.log(toMove);

fromPath.feedPathValues(toMove);
fromPath.moveToInteractive(toPath, saveMode);

Some question(s) about managing those script(s)

What are you using for your scripts? Node, Python, Shell? And how do you make them readily available to not have to type out the entire path to the script and some programm (in this case node) to launch them. It’s not that much yet it’s currently mostly ‘work in progress’.

As of right now I simply put an alias into by bashrc being alias rename='node ~/scripts/node/rename.js' while this works perfectly fine I don’t really want to create a bijillion aliases for every single thing.

What do you think is the best way to do this? So I can have multiple scripts every one of them accessable without typing out the entire node “path to index-script.js”.

One possible solution would be imo to put the scripts into sub folders named by their alias name. And then make something that combines all the script in the folder (webpack?) put the resulting script(s) into another folder bin/compiled/transpiled/i don’t care. And then generate some partial .bashrc-node-scripts with all the aliases generated into it. That I then can include into my bashrc.

Though I don’t think this is the most strange usecase ever for node. There may be a better very readily done node package available that just do all of this. Or creates executeables from my scripts, so I can just include the path of the folder I put them in and be done.

Git multiple repos in one repo (some private some public)

Since this and all the other script-like things I ever did are currently in a private gitlab repo that I don’t want to share with anyone (because it also contains some potentially privat information).

I wonder if anyone has done something to put a subdirectory of a repo into another repo to be able to have part of it be easiely shareable and the rest be private.

I’m not really sure yet if this is possible. But since I wrote a roman I thought I might as well throw this in. Don’t really expect much and I have not looked into it yet. It might not be possible. It might be possible.

Gitlab also has a devops pipeline. I (probably) can use that to push specific folders to a public github repo. I’d rather have it be github than my server then. Because unlike my little gitlab server they can serve an infinite amount of people. On my server I get 50-100% cpu spikes from browsing repos with a single user sometimes.

1 Like
2

Node.js has sync versions of a huge number of functions (e.g. fs.readdirSync()), so you can just use those if you want to build synchronous utilities.

For the functions that don’t have an inbuilt sync version, there’s always libraries/modules like https://www.npmjs.com/package/sync and https://github.com/alexeypetrushin/synchronize (and others) to choose from.

As far as making those utilities easier to call, just add PATH=$PATH:~/your/utility/directory to your ~/.profile (or ~/.bash_profile if you only use bash). No more paths required. If you want/need to have your utilities scattered all over the place, you can fill that directory with symlinks to them and it will still work.

3

I know that very well. But there isnt for reading console inputs at the very least not included in node. I found it silly to npm install a module for this one script. So I just rolled with the async function. But I could also just do it async. Could have advantages if done right. In my case though it absolutely does not. Though since im probably gonna do more of that I should go find some pretty output libs.

Kinda, but the you still have to do node script.js instead of script.

4

If you make the first line of your script…

#!/usr/bin/env node

…then neither node nor .js are needed on the command line.

1 Like
5

Oh, thats neat!

Gonna try that today when I get home. Thanks for the tip.

2 Likes
6

If you want to share things between your scripts, keep them in one repo, it’s easier to develop.

git supports shallow and narrow clones, if you need them. You can probably come up with a script to package your script into a distro package you can install on your favorite distro.

7

I wasn’t really thinking about deviding the repo into 2 in a way that I won’t have all of them in the private repo. That might turn out unconvinient (as you say). But rather creating a copy of select things either whitelisted or blacklisted on file or/and folder basis to another repo on github.

I don’t see a problem with that, because if some script depends on anything that I want to keep private it’s likely I’ll want to do the same with the script that depends on it. So it won’t break things. It might on the public repo if I mess up things, but not on the private one.

This works fine on both gitbash on windows and fedora. Both still need the .js at the end assuming you call the file that. And your user must have permissions to execute that .js file. But that’s a great uncomplicated solution for sure! :slight_smile:

I already had lot’s of #!/usr/bin/env bash in my sh files, because gitbash needs them for those to be executeable. Since there is not executeable permission on Windows (I don’t think) that’s how it handles it. But I never figured that’s actually good for something else too… :sweat_smile:

1 Like
8

For decades I wrote all my utilities in bash/Perl. Now I write them all in bash/Node. Can’t even remember the last time I’ve needed an extension on a Linux distro — it’s been a while. Currently using Ubuntu and no extensions are required on that.

If anyone finds themselves on a platform that requires an extension on the file itself, it might be possible to side-step the need to type the extension on the command line by using a symbolic link:

$ ln -s /path/to/myNodeUtilityWithAnExtension.js shortName
$ ./shortName

Yep, execute permissions are required (naturally). If you want/need multi-user access to your utilities, then you could put copies in /usr/local/bin/ and chmod g+x them. No need to modify anyone’s PATH that way.

I’d never heard of gitbash before — sounds neat.

9

I tried this when I started with the bashrc madness. Does not quite work on Windows/gitbash. But on Linux this is fine (any linux). And you dont then end up with your library things being in the path either.

In gitbash its seems to just copy the file so ln does not actually ln. Everytime I update something in that file it works everywhere, except on every Windows install I have to run that again.

Im not sure if I like removing the file extension entierly from the file itself. But it seems to be fine since vscode can still figure it out too. So i just did that now anyways. Cause its simple.

10

I’ve now found another way to do this using npm you can define executeable js files in the package.json.

  "bin": {
    "rename": "./rename.js"
  }

Then you can link them with npm link and call them from anywhere with the name at the left side.

I think it’s cleaner since you can call your files anything that makes sence including .js, still have the comfort of being able to define commands that you can use globally with a name you can choose. And you don’t gunk up your cli with js files that are merely dependencies of a larger script.

If you write things that need sudo (had a few where I needed to stop a service then do something, then set some permissions to the user it should technically probably run as, but does not and then start the service again… update script of sorts). You can install those with npm install -g (or sudo npm install -g in my case as my user has it’s own global node_modules directory because you can’t access the default global node_modules directory with your user).

1 Like