Devember - JRiver wireless (automatic) syncing APP for Android

I´m gonna join on this Devember challange too this months. :slight_smile:
Don´t know if I´ll get anywhere with it. But I would sure hope so!


Project Details

Short Description

Sync your Music library from a JRiver instance somewhere in your local Network, including Playlists, threw your local WIFI. In the best case this would work completely without user interaction. At least when you´re in your known home network. I don´t want it to be looking for the server when I´m miles away from home. I would like it very much if music just gets copied over the moment I add it to my JRiver library. So I have it with me when I leave the house without the possibility of me forgetting about having to plug my phone into the computer. It´s also important for me that the program does not unnessesarily resync files. The first time might take forever (it´s WIFI). But once you have your library synced it should be much MUCH quicker as you´ll usually have only 1 album or song to sync and nothing more.

Project Name: JDrop (??)

I´m thinking maybe “JDrop” as in dropping the JRiver into your phone “drop by drop” or something like that. You have any better ideas? I´ll just call it that for now. Can always change it.

Platform

  • Android
    Witch version I don´t know yet. At least Android 9 lul. I´ll experiment with lower versions once I have it working to begin with.

Programming Language

  • Kontlin
    Maybe some Java, but planning to stick with Kotlin

What I got so far?

Mostly theorizing what I´m actually gonna do or can achieve.

JRiver hosts a local API on your Server. Can´t really show you the docs as the docs are also hosted locally and not in some wiki on the web. There is a wiki and it tells you open http://localhost:52199/MCWS/v1/doc and replace localhost with whatever.

Useful JRiver API calls

Those are some of the API methods that I found, witch I will likely be using to retrieve the music.

Get the Files

 Files
 
      Search
         Perform a database search for files.
         Parameters:
            Query: The search string (empty returns full library) (default: )
            Action: The action to perform with the files (MPL: return MPL playlist; Play: plays files; Save: saves the files (as a playlist in the library, etc.); Serialize: return serialized file array (basically a list of file keys); M3U: saves the list as an m3u). (default: mpl)
            Shuffle: Set to 1 to shuffle the files. (default: )
            ActiveFile: A file key to set as active (used as the file that playback starts with, etc.). (default: -1)
            ActiveFileOnly: Set to 1 to trim the returned files to only contain the active file. (default: )
            PlayMode: Play mode flags delimited by commas (Add: adds to end of playlist; NextToPlay: adds files in the next to play position). (default: )
            Fields: The fields to include in an MPL (use empty to include all fields). (default: )
            NoLocalFilenames: Set to 1 to filter out local filenames from MPL output (since they might be meaningless to a server). (default: )
            PlayDoctor: Set to 1 to change the files to a Play Doctor generated playlist using these files as a seed. (default: )
            SaveMode: Playlist: playlist (overwrites existing; returns ID) (default: )
            SaveName: A backslash delimited path used with the action 'Save'. (default: )
            NoUI: Set to one to put the player in no UI mode. (default: )
            Zone: The zone the command is targetted for. (default: -1)
            ZoneType: The type of value provided in 'Zone' (ID: zone id; Index: zone index; Name: zone name). (default: ID)
         Response:
         Examples:
            Click here

File
 
      GetFile
         Get the contents of a file in the database.
         Parameters:
            File: The key of the file. (default: -1)
            FileType: The type of value provided in 'File' (Key: file key; Filename: filename of file). (default: Key)
            Helper: Allows getting sidecar / helper files (used internally). (default: )
            Conversion: The conversion settings to use. (default: )
            Quality: The conversion quality to use (low, medium, high, etc.). (default: )
            Resolution: The resolution of the target device (allows making better conversion decisions). (default: )
            AndroidVersion: The Android version of the target device (if applicable). (default: )
            Prepare: Set to 1 to prepare the file (useful when waiting for video conversion, etc.). (default: )
            Playback: 0: Downloading (not real-time playback); 1: Real-time playback with update of playback statistics, Scrobbling, etc.; 2: Real-time playback, no playback statistics handling. (default: )
            Start: The start position for playback. This is normally seconds (decimal supported), but usage can vary based on playback type. (default: )
            MimeType: The mime type to use in the response (leave blank for default mime type). (default: )
            HLS: Use HTTP Live Streaming. (default: )
            Context: The context used to access the file (used for HTTP Live Streaming). (default: )
         Response:
            PercentPrepared: The integer progress percentage of a file preparation operation, such as transcoding.
         Examples:
            Click here

The search function will return all the files in the currently selected library if no search string is passed to it. With the GetFile method it is possible to download that file.

Potential problem: Audio files may or may not be compatible with Android. Especially since some of my library are m4a´s from itunes / ipod times (apples version of mp3´ish, but does not have to be mp3´ish, could be lossless too, but I know mine are not).

Get the Playlists

 Playlists
 
      List
         Gets a list of all playlists.
         Parameters:
            Group: Only return playlists within this group. (default: )
            IncludeMediaTypes: Return the media types of files in the playlist (comma separated list). Only valid for regular playlists, not smartlists. (default: )
         Response:
         Examples:
            Get list of all playlists.
            Get list of all playlists within the 'Smartlists' group.
            Get list of all playlists and return the media types.
 
   Playlist
 
      Files
         Gets the files of a playlist.
         Parameters:
            Playlist: The playlist the command is targetted for. (default: )
            PlaylistType: The type of value provided in 'Playlist' (ID: playlist id; Path: playlist path). (default: ID)
            Action: The action to perform with the files (MPL: return MPL playlist; Play: plays files; Save: saves the files (as a playlist in the library, etc.); Serialize: return serialized file array (basically a list of file keys); M3U: saves the list as an m3u). (default: mpl)
            Shuffle: Set to 1 to shuffle the files. (default: )
            ActiveFile: A file key to set as active (used as the file that playback starts with, etc.). (default: -1)
            ActiveFileOnly: Set to 1 to trim the returned files to only contain the active file. (default: )
            PlayMode: Play mode flags delimited by commas (Add: adds to end of playlist; NextToPlay: adds files in the next to play position). (default: )
            Fields: The fields to include in an MPL (use empty to include all fields). (default: )
            NoLocalFilenames: Set to 1 to filter out local filenames from MPL output (since they might be meaningless to a server). (default: )
            PlayDoctor: Set to 1 to change the files to a Play Doctor generated playlist using these files as a seed. (default: )
            SaveMode: Playlist: playlist (overwrites existing; returns ID) (default: )
            SaveName: A backslash delimited path used with the action 'Save'. (default: )
            NoUI: Set to one to put the player in no UI mode. (default: )
            Zone: The zone the command is targetted for. (default: -1)
            ZoneType: The type of value provided in 'Zone' (ID: zone id; Index: zone index; Name: zone name). (default: ID)
         Response:
         Examples:
            Click here

Those two methods I can use to get the playlists.

One big thing that I have to figure out is how not to break any of the paths for music that has non-conventional artist or album names (since paths are limited in different ways on Android than they are on Windows or Linux). It´s actually likely I will just name the music on the phone with the ID it has in the JRiver library, with sub-folders for each library (dodging the problem essentially :)). As far as managing the playlists on android there is a system wide playlist sqllite database that would work with any music player that didn´t invent their own way to do it.


Things that I will be mostly ignoring for the first working build

  • Sync Multiple JRiver Libraries (I have one… I would expect most people to have one), but I will make the folder structure in a way that this would theoretically work too. But a problem with that is multiple instances of the same song could exist in different libraries and Android music players won´t really care much about that. Say you sync yours and the library of a (boy/girl)friend or relative that lives with you. I´d say it´s probably not exactly the most practical thing ever, unless I´m also able to exclude duplicates reliably.
  • Selective syncing - I will just sync everything without exceptions. For me this is fine as my entire music library is 26.5GB (and that´s 2263 files…). So honestly I have about 59gb free space on my phone right now with all the music coppied to it. For me it´s no problem. But it might make it unuseable for many others if it ever comes to that.
  • Support for old Android versions (will be handled once I have something that works for me)
  • UI Design. Won´t do much for the start. Just 3 text boxes for server ip / username / password and a sync button. This does not HAVE TO BE a great looking program to start out with. The UI isn´t important to the functionality. So I will absolutely half ass the UI until I have working program. Clean code / code-design is way more important than pretty UI here.
  • Selective playlist syncing. For the start I won´t bother with this either. But there are a lot of automatic playlists. Some of those I have disabled in the GUI, but they are still alive and well in the JRiver database. So… I still eventually want to not have them synced to my phone. But for the very first build I won´t even be syncing playlists to begin with. One step at a time.
  • Convert lossless to mp3´s to save space. Thankfully JRiver actually has a built in mechanism for doing that and keep track of the duplicated entries. Won´t be doing that either at the start. But deffinitely a thing on the list that I want to include eventually. This is genually more useful than excluding files entierly imo.

What do I plan to do with it once it´s done?

I don´t know what I´ll do with it yet once I´m “finished”. For starters. I´m building it because I personally want it. After that the possibilities are open. I could put it into the playstore, make a public github repo, or both. Or I could gift it (maybe even sell it?) to JRiver themselves. Well see. For now I gotta make it. For now I hope I can make it work for me and everything else comes after.

2 Likes

that sounds interesting!
Good luck with your project.

1 Like

Was setting up all the things I need today to actually start doing something on the project.

An extra JRiver library with limited songs (so I don´t get old syncing and have a select amount of different formats), smb share to drag stuff into the library and all the Android studio things.

One annoyance I found out is that Android VMs ain´t happening for me right now as I have a RYZEN processor and with AMD you require

  • AMD Processor - Recommended: AMD® Ryzen™ processors
  • Android Studio 3.2 Beta or higher - download via Android Studio Preview page (it´s no longer in beta now)
  • Android Emulator v27.3.8+ - download via Android Studio SDK Manager
  • x86 Android Virtual Device (AVD) - Create AVD
  • Windows 10 with April 2018 Update
  • Enable via Windows Features: “Windows Hypervisor Platform”

“Windows Hypervisor Platform” isn´t hyper v and is only available from Windows April 2018 Update moving forward. And LTSB did not get that. And the feature isn´t available to install.

That´s a bummer maybe it´s time to get rid of LTSB again. But I can still do testing on my phone or my notebook. But doing the testing straight on my phone isn´t great as I kinda would have to get rid of the songs before. So… that´s that.

That´s the first thing now I can put on my “didn´t work on LTSB” list.

Bluestacks works. I tried that because I knew it was an Android emulator that a friend uses for some Android Game. Well… it does not seem like anything you can use well for devoping…

If anyone has a recommendation of another android amulator I could try that would work well for Android developing with Android Studio, please tell me about it. (I could also try it without Intel HAXM, but that´s probably slow AF if its even supported by now).

Otherwise, I´ll probably not try to get an emulator working on this machine with it´s current OS anymore. Seems like more work than it´s worth. I´ll just make do with what works (phone and notebook).


So tomorrow I´ll actually have some developement planned. Figuring out how to build an “Android UI”. How to program background workers that can give you notification bar progress updates. And how rest requests work in kotlin.

1 Like

Is Jriver lib not just a Java class library? You don’t need android studio to program for android.

Also, you could make them in C/C++ you just need to X-compile for ARMx86 & ARMx86_64.

Jriver is my music player now, that should sync with my phone over Wifi. Dont ask me why its called Jriver, its written in C++ to my knownledge.

But I want it! xD

It´s working fine. Just the emulator does not. But I do have a physical Android phone or I can run the emulator on my notebook until I put a different Operating System on my Desktop. :slight_smile:

ahhh okay. :+1:

Slacked a bit until now, cause lots of other things to do. But now I have something doing something. Coppied a mp3, a mp3’ish m4a, a flac and a wav file. All of them play fine. Wav files dont have embedded metadata. So I have to get rid of the 1 or two I have anyways. That’s to be expected though. Wav is just a bad format.

But the project is deffinitely easiely doable. Most of the work now is gonna be parsing threw loads of xml files I get from the music player to figure out what to actually sync and what are duplicates and keep track of that. So I’ll also be creating a sqlite db.


DB Tables JDROP

jd_library (table for library, since there could be multiple)

ID NAME IP PORT LOGIN_SETUP USERNAME PASSWORD LIBRARY_ROOT
Key String String Integer Bool String String String

jd_music (table for music)

ID CREATED_AT LAST_UPDATED PATH_ON_DEVICE LIBRARY_ID JRIVER_SONG_ID
Key Timestamp Timestamp Path JD_LIBRARY_ID Numeric

The data types are whatever. Sqlite does not really have lots of data types. But you still have prebuilt wrappers for many more datatypes usually. So I’ll figure it out. Just random notes for now how it roughtly will look like.

jd_sync

Log file(s)?


Current UI (as I said nothing to look at xD)

Some synced files
Selection_005

Actually playing them


That’s how I got the files synced now. Far from anything complete. Only syncs one song. The inputStream can probably throw one or more exceptions if the server leaves town. And I’m currently loading an entire song into ram each time. But that might be not the worst thing ever makes it a one-liner and I don’t have to deal with cleaning up half way written files. We have 8gb ram phones I can load a song that’s max 50mb (probably) into ram and not die from it for now.

/*
* File Syncronization
*
* Search files: 192.168.1.113:52199/MCWS/v1/Files/Search?Action=mpl&ActiveFile=-1&Zone=-1&ZoneType=ID
* Get File with id 2: http://192.168.1.113:52199/MCWS/v1/File/GetFile?File=2&FileType=Key
* */
class Syncronizer(credentials: JRiverCredentials, context: Context) {

    val credentials: JRiverCredentials = credentials
    val context: Context = context

    /*
    * Synchronizes everything
    * */
    fun syncClean() {
        syncFile(6);
    }

    fun syncFile(key: Long) {
        doAsync {

            val connection = URL("http://${credentials.ip}:${credentials.port}/MCWS/v1/File/GetFile?File=$key&FileType=Key").openConnection() as HttpURLConnection
            val auth = Base64.getEncoder().encode(("${credentials.user}:${credentials.password}").toByteArray()).toString(Charsets.UTF_8)
            connection.addRequestProperty("Authorization", "Basic $auth")
            connection.connect()

            val bytes = connection.inputStream.readBytes()
            val music_root = Environment.getExternalStoragePublicDirectory(Environment.DIRECTORY_MUSIC)

            var music_exists = if (music_root.exists()) true else music_root.mkdir() //ultimately should be music/jdrop/library_folder


            if (music_exists) {
                val dest = File("${music_root}/test.wav")
                try {
                    dest.writeBytes(bytes)
                } catch (e: Exception) {
                    //TODO: handle io exception
                    context.toast("failed to write the file")
                }
            }

            uiThread {
                context.toast("Done.")
            }
        }
    }
}

Still, not a lot has happened honestly. But at least it “does something”.

So now… let’s parse XML files (yey) and build a database.

2 Likes

You’re doing great stuff bro!

Keep up the great work.

2 Likes

I picked this project up again a few weeks ago. And actually got something sorta-kinda working at this point (but it’s not great yet it misses a few features I want, but I’m already using it to sync my 28.6GB music library and so far the only two failed songs are one broken link in JRiver itself and one that i didn’t really figure out yet what it is, it has no name, no artist and the URL is unreachable).

Made a little demonstration video if you’re interested how that’s all gonna work out roughly (very pre-alpha by todays standards, but it works)

(Yes that Song number 8 takes forever, that’s the song that broke readBytes now I copy byte for byte with a buffer lul, it’s 185.3MB worth of song)

So mainly I have to work a bunch on the UI and also I can’t (yet) update songs that have already been synced that have changed in one way or another. I also want to implement ‘two-way’ syncing for playlists, so that I also can sync playlists from my phone back to my library, witch the ‘plug your phone always into the same OS every time with a wire’-solution didn’t do either.

Currently, I’m working on automatically updating the ListView when the ForegroundService deletes songs from the list that have been synced successfully. The problem with that right now is that when the ListView decides to update it calls getItem sometimes on an item that no longer exists at this point (it probably did when it got the index for the item). So i get an IndexOutOfBoundsException. First thing I tried was just return null it’s fine, right? No it’s not, because you get another exception when getView returns null as it’s View. You can’t return null as the View, but I also don’t want an EmptyView as it’s eventually supposed to be clickable to show extra stuff. I’m not sure how, but I’ll solve it eventually. I tried using ConcurrentLinkedQueue witch is supposedly thread save (sounds great right?), but that does a whole lot of nothing since it just instead of throwing an IndexOutOfBoundsException returns null, so I have the very same problem regardless and it’s probably very very bad compared to ArrayList when you scroll to index 1000++ it would have to count to a 1000++ for every item it tries to display on the screen.

Besides UI prettiness there is also a lot of other things that should be done eventually. Like I probably want some sort of failed/ignore list. For items that either will fail everytime (like the one I for some reason have in my real music library that’s the +1 of my 2434 songs without any name or artist and an unreachable url). And also generally to be able to somehow manage what you want to sync and what you don’t either by whitelisting or blacklisting songs/playlists/albums/artists. I also want a make a seperate ‘loading library’ screen as the bunch of list views are gonna be pretty empty before the xml is loaded and parsed. And I wanna be able to press sync right away and it should happen as soon as the parsing is done (currently you gotta wait for it). Also playlists should be moved to the syncQue, currently they are their own thing and you have to do the playlist syncing after the songs have synced or parts of your playlists will be missing, since the songs aren’t there.

I also figured I don’t (really) want this thing to be a one trick pony (since I might not use JRiver forever). So I’m also (admittedly passively right now) working on making things not quite as JRiver only (for now that’s only from a code base standpoint, so naming things as if they would be universal and use String for remote IDs even though JRiver might return numeric ids other players may use UUIDS. This is for now more of a dream than an I’m deffinitely gonna do this. Though I can say, one likely canditate for a proove of concept of this is gonna be plex as I also have it installed and know friends who use it for music that report that the music experience on phones is utter garbage. Witch is basically the same thing I concluded when I tried using plex for music, except I concluded for myself it’s unuseable for anything music at all. But however, for now I’m mostly focusing on getting JRiver working well (!) and then maybe releasing it in one way or another. Currently I don’t believe the user experience is quite there yet to really put it in front of anyone that does not know it’s quirks. But eventually it will. If it takes till next devember, so be it. I think it’s gonna be great regardless! :wink: