You're right, the implementations of ARM designs are not open source in many instances, but they are in others. ARM designs themselves are open source though, and very well documented.
In the end, whether or not the hardware manufacturers blow the fuses on their SoC's or apply raisin to obfuscate parts, is not as important as the fact that manufacturers like Samsung, AMD, Freescale, etc... drive the innovations in hardware and software these days.
The remarkable thing is that they've set a new standard, and in reaction, x86 is not moving towards more performance, in fact, with the major software platforms, someone with a core duo (first gen) or AMD X2, will be productive enough to make money and compete in the market, there is no real need for anything faster, because the technology that would necessitate faster hardware is not released to the public. X86 is rather moving towards lower power and lower thermals, to compete with RISC SoC designs, because miniaturization is always more important than performance increase. With miniaturization comes modularity and flexibility, and the world has become so complex nowadays that it's just not possible for one coglomerate of monopolists to cover the whole technological market, unless it's a niche market. That's why open source has become more important than closed source. That's why Intel doesn't run Android 5.1.1, and Intel has no Android 5.1, or even a 5.0 image available for anything but select Z-chipsets that are only found in phones and some tablets, and those images are courtesy of Rockchip in China, and did not come cheap for Intel... all of this while Android 5 is already the past, because the Google Devcon this year is behind us, so Android M is what it's all about now... and that's just one example, there are many more, like the obvious backlog of Intel in keeping up with the Linux kernel, that is pushed forward aggressively by the ARM-based developers.
The entire added value these days lies not in performance, but in miniaturization and lower power. The Wintel Alliance knows this. That's why they sell the Surface 3 (non-Pro) 10inch lightweight (and much more practical, at least in the 4GB RAM version, the 2GB version is pretty useless) for just under 800 USD including the keyboard and pen. A better device from Lenovo, that also includes a 500 GB harddrive, costs only half of that, but it weighs 300 grams more and is 3 mm thicker, even though it has a longer battery life and a stronger Intel processor... less is more these days. Windows 8 already has WIMBOOT, Windows X will use NTFS compression on system files to reduce the system image to about 15-16 GB of storage space without full WIMBOOT. A WIMBOOT image comes in at about 4 GB for a very minimal arrangement, which is still about twice as much as a non-compressed full KDE 5 or Gnome 3.16 Linux Install, or a full blown Android 5.1.1 or M install. Intel has shown preparedness to sacrifice huge amounts of system performance (e.g. extreme throttling to a point where the CPU's underperform greatly, even compared to pretty cheap ARM SoC's), Microsoft has only removed Windows-functionality since Windows Vista/7 for the same reason, there is no technical advancement (NTFS compression is hardly new, even when it's applied to system files...) at all, all they do is cut user functionality, and the only thing they add is spyware functionality, which is cloud-based and runs on their servers, not on the local device, much like Google Now and Siri.
And that's where your wishes will come true to have ARM on the desktop: to be perfectly honest, I wish I could use Android on the desktop sometimes, because it is thousandfold more efficient than commercial desktop platforms. For instance, I have a Cherry Trail based convertible, and it was really hard to get Linux to work on that thing, and every time the kernel is updated, it takes me a lot of time to tweak the new kernel to work on that device. The experience of Gnome 3.16 is great, but to be honest, for instance using LibreOffice or Darktable with touch screens is not the best experience ever, it's about on par with MS-Office 365 or Lightroom CC, so it feels kinda old. Using Google Docs and Snapseed on an Android touch screen device on the contrary is real progress in terms of efficiency.
Everything evolves. Some time ago, it would be inconceivable to do commercial grade photo editing on anything else but a powerful desktop computer. Nowadays, there are cameras that allow for live viewing the exact end result while shooting, and print is not the primary destination of the imagery, but rather mobile device screens are, so shooting jpeg is more than adequate enough. With that come really efficient software solutions like Snapseed, which are also free to use. The same thing goes for office software. These days, the documents are shorter and are sent back and forth at a much higher rate, and online cooperation on documents and version tracking have become the main tools of office productivity. All of the bloatware functionality of heavy office suites has become less necessary, because these applications are made for monk-like heavy editing, which is great, and I prefer it personally, but the world doesn't require that kind of dedication any more these days, now everything has to show constant motion, move very fast, touch a lot of people at the same time. Google docs is very good for that, so is Inbox. The big benefit these newer softwares have, is that they are tiny and run very efficiently, and are based upon open source code and projects, even though they always kept open source or open source isn't always credited or even respected... but they inherit the benefits of their open source roots... for instance object linking and embedding, something that has never ever worked as it should on commercial x86 platforms (and those that have written applications that tie in with MS-Office in the past know exactly how problematic this has always been), is super smooth on Google's commercial Android platform, because it's based on open source, where this just works, and requires little overhead. Another thing that really matters, is that C++ for instance, was never made to be lean and mean, it doesn't deliver the same small footprint as C, it's bulky to use, not suitable for the mobile world, even if the .net tools are made available for free now, and some of it is open sourced, the problem of code efficiency remains.
In conclusion, having an ARM-based desktop device isn't a bad idea, because a lot of modern more efficient software runs better on ARM, and is efficient enough to give an experience that is better than old style commercial software on x86 devices that are made to compete with ARM. So I definitely share your wish for a desktop ARM device.