Devember - Screep Studio

Haha, I was just about to post this (in the hopes of confusing the Americans as much as they’ve confused me)

jif_cif

3 Likes

Haha I know, but thanks for the heads up in case I didn’t.

Yup…

Day -2

Fun and games with HTTP/Screeps authentication :thinking:

Not helped by the fact that there are about 100 different ways to authenticate with the Screeps server and differences between how it’s done
on the live server vs. the private server. The error messages aren’t great either so I’ve been reading through the JavaScript source for the server back-end a lot.

RestNetworkManager::onRequestComplete(): Error= QNetworkReply::NetworkError(AuthenticationRequiredError)  -  "Host requires authentication"
ScreepsClient::onResponseReceived(): Request= "http://192.168.1.11:21025/api/game/room-overview?room=W9N1&interval=8&shard=shard0"
ScreepsClient::onResponseReceived(): Request= "/api/game/room-overview"
RestNetworkManager::onRequestComplete(): Error= QNetworkReply::NetworkError(NoError)  -  "Unknown error"
ScreepsClient::onResponseReceived(): Request= "http://192.168.1.11:21025/api/game/room-terrain?room=W9N1&encoded=1"
ScreepsClient::onResponseReceived(): Request= "/api/game/room-terrain"

Gonna need a bigger hammer to crack this one …

Shecks

2 Likes

I wish I could help here, but I’m not experienced with Screeps. :confused:

1 Like

Man, that looks great! :sunglasses:

But you guys don’t need that much of a head start.
If anyone needs a head start it’s me. :thinking:

1 Like

Hey, we’re just having fun. :stuck_out_tongue:

3 Likes

I’m surrounded by crazy people! fun he says …
:wink:

3 Likes

Look what I found! … The sneaky little screepers are using Steam for authentication even on the private servers!

Starting CLI server
STEAM_KEY environment variable found, disabling native authentication
Connecting to Steam Web API
CLI listening on localhost:21026
Starting game server (protocol version 13)
SockJS v0.3.19 bound to "/socket"

See it is all fun :rofl:

1 Like

Sounds like if you launch it with STEAM_KEY=, you’ll be set.

If you need help setting up systemd environment vars, let me know.

1 Like

It’s ok thanks, that’s in the server config file … I wouldn’t mind but when I was first setting up the server and trying everything to get it to run I was in that file and saw it there, should have known…

I’ve been looking at the “/api/auth/signin” REST end point which is the old way to authenticate that I thought was still active on the private server, but looks like they disable that method because the steam key is there.

*** RestNetworkManager::onRequestComplete(): Request= QUrl("http://192.168.1.11:21025/api/auth/signin")
*** RestNetworkManager::onRequestComplete(): Error= QNetworkReply::NetworkError(AuthenticationRequiredError)  -  "Host requires authentication"
*** RestNetworkManager::onRequestComplete(): Body= ""

This makes sense now … you need authorisation to authenticate :rofl:

Getting closer … I removed the steam key and it went crazy. I did a little more digging and found the code, looks like when it has no Steam key it tries to use the Steam client to authenticate(which doesn’t exist on my server) using some library called “greenworks” which I believe is used by Steam Electron apps …

Day -1

Well that was more difficult that it should have been!
But I finally got the username/password authentication working and I can now make calls to the REST endpoints that require the auth token.

RestNetworkManager::postRequest() Body: "{\n    \"email\": \"Shecks\",\n    \"password\": \"mysecretpassword\"\n}\n"
ScreepsClient::onResponseReceived(): Request= "http://192.168.1.11:21025/api/auth/signin"
ScreepsClient::onResponseReceived(): Request= "/api/auth/signin"
ScreepsClient::onResponseReceived(): Request= "{\"ok\":1,\"token\":\"948ddc6661b3c1e304e7141b8525778e168b472e\"}"
ScreepsClient::onResponseReceived(): Success= true
MainWindow::onClientAuthReceived(): Token= "948ddc6661b3c1e304e7141b8525778e168b472e"
ScreepsClient::onResponseReceived(): Request= "http://192.168.1.11:21025/api/game/room-overview?room=W9N1&interval=8&shard=shard0"
ScreepsClient::onResponseReceived(): Request= "/api/game/room-overview"
ScreepsClient::onResponseReceived(): Request= "{\"ok\":1,\"owner\":{\"badge\":{\"color1\":\"#166b9c\",\"color2\":\"#e1da64\",\"color3\":\"#b5fb01\",\"flip\":false,\"param\":26,\"type\":12},\"username\":\"JackBot\"},\"stats\":{},\"statsMax\":{},\"totals\":{}}"
ScreepsClient::onResponseReceived(): Success= true

There is another mechanism to authenticate via a permanent token associated with your steam account but I can’t seem to get that to work on my private sever as it makes a call to a Steam Web API for token validation and I don’t think it’s configured correctly.

Anyway … user/password auth is good enough for now. Now that I have the token I can go ahead and start working on the Websocket API

Shecks

2 Likes

Day 1

Devember is finally here :grinning:

Plans for today

Code refactoring/feng shui - turning all the little bits of prototype code I have been working on into a more structured based that I can build on.

  • I already have a data layer for the REST API built (QT doesn’t have direct support for REST so I wrote something simple) and have added support for a couple of the core game objects (player info, room terrain, room overview/ownership etc) . but I’ve written it in such a way that I just need to write new subclasses of my RestRequestBuilder and RestResponse classes for each new object I need to support.

  • There’s a placeholder for the WebSocket network manager, but I am going leave that until after I have the overall code structure cleaned up

  • Because there are two sources of data available from the Screeps server (REST API for static/pulled content and Websocket for realtime events) I want to consolidate them so that the UI components only need to be concerned with a single data model. So I am planning on building an intermediary pub/sub broker component for both sources which will serve as the data source (via QT signals/slot) for a set of models (e.g RoomModel, PlayerModel, ConsoleModel…) which in turn will serve the UI components.

Shecks

2 Likes

I just looked into your project. Regarding the technical and code related stuff I have no idea what’s happening but this looks cool anyway. I’ll be lurking. :popcorn:

Maybe I even learn something. :smiley:

1 Like

Day 2

Nothing exciting to report today, still refactoring code and trying to get things clean before I start adding features (really want to get on to building the UI and doing the graphics for the game room scene). I’ve had to go and brush up on some basic C++ I should know and some C++ 11 semantics (implicitly deleted copy constructors … eeew)

Oh, well back to the code…

Shecks

Day 3

Yay … I’ve finally done enough refactoring to the point where I am happy with the structure of the code. It’s clean enough now for me to start working on adding all the functionality I have in mind for Screep Studio.

Visually the UI doesn’t look very different from my original prototype but behind the scenes things have changed a lot (I will eventually get around to committing it to my GitHub).

There’s still lots of networking code to be done (websocket support so I can get the room dynamic contents) but I couldn’t resist adding some UI elements just to get the thought process going … I’m kind of weird in that I like to at least mock up UIs early on to give me a feel for the main features and structure of an application. From that I can figure out what kind of components I need to build to support the UI.

The sceenshot shows the following:

  • The central view (currently QGraphicsView) will be the main component that the user will interact with. It will show a live view of the selected room with animated Creeps going about their business. Currently it’s capable of displaying the fixed room terrain (from my RoomModel::TerrainModel), it supports panning via the mouse and can be scaled/zoomed (In my prototype I had a slider to change the zoom but I’ve removed that and will add support for zooming via the mouse wheel once I subclass QGraphicsView and replace it with a custom RoomGraphicsView widget)

  • On the right is a placeholder for the “Properties” panel. The idea here is that once I have implemented the Websocket interface and have populated the room with live entities (Creeps, storage containers, spawns, ramparts etc), the user will be able to select any cell in the room grid and the properties of the selected entity will be displayed in the properties panel. Kind of like an object inspection view in UI designers and IDEs,

  • On the bottom is a place holder for the “Console”. The Screeps game API (the JavaScript API used to programming you A.I) provides methods to output text to a console stream which a lot of user use for debug information to give feedback on what the Creeps are upto. I plan to subscribe to the player console stream (Websocks again) and display this data here. Long term I also would like to allow the user to issue console commands here too.

  • Note shown above, but planned for the future. I would like to have a “Tools” panel with a palette of all the objects that can be constructed in the room. If I ever get to the point where the application becomes more than just a viewer.

I’d like to support viewing/working with multiple rooms at once (my ScreepsModel already has support for managing multiple RoomModel instances) but I am still considering how this would work in the UI. My current thoughts are that the RoomGraphicsView will be moved into a QTabWidget and a new tab and RoomGraphicsView will be added for each room the user opens. I think the Properties and Console views can be global. (As far as I know there’s only one console stream per user rather than per room anyway)

Anyway, that’s where things stand currently. Now I need to drag myself away from playing with the UI and get working on the Websocket code so I can put something interesting in the room.

Shecks

2 Likes

Still Day 3

Behold! I will now open a websocket connection to the Screeps server …

qt.network.ssl: QSslSocket: cannot resolve CRYPTO_num_locks
qt.network.ssl: QSslSocket: cannot resolve CRYPTO_set_id_callback
qt.network.ssl: QSslSocket: cannot resolve CRYPTO_set_locking_callback
qt.network.ssl: QSslSocket: cannot resolve ERR_free_strings
qt.network.ssl: QSslSocket: cannot resolve EVP_CIPHER_CTX_cleanup
qt.network.ssl: QSslSocket: cannot resolve EVP_CIPHER_CTX_init
qt.network.ssl: QSslSocket: cannot resolve sk_new_null
qt.network.ssl: QSslSocket: cannot resolve sk_push
qt.network.ssl: QSslSocket: cannot resolve sk_free
qt.network.ssl: QSslSocket: cannot resolve sk_num
qt.network.ssl: QSslSocket: cannot resolve sk_pop_free
qt.network.ssl: QSslSocket: cannot resolve sk_value
qt.network.ssl: QSslSocket: cannot resolve SSL_library_init
qt.network.ssl: QSslSocket: cannot resolve SSL_load_error_strings
qt.network.ssl: QSslSocket: cannot resolve SSL_get_ex_new_index
qt.network.ssl: QSslSocket: cannot resolve SSLv23_client_method
qt.network.ssl: QSslSocket: cannot resolve SSLv23_server_method
qt.network.ssl: QSslSocket: cannot resolve X509_STORE_CTX_get_chain
qt.network.ssl: QSslSocket: cannot resolve OPENSSL_add_all_algorithms_noconf
qt.network.ssl: QSslSocket: cannot resolve OPENSSL_add_all_algorithms_conf
qt.network.ssl: QSslSocket: cannot resolve SSLeay
qt.network.ssl: Incompatible version of OpenSSL

Eh … not so fast mate! … Doh!!! :neutral_face:

Looks like some research is in order but initial indications seem to point to a known Qt bug affecting Linux distros with newer versions of OpenSSL than the version used to build the Qt libraries. :thinking:

Shecks

2 Likes

Day 4

Warning: Epic post ahead, skip to the TL;DR section

Well #devember2k18 was progressing nicely, I got a fair bit of work done in Day 3 and started to look into what was required to receive live events from the the Screeps server via the Websocket API. So I figured out the endpoint (ws://[host]:[port]/socket/websocket), created a QWebSocket object, connected a slot to the QWebSocket::connected signal and called QWebSocket::open() before eagerly opening the debug console expecting to see my debug output…

"NetworkManager::onSocketConnected(): Socket connected!"

Instead I was greated with …

qt.network.ssl: QSslSocket: cannot resolve CRYPTO_num_locks
qt.network.ssl: QSslSocket: cannot resolve CRYPTO_set_id_callback
qt.network.ssl: QSslSocket: cannot resolve CRYPTO_set_locking_callback
qt.network.ssl: QSslSocket: cannot resolve ... other stuff
qt.network.ssl: QSslSocket: cannot resolve ... more stuff
qt.network.ssl: QSslSocket: cannot resolve ... yet more stuff
qt.network.ssl: QSslSocket: cannot resolve ... does my SSL lib have any exports?
qt.network.ssl: QSslSocket: cannot resolve ... it's not stopping
qt.network.ssl: QSslSocket: cannot resolve ... <snip>

qt.network.ssl: Incompatible version of OpenSSL <-- Oh I see :( 

That’s not good! … but I’m probably just missing some SSL libriaries … no problem … some research and all will be well again.

A few minutes later I find this … QTBUG-68156

Ok, that’s not too bad, it’s a known bug … so what’s the solution? Oh the bug is still OPEN and there are recent comments asking when the fix will be released :frowning:

My understanding of the issue/bug

As it turns out this a bug either in the Qt libraries (or whatever part of the run-time is responsible for resolving dependencies when loading executable binaries). The current release of Qt is built against libssl.so.1.0.2 but the binaries have a dependency to libssl.so which on Linux is usually a symbolic link to the latest version of libssl available in /usr/lib (in my case this is libssl.so.1.1.1). So when my application is loaded the symbolic link is resolved and libssl.so.1.1.1 is also loaded … which is not binary compatible with libssl.so.1.0.2

So to make a long story even longer, there were hints that the bug was fixed in Qt 5.12 (I’m using 5.11.2) and, as luck would have it, 5.12 is due for release this week!

Excellent, I can just work on some UI stuff until the release …

Errrm no… based on comments from people testing 5.12 RCs the bug has not been fixed … and after some further digging I found vague references on the Qt developers mailing list about a bug fix that’s planned for early January 2019 :neutral_face:

80% of the data required from the Screeps server for Screep Studio comes from the Web Socket and I really didn’t want to start looking for a 3rd party Web Socket library I could use (it would probably mean integrating a dodgy/undocumented C library with Qt and based on past experience using SNMP-Net from Qt … it’s no fun at all)

It was looking like either my #devember2k18 was going to come to an early end or I was going to have to move my development to MS Windows (I think I’d sooner write my own web socket lib for Qt) as the bug is specific to Linux. I was about to give up when I stumbled upon a post from a developer who had come up with a potential work-around.

The Solution
Trick the linker into using the correct SSL lib without messing up the SSL installation on my dev box. To do this I created a ‘lib’ folder in my source tree and added a symbolic link for libssl.so that pointed to libssl.so.1.0.2

[shecks@red-dwarf lib]$ pwd ; ll
/home/shecks/Source/C++/Screep-studio/lib
total 0
lrwxrwxrwx. 1 shecks shecks 27 Dec  4 00:31 libssl.so -> /usr/lib64/libssl.so.1.0.2o
[shecks@red-dwarf lib]$ 

Then I defined the LD_LIBRARY_PATH envirionment variable in my project and pointed it at my /lib folder …

And it worked!

NetworkManager::onRequestComplete(): Request= "http://192.168.1.11:21025/api/auth/signin"
ScreepsModel::onRESTResponseReceived(): Request URL= QUrl("http://192.168.1.11:21025/api/auth/signin")
*** Auth Signin Response:  "{\n    \"ok\": 1,\n    \"token\": \"da081ba35305291f32c1af7b50733219849bfc73\"\n}\n"
NetworkManager::openSocket(): Attempting to connect to:  "ws://192.168.1.11:21025/socket/websocket"
RoomModel::RoomModel(): Created: Obj= RoomModel(0x27c3f80)
NetworkManager::onSocketConnected(): Socket connected!

There was much rejoicing … then I went to bed

TL;DR
The SSL libaray issues has been worked around … all is well with the world again and my #devember2k18 is back on track :beer::beers:

Technically since I was working on the fix at 00:30 4th Dec this morning I’ve already met my 1 hour commitment for today … so I’m off to do something exciting instead …

Yeah right, I have no life … back to websockets so.

Shecks

1 Like

If you can’t use websockets, then consider doing long-polling instead.

1 Like