Return to

Porting a Raytracer to Rust - Devember Challenge



Day 06:
Tests. After my chat with @AdminDev and @anotherriddle I started to think about tests. Designing tests does not come naturally to me, but thankfully the blocks of code I need to test are still reasonably small.

Testing remains an alien concept to me because in my day job (I am an EE, sorta) most of the code I write integrates with some piece of hardware. Mocking Databases is easy compared to mocking a camera on the end of a USB pipe.

But mercifully this project doesn’t have any IO (yet, I dread having to test the result of the images) so I started by writing a test for the hit detection I built yesterday. Its a simple test that hard codes an object and compares it to a hard coded incident ray.

The test immediately highlighted a bug - in the random number generator*, so time to write more tests… and fix the bug. The bug was an overflow, intended, but Rust will panic if an integer overflows, so we need to tell rust we are OK with the values wrapping around using the Wrapping<T> class.

After this I wrote 6 more tests for the Sampler.

*HQZ uses its own random number generation to create a sequence of numbers that change slowly.


Day 07:
When I started this project I said I didn’t want to write my own quadtree… but as time progresses I am feeling it is more and more likely that I will need to.

First thing this morning was copying an existing quadtree implementation inboard to the project, and modding it to work on f64 data instead of f32.

After that was working, I removed the Vec2 implementation and replaced uses with Point and Vector types form the quadtree implementation.

This was followed by more test writing and and general code clean-up.

I have built enough framework now that I am back at my original problem; how do I do the collision tests on the quadtree.

This problem is starting to burn me out… so I might need to find a side task to work on tomorrow.


Continuing to work on this this evening I realized I am starting to burn out on this project, gotta learn to pace myself.


Same! I suggest coding something silly to ease yourself back into the groove. That’s what I had to do anyways, but hopefully everything works out :slight_smile:


Day 08, Day 09:
Life finally got in the way… sigh

But I am still in the challenge! No new code over the weekend, but Saturday I managed to spend an hour working on a blog post for my main blog about Devemember and what I am learning, which I will publish at the end of the challenge, and Sunday I did some research on quad trees and binary trees.

Monday I am hoping to get back to coding.


Day 10:
Back to the notebook

Days 6 and 7 saw a lot of code progress and I was starting to believe I could see the finish line. Looks like there are a couple more hurdles to cross.

I started writing in Jupyter again, creating notes about where I was stuck in the app design, and before long I had a view onto a solution. I restructured a lot of the raytracing logic into the ray. Its not pretty but it works,

An hour and a half later and the raytrace code was implemented for a list of objects to test against and the unimplemented flags had all been cleared.

Umm… its built.*

I guess tomorrow I get to find out if it works.

*for certain naive values of built.


Finally applicable in a non sarcastic way.

Pics or it didn’t happen!

Good stuff. Now if it works it is all bonus work. Make pretty and fun additions.


It panics when you run it, and even if it didn’t there is no image output yet.


Day 11:
Well it doesn’t work.

I wrote a test to execute a raytrace and it panics with “Ray Exists outside of viewport”. This error is triggered when a ray is created outside of the viewport and never enters it, so there is nothing to render. This error should never happen so the re is a bug in one of my collision functions… Which guess what don’t have tests.

So Guess what I need to do next.

(sorry for the late post life is mental rn)


Days 12, 13:
I slipped but I hope I can be forgiven.

Wednesday I had an Exam, and Thurs was my last day at uni for the semester, which involved me boarding a plane and flying to Germany.

So no work for 2 days.


Day 14:
I woke up in a foreign country this morning, but a familiar city.

I am traveling in Berlin at the moment, but hackerspaces exist and so do hackers, so I found time to do some work on zenphoton.

Working in the CCCB with an old friend I spent the evening investigating how the pseudorandom algorithm used with the old HQZ effects the generated image.

What we discovered was that as long as the distribution is largely unaffected the image retains its quality.

For example this is an original image created by old HQZ:

The Grain in this image creates a spectacular effect.

Here is the same example run through HQZ with the prng replaced with a 16 bit pcg generator shifted to the upper portion of the 32bit output word:

Visibly the characteristic grain of the image is retained, something I feared loosing by changing prng, however increased quantising in the reflected ray is clearly visible. this is an artefact of the returned number being in the format 0x????0000 this is quantising the output values. It looks cool though.

This image was created by replacing the PRNG with a call to rand() provided by the stdlib, which was then properly normalised. There are some visible differences to the first image, and I think it might actually be slightly darker. However qualitatively this render looks just as good as using the original prng.

Therefore I feel I can make the following conculsions about the use of random numbers in zenphoton:

  1. The bit depth effects the quantisation of the simulation. (Duh when you think about it)
  2. The actual algorithm used has no effect on the quality of the image produced,
  3. maintaining state across the PRNG instances has little effect.

Not shown is 4 hours of failures and weird as hell images that we created at 5am. I’ll save the best of them for the final blog post.

Finally I mentioned I am traveling this weekend. I have enough internet for blogging but not for syncing large code repos, sorry the commits from this weekend won’t be public until next week.


Day 15:
Still in Berlin, waiting for a friend to turn up, so lets code. The failing test that shows the raytracer as not working is making me said, so lets focus on png output.

Rust has a library for outputing PNG images… its called ‘png’…

PNG takes a stream of u8 values to write to its file. so we need to implement a to_RBG8() function on the image struct. We may also want to implement a to_RGB16() function for creating 16 bit PNG’s.

Sadly the normalisation function requires doing min and max functions on floats, this is an unstable feature rn, and trying to set it up to work thwarted me after several hours of hacking.


Day 16:
So I was hoping I could code on the plane.

Super economy seating and a desperate need to sleep disagreed.

Travelling and Devemeber do not mix well. Thankfully I am now back home, and will be for the rest of December and can get back to work.


Day 17:
I am starting to feel lost in this project.

There is a growing list of things i should probably refactor, but I am resisting under the banner of “premature optimisation”

There is only one thing to do when you’re this lost: fix bugs. RUST_BACKTRACE=1 cargo test is a great way to tell me what’s wrong. The raytrace test is still failing, inside the furthest_aabb test. So lets fix that.

First we need to write some tests for furthest_aabb.

So I wrote two tests, the horizontal test worked, the vertical one fails. I tracked the bug to the rayintersect function, which is returning distance based on X coordinate.

I need to check if I’ve copied the algorithm from HQZ wrong, or its a bug that’s carried through the port. Either way its tommorrow’s work. Its late I am tired.


Why? The performance on Rust is bad enough. Why would you possibly want to make it even worse? They haven’t even fixed everything already which is still wrong with it. It’s also most likely never going to be finished. There has been zero progress over the last year.


As far as I can see, Devember is not about practicality but learning. Why crap on someone’s fun just because you don’t like X about their project?


Rust the Programming language, not rust the game…

Maybe read the thread before commenting, thanks, bai.


Day 18:
So the incorrect horizontal test seems to be a bug carried through from HQZ. My code is identical in function to the original, but somehow this works in HQZ but not my code.

I spent some time refactoring the intersect AABB function but to no avail, so I moved onto adding bounds checking to the plot function in the image.

With that more than a couple of hours had passed and I needed to get back to the real world.


Day 19:
You know that one bug you spent a week working on? The one your boss was breathing down your neck to see progress on? The one where you don’t know how to tell him "I don’t know how to fix this yet " without loosing your job?

Yea I have one of those bugs… The bug in how the ray intersects with a box is being stubbornly an algorithmic problem in an algorithm I do not understand.

So work is happening but progress is not. I have been forced to reach out for help.

If anyone is interested in supporting me in some rubber duck coding PM me!


Day 20:

I got in touch with the original author of HQZ, She gave me some helpful pointers and we found the bug in the algorithm. Intersecting a perfectly vertical ray was an error that never came up in the original variant, but if that condition did happen a division by 0 error would occur which is the same bug I am facing now.