Not much has been done, did not have time to develop anything during the weekend. Frustrating, as I need to gain some momentum now. Hopefully I won’t skip as much anymore.
I’ll be programming on the road for the remainder of my Devember, as I’m done with my job I’m off to travel for a bit. I’ll try to put in my hours, and will mostly be able to, but I won’t always have an internet connection lol. So I might miss days 
With a…?
Well, of course with a steamrol… ahem… I have my ThinkPad with me.
Almost finished the training mechanism, just need a few more tests to try and break it. I’ve worked through a race condition on it, great thing go test has a -race flag to help with finding them. Also went ahead and implemented nums new command, since it’s an easy thing to do. Now it makes me a randomized network file as it should. Today I reaaaly should finish up with the Trainer, it’s getting boring haha.
Added a couple more tests of different cases to Trainer, I think it’s good enough to go. Implemented initial nums train command, and I’m now encountering deserialization errors from the neural network files when attempting to train. Tests weren’t good enough I guess. So I’m hunting this down tomorrow.
Worked out the errors, mostly dumb stuff. I was trying to load a gzipped file and was wondering why stuff was off. idx should’ve caught that the magic number was wrong, so I added a test for that and implemented it. Now we’re churning through, yay.
But now I’m getting a bunch of ErrIncorrectArgument = errors.New("incorrect argument, probably incompatible with target") errors piling up constantly. There’s a couple of places where that error could happen, mostly I return it when the input vector is not the right size for the layer of activations, or some other vector or delta is incompatible with one another etc. I’ll have to fix that before I continue. Not today unfortunately, my hour’s done.
Fixed more dumb stuff, found that I failed to implement one method (great practice to make those methods return a “not implemented” error). Among that I made some logging and error improvements, most notable of which is implementing a more descriptive error type for mismatching lengths of vectors and matrices at runtime, where unit tests are of no help:
// IncompatibleError is returned by a function when matrices or vectors are of incompatible
// size/length. This error is initialized with
type IncompatibleError map[string]int
// Error builds and returns the errror string from Fields
func (ie IncompatibleError) Error() string {
var builder strings.Builder
fmt.Fprintf(&builder, "incompatible error: ")
for k, v := range ie {
fmt.Fprintf(&builder, "{%s -> %d} ", k, v)
}
return builder.String()
}
It is used in place of ErrIncorrectArgument like so:
func (n *LinearRegression) Backpropogate(inputActivations []float64, desiredOutput []float64) (Delta, error) {
if int32(len(inputActivations)) != n.layers[0] {
return Delta{}, &IncompatibleError{
"given input activations": len(inputActivations),
"length of layer 0": int(n.layers[0]),
}
}
// ...
}
And gives me much more descriptive output when I’m training the network like:
trainBatch() failed to apply delta with incompatible error: {index of the matrix -> 0} {length of the matrix data -> 7850} {length of the ratios data -> 785}
trainBatch() failed to apply delta with incompatible error: {index of the matrix -> 0} {length of the matrix data -> 7850} {length of the ratios data -> 785}
trainBatch() failed to apply delta with incompatible error: {length of the matrix data -> 7850} {length of the ratios data -> 785} {index of the matrix -> 0}
trainBatch() failed to apply delta with incompatible error: {index of the matrix -> 0} {length of the matrix data -> 7850} {length of the ratios data -> 785}
So much better! But there still seem to be things to iron out, so marching onward, with the little time I’ve got to give this now.
On a somewhat unrelated note, I’ve visited Mt Cook and it’s receding hairli… glacier with way too many tourists (super annoying) today. Pretty cool day, perfect weather lol.
Put in a few hours these couple of days. Fixed all the errors, now I’ve got a clean run! Except the run somehow returns some weird results, so I’m far from done 
. It’s hard to find something like that, it looks like it’s time to write some integration tests, simple cases for the network implementation. Then I’ll be able to debug them and figure out what the issue is.
Not stoked about a week remaining and I’m still not optimizing. I’m tempted to regard this as a good enough for performance optimizing. But I’ll never finish it in that case, gotta keep going.
