Epic How to build AI?

I dunno where the book is but I have the CD handy.

"Charles River Media - AI Application Programming: By M. Tim Jones"

is it this?
and wow already from 2005....times goes fast.

LOL MINE WAS FROM LIKE 2002
It gets updated regularly.

Back propagation does not guess random numbers. It goes towards the negative gradient of the risk function.

Do you have any paper that talk about this? If so, what kind of architectures they have tried? I have never heard of something like that.

True

Why do we need to manufacture 3D electronics? Why cant you model something like that in TensorFlow?

Back propagation isn't usable on Restricted Boltzmann machines because of their nature. The method used instead is called Contrastive Divergence where a sample is selected at random and a change is made based on probabilities compared to a random number.

That's because of a simple fact of VLSI design. Specialized hardware implementations will always perform better than software implementations working on non-specialized hardware. That's basically why the GPU was invented in the first place. When you implement an ANN neuron in Tensor flow it takes around 200-300 operations at the hardware level to do just a single weight update and backpropagation. If you were to do the same but with neurons designed on hardware it would take around 4-5 operations to do the same because everything is happening on the same clock tick. There is a reason the creators of TensorFlow are creating their own AI chips.
The only issue we still have in this regard is the size limitation of flat chips. You can place a square grid of many neurons but you can't connect all of them to each other without using jumpers which is a very rudimentary form of a 3D chip. If we can actually create 3D chips we can have more neurons on the chip that are connected to all others. Not to mention the increase in total number of neurons possible on the chip as per square-cube law.

I don't believe there is a paper from anyone who has tried this but I can tell you why I make this claim. You can think of the complexity of a task to be a function of it's input space (number of all possible inputs) and output space (number of all possible outputs). Multiplying them together gives you a complexity number that you can compare roughly speaking to the number of neuron connections in your network to see if your network has enough learning power (for further reading on this look into VC-Dimensions). If you want to train multiple tasks on the same network you don't just add the complexity numbers for the two tasks but you have to MULTIPLY them together. That means you need way too many neurons/connections as compared to if you were just training two separate networks. Therefore it will take you more time and more data to train them as well.

1 Like

Thanks for taking the time to respond to my questions. Although is not about what the original post was, know I understand what you were trying to say.

1 Like

you might want to look into neural networks which are self optimizing networks that learn how to do tasks and slowly gets better and below is a link to a neural network that learns to play Mario world by itself

This was already posted :)

sorry didn't see that xD

it also ended up in an AI mayor discussion XD and i see it got moved to science and tagged with AI tag. its interesting you think a lot of these things but always handy to see others type it in words.

1 Like

Well i think that we should try to come up with a stack that makes clarity labs project Lucida useful to us

I still sadly want somebody here on this forum to play with it. XD you can launch a docker 16 GB to test it out, but i lack 16 GB and not buying it soon.

But i still wonder how good that Lucidia is if somebody from our forum would tweak with it or test it out.

1 Like