How to RTX 3090 and TensorFlow like a Pro

Even I had same questions and you beat me to it. I think it is fp16 for all stages which is really not how I have seen anyone train their models. I would at least use tf32 to handle those vanishing/exploding gradients better.

one more question I had was, what was the input stage like? How performant is the storage? where are the images stored? I have started using tf.data.xxx and it has helped me optimize my input pipeline and improve speeds.

Interesting, also doing a lot of work with BERT variants at the moment, hence why I went for the 3090 with 24GB as was exhausting my vram on 1080 as you can imagine.

2 Likes

TX2 is old Pascal architecture and does not compare favorably at all with Desktop or server. The target use case of Jetson family is in robotics, so the comparison never makes sense anyway. If yo are going to go for a TX2 type product, I will suggest the Xavier NX. Much better cost to performance ratio than TX2 from a pure inference pov.