Return to Level1Techs.com

My Devember2020 Project

My devember project is one I’ve been thinking about tackling for a few months and this is the perfect opportunity!

Background:
Have you ever taken photo and it turned out blurry? Not enough lighting and the image was noisy? Sure, every phone ships with a beauty filter baked into the camera, but wouldn’t it be nice to have a friendly neighborhood “AI” to help you clean up those blurry noisy photos?
As someone that works in medical image processing I feel qualified to spend my devember trying to 1 up the makers of the “AI Camera” mode on my LG V40.

The Plan:

  1. Generate data from my camera roll. One of the hardest steps in training sand to thinking is providing it with useful training data. I need to be able feed the AI a blurry and/or noisy photo and have a clean 1 to use as the ground truth. I could spend hours taking a blurry photo then a good photo or… I could use data augmentation. I’m picking the latter. I will use my existing photo library and generate training data by applying random noise (I need to check if it should be a Gaussian or Poisson distribution) and smoothing (to mimic blurring). This will give me the original image as the ground truth image.
  2. Decide on neural net architecture. At the moment I am leaning towards an architecture with a combination of linear filters with varying feature sizes and planar filters for the color channels. However, I will likely tackle this in 2 steps. First, creating a proof of concept net that works on 2D (greyscale) images and second, expanding that to a net for 3D (color) images. My thought being that noise is likely independent across color channels but is proportional to the number of photons received by the detector in each channel. Thus a relationship may exist (and be exploitable) between the amount of pixel to pixel variation in a 3x3 square and the true intensity of the color channel for that 3x3 square.
  3. Train and see how she looks.
  4. Execute the plan. Expect the plan to go off the rails. Throw away the plan.

I’m super excited to see how this turns out! Happy #Devember2020

5 Likes

This sounds like a hell of a project! I have a bunch of questions!
Mostly because I’m also tryign to get better at “programming 2.0” type stuff, and would love to share experiences.

Pytorch or Tensorflow?

You mentioned you’re generating the dataset from your own camera roll. Why not use a public dataset? Is it for the extra challenge of building the dataset too?

Do you know about paperswithcode.com? They have a bunch of different ML tasks, and solution papers (with code!) ranked by how well they do on a benchmark. It looks like they even have an Image Denoising task listed.
…maybe the Damstadt Noise Dataset could be of use to you?
I imagine the papers in there have thoughts on model architectures either way.

Do you have some way of benchmarking your LG’s denoising algorithm? I think it could be really interesting to actually put them head-to-head.

Is performance a priority? Usually it isn’t for a lot of ML people, but since you’re talking about models that run on phones, I got curious…

Are you planning on deploying it somewhere, or just on your dev box?
Surely you don’t mean to actually run it on a phone?
(I hear linode have GPU boxes… I wonder if they might be able to help).