Puppy sleep/wake classifier for security camera

Motivation: 

Asleep

I wake up at a consistent time every morning to take my puppy out of her play pen / crate and give her breakfast. However, sometimes after late nights of work I prefer to sleep in a little if possible, and sometimes my puppy is not awake at the early hours of the morning. If I can time my alarm to match when my puppy wakes up, I may be able to get some more sleep.

Solution: 

Using a security camera and a ML model, I can classify whether my puppy is asleep or awake, then send this information to an iPhone app that wakes me up if necessary.

Awake

Progress: 

So far I have completed the data collection and training of the model. In total, I collected and manually labeled 1137 photos using a script that retrieves photos from the security camera. I then trained a CNN (transfer learning using ResNet-50).  These are the bootstrap 95% confidence intervals of key metrics:

• Accuracy: 0.951 to 0.991
• AUROC:  0.988 to 0.9996
• F1 Score: 0.934 to 0.988

Some techniques that allowed the good performance are as follows:

• Weighted cross-entropy loss to account for the class imbalance
• L2 norm regularization (weight decay) to reduce overfit
• Drop out layers to reduce overfit
• Cropping all images during preprocessing to isolate puppy's playpen, ignoring irrelevant noise in the photos
• Using a large learning rate paired with step decay learning rate scheduler to find better optima
• Hyperparameter search

Github: https://github.com/kevsup/umi_vision