Lesson Video:


This article is also a Jupyter Notebook available to be run from the top down. There will be code snippets that you can then run in any environment.

Below are the versions of fastai, fastcore, and wwf currently running at the time of writing this:

  • fastai: 2.1.9
  • fastcore: 1.3.12
  • wwf: 0.0.5

Using your Google Drive!

The paths in this lesson don't use the Google Drive, however, if you do want to use the Google Drive, you can replace birds with drive/My Drive/birds.

Note that content stored in data will be deleted whenever you close the notebook.

from google.colab import drive
drive.mount('/content/drive')
Go to this URL in a browser: https://accounts.google.com/o/oauth2/auth?client_id=947318989803-6bn6qk8qdgf4n4g3pfee6491hc0brc4i.apps.googleusercontent.com&redirect_uri=urn%3Aietf%3Awg%3Aoauth%3A2.0%3Aoob&scope=email%20https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fdocs.test%20https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fdrive%20https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fdrive.photos.readonly%20https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fpeopleapi.readonly&response_type=code

Enter your authorization code:
··········
Mounted at /content/drive

Creating your own dataset from Google Images

by: Francisco Ingham and Jeremy Howard. Inspired by Adrian Rosebrock

In this tutorial we will see how to easily create an image dataset through Google Images.

First let's install the new library again (if you do not have it already)

And import our libraries

from fastai.vision.all import *

Below you will find the exact imports for everything we use today

from fastcore.foundation import L
from fastcore.xtras import Path # @patch'd properties to the Pathlib module

from fastai.callback.fp16 import to_fp16
from fastai.callback.schedule import fit_one_cycle, lr_find 

from fastai.data.block import CategoryBlock, DataBlock
from fastai.data.transforms import get_image_files, Normalize, RandomSplitter, parent_label

from fastai.interpret import ClassificationInterpretation
from fastai.learner import Learner # imports @patch'd properties to Learner including `save`, `load`, `freeze`, and `unfreeze`

from fastai.vision.augment import aug_transforms, RandomResizedCrop
from fastai.vision.core import imagenet_stats
from fastai.vision.data import ImageBlock
from fastai.vision.learner import cnn_learner
from fastai.vision.utils import download_images, verify_images

Get a list of URLs

Search and scroll

Go to Google Images and search for the images you are interested in. The more specific you are in your Google Search, the better the results and the less manual pruning you will have to do.

Scroll down until you've seen all the images you want to download, or until you see a button that says 'Show more results'. All the images you scrolled past are now available to download. To get more, click on the button, and continue scrolling. The maximum number of images Google Images shows is 700.

It is a good idea to put things you want to exclude into the search query, for instance if you are searching for the Eurasian wolf, "canis lupus lupus", it might be a good idea to exclude other variants:

"canis lupus lupus" -dog -arctos -familiaris -baileyi -occidentalis

You can also limit your results to show only photos by clicking on Tools and selecting Photos from the Type dropdown.

Download image URLs into file

Now you must run some Javascript code in firefox which will save the URLs of all the images you want for you dataset.

Press CtrlShiftK in Windows/Linux and CmdOptJ in Mac, and a small window the javascript 'Console' will appear. That is where you will paste the JavaScript commands.

You will need to get the urls of each of the images. You can do this by running the following commands:

urls = Array.from(document.querySelectorAll('.rg_di .rg_meta')).map(el=>JSON.parse(el.textContent).ou);
window.open('data:text/csv;charset=utf-8,' + escape(urls.join('\n')));

Do this one at a time as you'll want to rename your files to SomeFileName.txt (with SomeFileName being replaced)

Create directory and upload URL file into your server

Choose an appropriate name for your labeled images.

folders = ['blue_bird', 'red_wing_black_bird', 'blue_jay']
files = ['urls_blue.txt', 'urls_rwbb.txt', 'urls_jay.txt']

Now let's make a folder for each category

Now, upload your urls files. You just need to press 'Upload' in your working directory (birds), select your url files and confirm the upload.

Download images

classes = ['blue_bird', 'red_wing_black_bird','blue_jay']
path = Path('birds')
path.mkdir(parents=True, exist_ok=True)

Let's download our images

for i, n in enumerate(classes):
  print(n)
  path_f = Path(files[i])
  download_images(path/n, path_f, max_pics=50)

And check they are all good. If not, we will delete them

imgs = L()
for n in classes:
  print(n)
  path_n = path/n
  imgs += verify_images(path_n.ls())
blue_bird
red_wing_black_bird
blue_jay
imgs
(#1) [Path('birds/blue_bird/00000047.jpg')]
for im in imgs:
  im.unlink()

Let's double check

imgs = L()
for n in classes:
  print(n)
  path_n = path/n
  imgs += verify_images(path_n.ls())
blue_bird
red_wing_black_bird
blue_jay
imgs
(#0) []

View data

Let's make our DataBlock again.

birds = DataBlock(blocks=(ImageBlock, CategoryBlock),
                  get_items=get_image_files,
                  splitter=RandomSplitter(0.2),
                  get_y=parent_label,
                  item_tfms=RandomResizedCrop(460),
                  batch_tfms=[*aug_transforms(size=224, max_warp=0), Normalize.from_stats(*imagenet_stats)])
dls = birds.dataloaders(path,  bs=32)

Good! Let's take a look at some of our pictures then.

dls.show_batch(max_n=9)
dls.vocab
(#3) ['blue_bird','blue_jay','red_wing_black_bird']

Train model

learn = cnn_learner(dls, resnet34, pretrained=True, metrics=error_rate).to_fp16()
learn.fit_one_cycle(4)
epoch train_loss valid_loss error_rate time
0 2.114906 1.519265 0.482759 00:04
1 2.065674 1.319101 0.482759 00:04
2 1.984945 1.072412 0.448276 00:04
3 1.925143 0.852549 0.344828 00:04
learn.save('stage-1')
learn.unfreeze()
learn.lr_find()
learn.fit_one_cycle(2, lr_max=slice(1e-4, 1e-3))
epoch train_loss valid_loss error_rate time
0 1.170554 0.385774 0.172414 00:04
1 0.802616 0.180987 0.103448 00:04
learn.save('stage-2')

Interpretation

learn.load('stage-2');
interp = ClassificationInterpretation.from_learner(learn)
interp.plot_confusion_matrix()
interp.plot_top_losses(3, figsize=(9,9))