Web scraping - case study - OCR

# Packages:
  
require(magrittr) # pipe
require(rvest) # web scraping
require(stringi) # text cleaning
require(readr) # file reading
require(magick) # image processing 
require(tesseract) # OCR
require(googledrive) # Google Drive

# download webpage
s <- paste0( 
  'https://www.analizy.pl/',
  'fundusze/fundusze-inwestycyjne/notowania') %>%
  html_session

# data extraction
notowania <- s %>%
  # noteTable is the only element on the page
  html_node("#noteTable") %>% 
  # the header has two levels
  html_table(header = TRUE) %>% 
  .[,-1] %>% 
  set_colnames(.[1,]) %>% 
  # the first column is omitted (its image)
  .[-1,] %T>% 
  # summary
  str

notowania$`j.u. netto` <- s %>%
  html_nodes("tbody img[alt*='kurs']") %>%
  html_attr("src") %>%
  paste0("https://www.analizy.pl", .)

TESSERACT

This package is available on the GitHub. You can use a standard install.packages("tesseract"). Additional libraries need to be installed under Linux, but everything is explained in the source.

# OCR settings
engine <-
  tesseract(options = 
    list(tessedit_char_whitelist = " 0123456789,.",
         tessedit_char_blacklist = "!?@#$%&*()<>_-+=/:;'\"ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz\n\t\r",
         classify_bln_numeric_mode = "1"))

# text cleaning
text_clearing <- function(x) {
  x %>%
    stri_replace_all_regex("[ \n]", "") %>%
    stri_replace_all_regex("[,]+", ".") %>%
    stri_extract_first_regex('[0-9]+[//.]{0,1}[0-9]{0,2}') %>%
    as.numeric %>%
    format(2)
}

# sample conversion
notowania[['j.u. netto']][1] %>%
  image_read %>%
  ocr(engine) %>%
  text_clearing

(obraz) vs 1254.97 (OCR)

IT WORKS!!!

---

[UPDATE - March 2020]

When I published this post in 2018, the tesseract library did not work as well as it does now. To get good results, you needed to resize the image (by 30%) using the image_resize function from the magick library. As you can see, this tool is constantly being improved (but it’s still not perfect).

---

Below are the results for the first ten images:

(obraz) vs 1254.97 (OCR)

(obraz) vs 959.51 (OCR)

(obraz) vs 126.72 (OCR)

(obraz) vs 104.24 (OCR)

(obraz) vs 1481.4 (OCR)

(obraz) vs 129.41 (OCR)

(obraz) vs 101.11 (OCR)

(obraz) vs 107.41 (OCR)

(obraz) vs 104.46 (OCR)

(obraz) vs 83.55 (OCR)

After analyzing all the pictures, it turns out that this solution has an efficiency of 95%. Pretty good.

GOOGLE DRIVE

To use the googledrive package, you only need to have a Gmail account (it is necessary to authorize the connection with Google Drive). The principle of OCR based on Google disk is simple: I send a file to a Google disk and convert it to a Google document. Then I download the converted file in any form (e.g., as a text file).

OCRbyGoogleDrive <- function(urls) {
  # download images
  imgs <- list()
  for (url in urls) {
    imgs %<>%
      append(list(list(url = url,
                       fn  = tempfile())))
  }
  lapply(imgs, function(x)
    {image_read(x$url) %>%
      image_write(x$fn, format = "png")})

  # send images to Google Drive
  gd_imgs <-
    lapply(imgs, function(x)
      {drive_upload(x$fn, type = "png")})

  # convert to Google documents
  gd_imgs_cp <-
    lapply(gd_imgs, function(x)
      {drive_cp(as_id(x$id),
                mime_type = drive_mime_type("document"))})

  # write as txt files
  txt_files <-
    lapply(gd_imgs_cp, function(x)
      {drive_download(as_id(x$id), type = "txt")})

  # text cleaning
  res <-
    sapply(txt_files, function(x)
      {read_file(x$local_path)}) %>%
    stri_replace_all_regex("[ ,\\.]", "") %>%
    stri_extract_last_regex("[0-9]+") %>%
    as.numeric %>%
    divide_by(100)

  # remove temporary files
  imgs %>% lapply(function(x)
    unlink(x$fn))
  txt_files %>% lapply(function(x)
    unlink(x$local_path))
  gd_imgs %>% lapply(function(x)
    drive_rm(as_id(x$id)))
  gd_imgs_cp %>% lapply(function(x)
    drive_rm(as_id(x$id)))

  return(cbind(urls, res))
}

Below are the results for the first ten images:

(obraz) vs 1254.27 (OCR)

(obraz) vs 1959.51 (OCR)

(obraz) vs 126.72 (OCR)

(obraz) vs 104.24 (OCR)

(obraz) vs 1481.4 (OCR)

(obraz) vs 129.41 (OCR)

(obraz) vs 101.11 (OCR)

(obraz) vs 107.41 (OCR)

(obraz) vs 104.46 (OCR)

(obraz) vs 83.55 (OCR)

In this case, the problem appeared already in the first ten results - incorrect reading of the value 9 for the second image. However, the analysis of all images indicates that OCR by Google Drive has similar efficacy as a tesseract.

The results in both cases proved to be quite good, but unsatisfactory (I expect 100% effectiveness). In the next steps, you would have to think about other image modifications or additional cleaning functions. Alternatively, you can combine both methods. For more advanced users, you can try teaching tesseract to recognize characters on a specific data set.