Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Google Artwork Crawler #297

Open
wants to merge 26 commits into
base: master
Choose a base branch
from
Open

Google Artwork Crawler #297

wants to merge 26 commits into from

Conversation

marina-ferreira
Copy link

Summary

  1. This PR creates a gem for implementing artwork crawler solution

  2. Adds a new module for parsing artwork information from Google search results HTML pages, including artwork titles, links, images, and metadata.

    The following main classes were added:

    • Google::Artwork::Crawler
    • Google::Artwork::Parser
    • Google::Artwork::ImageReplacerScriptParser

    The images without base64 information fallback to the resource url.

  3. Adds a rake task for testing purposes

    Available Artists:

    • Van Gogh
    • Picasso
    • Leonardo da Vinci

Usage

rake search_artwork

@marina-ferreira
Add gem files
4170f19
@marina-ferreira
Add nokogiri dependency to gemspec
d171aa7
@marina-ferreira
Load rake tasks in Rakefile
f21ed3c
@marina-ferreira
Add search_artwork rake task
7edd138
@marina-ferreira
Add script to find artist html file for testing
f720d94
@marina-ferreira
Add Google::Artwork::Crawler class
8dd927e
@marina-ferreira
Add Google::Artwork::Crawler spec
22522e6
@marina-ferreira
Add app errors
6fc95b3
@marina-ferreira
Google::Artwork::Crawler non-existent file test case
4a74474
@marina-ferreira
Google::Artwork::Crawler unreadable file test case
aecea0e
@marina-ferreira
Google::Artwork::Crawler artwork not found test case
f0c0a6f
@marina-ferreira
Google::Artwork::Crawler artwork links not found test case
1513b92
@marina-ferreira
Update Rubocop version
f97d76f
@marina-ferreira
Helpers::String sanitize
70eafe2
@marina-ferreira
Artwork Parser class and spec
f7148c0
@marina-ferreira
Google::Artwork::Crawler return artwork title, link and extensions
8b9236b
@marina-ferreira
Add ImageReplacerScriptParser class
e1dcb6d
@marina-ferreira
Rename string helpers
f2bc796
@marina-ferreira
Add image mapping to crawler
4726bc5
@marina-ferreira
Extract Helpers::Base64.sanitize
8723151
@marina-ferreira
Add Picasso paintings for testing
4ce7527
@marina-ferreira
Add Leonardo da Vinci paintings for testing
9ba23ed
@marina-ferreira
Update rubocop rules
be673d7
@marina-ferreira
Add documentation comments to classes
3b17c66
@marina-ferreira
Extract parameterize to Helpers::String
0050d2d
@marina-ferreira
Update ruby version on gemspec
f6487a2
@andypple83 andypple83 self-assigned this Dec 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees

@andypple83 andypple83

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@marina-ferreira @andypple83