DeviantART gallery scrapper

This is the place to post any code that you want to share with the community. Only completed scripts should be posted here.

DeviantART gallery scrapper

Postby kamiyl » Sun Feb 17, 2013 6:50 pm

As my first post, I wanted to share a fairly simple but ( I hope ) useful script.

You see, I spend a lot of time browsing deviantART, and there are some pretty good artists whose work I like specially. So much that I sometimes feel like downloading their entire gallery! But because individually downloading 20 and so pictures is boring, I decided to make a small program to automate that for me. It works in two phases, two functions:
  1. getImgUrls, which pulls the images' urls from the gallery's HTML source code using a regular expression;
  2. downloadImgs, that, well... downloads the images one by one onto your filesystem.
It's not very well documented, although both functions have a ( unintentionally weird ) description, and some comments are included. It works well ( ignoring urllib2's speed problems ), but it can't download the entire gallery, so to speak. It only downloads the images in the page whose url you passed as an argument. So, if you want to download more than one page of images, you must enter all pages' urls individually. I will try to tackle this problem in the meantime, as well as the fact that if you want to save in the images in a folder you must create it manually and then pass the path as an argument to downloadImgs.

As for its usage, I recommend that you import the script into another .py file and use its functions there, for the sake of modularity. Here's an example usage:
Code: Select all
from deviantARTdownloader import getImgUrls, downloadImgs

# url is the gallery's address / core_name is used to name the images
d = getImgUrls(url="http://blueraincz.deviantart.com/gallery/", core_name="blueraincz") # getImgUrls returns a dictionary

# urlDict is a dictionary returned by getImgUrls / imgFormat is the image's format; 'jpg' is the default / path is the path to which the files will be saved
downloadImgs(urlDict=d, imgFormat='jpg', path="/home/myHome/blueraincz") # this downloads the file


Finally, the script is protected by the GNU general public license. Hence, as long as you do not violate the terms of the license, and distribute all sub-products under the GNU general public license, you can do whatever you wish this script.

WARNING: This works in version 2.7. Later versions maybe incompatible with this script.

deviantARTdownloader.py.zip
(1.29 KiB) Downloaded 134 times
kamiyl
 
Posts: 2
Joined: Sun Feb 17, 2013 5:56 pm

Re: DeviantART gallery scrapper

Postby micseydel » Sun Feb 17, 2013 10:51 pm

Thanks for the script, and welcome to the forum! Also, thank you for using code tags and posting in the proper place. You might want to stick this on github and create issues for the changes you want to make. I don't use DeviantART, but someone else who does might fork your project or ask if they could contribute to yours!
Join the #python-forum IRC channel on irc.freenode.net!
User avatar
micseydel
 
Posts: 940
Joined: Tue Feb 12, 2013 2:18 am
Location: Mountain View, CA


Return to Completed Scripts

Who is online

Users browsing this forum: No registered users and 1 guest