grabbing an image from a website.

This is the place for queries that don't fit in any of the other categories.

grabbing an image from a website.

Postby siestamadokaist » Sun Mar 31, 2013 4:54 pm

Could anyone help me?
So, I have this code: (to capture an image from a url)
Code: Select all
import  urllib
source = urllib.urlopen('http://gimg.netmarble.co.id/elsword/images/News/img_visual.jpg'); #just any random image from any website
image = source.read();
destination = open('test1.jpg','w'); #my file
destination.write(image);
# print(image)

this code do give me a file, that's "Almost" similar to the actual image, however, there are some bits that differs between the captured image, and the actual image.
could anyone give me a solution to fix this?

Thank You.
siestamadokaist
 
Posts: 12
Joined: Sun Mar 31, 2013 4:48 pm

Re: grabbing an image from a website.

Postby snippsat » Sun Mar 31, 2013 6:33 pm

You should open the file in binary mode(wb),otherwise you may corrupt the image.
Or better use urlretrieve().
Code: Select all
from urllib import urlretrieve

url = 'http://gimg.netmarble.co.id/elsword/images/News/img_visual.jpg'
filename = url.strip().split('/')[-1]
urlretrieve(url, filename)
User avatar
snippsat
 
Posts: 221
Joined: Thu Feb 21, 2013 12:04 am

Re: grabbing an image from a website.

Postby siestamadokaist » Mon Apr 01, 2013 1:39 pm

Oou, thanks, it finally work, :D
Is it possible to increase the bandwidth used to grab the image?
apparently, it seems, I'm only using less than 8KBps to capture it.
#pretty sure my connection is faster.

And uh, is it possible to trace how much data is sent overtime, I mean, it'd be troubling, if the header are far too big compared to the actual data per packet.

Edit:
Kinda Solved.
urllib2.urlopen()
and f.write(,'wb')
works like magic
damn you python >,<
siestamadokaist
 
Posts: 12
Joined: Sun Mar 31, 2013 4:48 pm

Re: grabbing an image from a website.

Postby tnknepp » Mon Apr 01, 2013 8:04 pm

Not totally appropriate for your work, but in case you ever get to the point where you need to download larger files (I use for 200MB+ files), this works well.

Code: Select all
import shutil
from urllib2 import urlopen

f = urlopen('address_with_extension')
with open('fileName.ext','wb') as lf:
    shutil.copyfileobj(f,lf)
Python: 2.7 via Anaconda
Numpy: 1.7
Pandas: 0.11
OS: Windows 7
IDE: Spyder/IPython
User avatar
tnknepp
 
Posts: 123
Joined: Mon Mar 11, 2013 7:41 pm

Re: grabbing an image from a website.

Postby snippsat » Mon Apr 01, 2013 10:19 pm

I will also mention Requests: HTTP for Humans
urllib can be painful, documentation and API suck :x
Requests is the future for HTTP stuff in Python,and has become very popular(this is how documentation and API should be done).

Download image with Requests.
Code: Select all
import requests

r = requests.get("http://gimg.netmarble.co.id/elsword/images/News/img_visual.jpg")
with open("img_visual.jpg", "wb") as f:
    f.write(r.content)


but in case you ever get to the point where you need to download larger files (I use for 200MB+ files), this works well.

For requests we can do it this way,set stream=True and choose a chunk size here i choose 1024.
shutil.copyfileobj also use 1024 it has a third default argument length=16*1024
Code: Select all
import requests

r = requests.get("http://download.thinkbroadband.com/10MB.zip", stream=True)
with open('10MB.zip.', 'wb') as handle:
    for block in r.iter_content(1024):
        if not block:
            break
        handle.write(block)
User avatar
snippsat
 
Posts: 221
Joined: Thu Feb 21, 2013 12:04 am

Re: grabbing an image from a website.

Postby tnknepp » Tue Apr 02, 2013 12:14 pm

Cool, thanks. Nice to see "get" finally coming into Python.
Python: 2.7 via Anaconda
Numpy: 1.7
Pandas: 0.11
OS: Windows 7
IDE: Spyder/IPython
User avatar
tnknepp
 
Posts: 123
Joined: Mon Mar 11, 2013 7:41 pm

Re: grabbing an image from a website.

Postby siestamadokaist » Tue Apr 02, 2013 7:31 pm

I guess I'm noobing here,but how do I install requests-master?
it looks really interesting with all the chunk-block and cookies.
Do I just download the zip from github and put it in my lib/site-packages? or do I put it in my system32?

Dayum, All those $ give me a linux feeling.
siestamadokaist
 
Posts: 12
Joined: Sun Mar 31, 2013 4:48 pm

Re: grabbing an image from a website.

Postby tnknepp » Tue Apr 02, 2013 8:00 pm

You are correct that this is for a Linux install, but it can be done in Windows as well.

1. Download .zip file
2. Extract to Desktop (or anywhere else, but I used Desktop)
3. Open a command window (Click "Start" and type "cmd", then return)
4. Change directory to Desktop (or wherever you extracted the files to) by typing "cd Desktop\kennethreitz-requests-54ed5ed"
5. Now type "python setup.py install"

That should get you set. Now open an IDE and you should be able to import requests.
Python: 2.7 via Anaconda
Numpy: 1.7
Pandas: 0.11
OS: Windows 7
IDE: Spyder/IPython
User avatar
tnknepp
 
Posts: 123
Joined: Mon Mar 11, 2013 7:41 pm

Re: grabbing an image from a website.

Postby snippsat » Tue Apr 02, 2013 8:36 pm

They maual way as posted by tnknepp is ok.
But you should really set up easy_install and pip.
pip is the preferred installer.

When it works you open command line(cmd) and do pip install requests
Then it will download and install for you all in one go,this work for most python modules pip install some_module_you_need
If look at PyPi there is lot to choose from.
User avatar
snippsat
 
Posts: 221
Joined: Thu Feb 21, 2013 12:04 am

Re: grabbing an image from a website.

Postby siestamadokaist » Fri Apr 05, 2013 1:48 pm

phew, finally installed requests and py2.exe =.=
I have some question regarding requests, and something else
  1. Let's say i want to grab a picture, is it possible to get its total size before capturing it entirely? for progress bar.
  2. Is it possible to print a word, without moving to the next line in python 2.7?
  3. What is the correct scenario to use cookies? I mean I tried doing this:
    Logging in, with request.post -> save the cookies -> open another page with cookies=mycookies, but it seems I'm wrong
siestamadokaist
 
Posts: 12
Joined: Sun Mar 31, 2013 4:48 pm

Re: grabbing an image from a website.

Postby stranac » Fri Apr 05, 2013 6:03 pm

siestamadokaist wrote:Let's say i want to grab a picture, is it possible to get its total size before capturing it entirely? for progress bar.

Probably, if the server sends a Content-Length header.
siestamadokaist wrote:Is it possible to print a word, without moving to the next line in python 2.7?

Yes, adding a comma at the end of the print statement will do that.
Code: Select all
print 'word',

siestamadokaist wrote:What is the correct scenario to use cookies? I mean I tried doing this:
Logging in, with request.post -> save the cookies -> open another page with cookies=mycookies, but it seems I'm wrong

Use a requests.Session.
Friendship is magic!

R.I.P. Tracy M. You will be missed.
User avatar
stranac
 
Posts: 1144
Joined: Thu Feb 07, 2013 3:42 pm


Return to General Coding Help

Who is online

Users browsing this forum: Baldyr, Google [Bot], Yoriz and 4 guests