urllib

Downloading a picture via urllib and python

by

So I’m trying to make a Python script that downloads webcomics and puts them in a folder on my desktop. I’ve found a few similar programs on here that do something similar, but nothing quite like what I need. The one that I found most similar is right here (http://bytes.com/topic/python/answers/850927-problem-using-urllib-download-images). I tried using this code: … Read more

How to send POST request?

by

I found this script online: import httplib, urllib params = urllib.urlencode({‘number’: 12524, ‘type’: ‘issue’, ‘action’: ‘show’}) headers = {“Content-type”: “application/x-www-form-urlencoded”, “Accept”: “text/plain”} conn = httplib.HTTPConnection(“bugs.python.org”) conn.request(“POST”, “”, params, headers) response = conn.getresponse() print response.status, response.reason 302 Found data = response.read() data ‘Redirecting to <a href=”http://bugs.python.org/issue12524″>http://bugs.python.org/issue12524</a>’ conn.close() But I don’t understand how to use it with … Read more

urllib and “SSL: CERTIFICATE_VERIFY_FAILED” Error

by

I am getting the following error: Exception in thread Thread-3: Traceback (most recent call last): File “/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py”, line 810, in __bootstrap_inner self.run() File “/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py”, line 763, in run self.__target(*self.__args, **self.__kwargs) File “/Users/Matthew/Desktop/Skypebot 2.0/bot.py”, line 271, in process info = urllib2.urlopen(req).read() File “/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py”, line 154, in urlopen return opener.open(url, data, timeout) File “/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py”, line 431, in … Read more

UnicodeEncodeError: ‘charmap’ codec can’t encode characters

by

I’m trying to scrape a website, but it gives me an error. I’m using the following code: import urllib.request from bs4 import BeautifulSoup get = urllib.request.urlopen(“https://www.website.com/”) html = get.read() soup = BeautifulSoup(html) print(soup) And I’m getting the following error: File “C:\Python34\lib\encodings\cp1252.py”, line 19, in encode return codecs.charmap_encode(input,self.errors,encoding_table)[0] UnicodeEncodeError: ‘charmap’ codec can’t encode characters in position … Read more

How to urlencode a querystring in Python?

by

I am trying to urlencode this string before I submit. queryString = ‘eventName=” + evt.fields[“eventName”] + “&’ + ‘eventDescription=’ + evt.fields[“eventDescription”]; 14 s 14 Python 2 What you’re looking for is urllib.quote_plus: safe_string = urllib.quote_plus(‘string_of_characters_like_these:$#@=?%^Q^$’) #Value: ‘string_of_characters_like_these%3A%24%23%40%3D%3F%25%5EQ%5E%24′ Python 3 In Python 3, the urllib package has been broken into smaller components. You’ll use urllib.parse.quote_plus (note … Read more

What are the differences between the urllib, urllib2, urllib3 and requests module?

by

In Python, what are the differences between the urllib, urllib2, urllib3 and requests modules? Why are there three? They seem to do the same thing… 1Best Answer 11 I know it’s been said already, but I’d highly recommend the requests Python package. If you’ve used languages other than python, you’re probably thinking urllib and urllib2 … Read more