Here are the examples of the python api scrapy.utils.url.add_http_if_no_scheme taken from open source projects. By voting up you can indicate which examples are most useful and appropriate.
1 Examples
3
View Complete Implementation : middlewares.py
Copyright MIT License
Author : TeamHG-Memex
Copyright MIT License
Author : TeamHG-Memex
@clastmethod
def cleanup_proxy_list(cls, proxy_list):
lines = [line.strip() for line in proxy_list]
return list({
add_http_if_no_scheme(url)
for url in lines
if url and not url.startswith('#')
})