Парсер бесплатных прокси

Спасибо забрал :)

гм гзип может содержать внутри только один файл о каких разных пхп скриптах идет речь?

spambot добавил 13.03.2009 в 00:31

ТС исправь расширение на тгз ;)

spambot добавил 13.03.2009 в 00:32

ну или тем у кого не распаковывается адекватно как у меня например в тотал командере смените расширение на тгз

спасибо за замечание - поправил

в догонку раз тут такая халява

поделюсь своими первыми 140 строчками из своего родимого списка прокси листов:

http://www.proxy-list.net/anonymous-proxy-lists.shtml

http://www.proxy-list.net/fresh-proxy-lists.shtml

http://www.proxy4free.com/page3.html

http://www.proxy4free.com/page2.html

http://www.proxy4free.com/page1.html

http://www.digitalcybersoft.com/ProxyList/fresh-proxy-list.shtml?

http://www.digitalcybersoft.com/ProxyList/fresh-proxy-list.shtml

http://www.digitalcybersoft.com/ProxyList/fresh-proxy-list.shtml?L1

http://www.digitalcybersoft.com/ProxyList/fresh-proxy-list.shtml?L2

http://www.digitalcybersoft.com/ProxyList/fresh-proxy-list.shtml?L3

http://www.digitalcybersoft.com/ProxyList/fresh-proxy-list.shtml?L4

http://www.digitalcybersoft.com/ProxyList/fresh-proxy-list.shtml?L5

http://www.digitalcybersoft.com/ProxyList/fresh-proxy-list.shtml?Transparent

http://www.digitalcybersoft.com/ProxyList/fresh-proxy-list.shtml?HTML

http://www.digitalcybersoft.com/ProxyList/fresh-proxy-list.shtml?80

http://www.digitalcybersoft.com/ProxyList/fresh-proxy-list.shtml?81

http://www.digitalcybersoft.com/ProxyList/fresh-proxy-list.shtml?1080

http://www.digitalcybersoft.com/ProxyList/fresh-proxy-list.shtml?3128

http://www.digitalcybersoft.com/ProxyList/fresh-proxy-list.shtml?8000

http://www.digitalcybersoft.com/ProxyList/fresh-proxy-list.shtml?8080

http://www.xroxy.com/proxylist.php?port=&type=Anonymous&ssl=&country=&latency=3000&reliability=9000&sort=reliability&desc=true&pnum=0#table

http://www.xroxy.com/proxylist.php?port=&type=Anonymous&ssl=&country=&latency=3000&reliability=9000&sort=reliability&desc=true&pnum=1#table

http://www.xroxy.com/proxylist.php?port=&type=Anonymous&ssl=&country=&latency=3000&reliability=9000&sort=reliability&desc=true&pnum=2#table

http://www.xroxy.com/proxylist.php?port=&type=Anonymous&ssl=&country=&latency=3000&reliability=9000&sort=reliability&desc=true&pnum=3#table

http://www.xroxy.com/proxylist.php?port=&type=Anonymous&ssl=&country=&latency=3000&reliability=9000&sort=reliability&desc=true&pnum=4#table

http://www.xroxy.com/proxylist.php?port=&type=Anonymous&ssl=&country=&latency=3000&reliability=7500&sort=reliability&desc=true&pnum=0#table

http://www.xroxy.com/proxylist.php?port=&type=Anonymous&ssl=&country=&latency=3000&reliability=7500&sort=reliability&desc=true&pnum=1#table

http://www.xroxy.com/proxylist.php?port=&type=Anonymous&ssl=&country=&latency=3000&reliability=7500&sort=reliability&desc=true&pnum=2#table

http://www.xroxy.com/proxylist.php?port=&type=Anonymous&ssl=&country=&latency=3000&reliability=7500&sort=reliability&desc=true&pnum=3#table

http://www.xroxy.com/proxylist.php?port=&type=Anonymous&ssl=&country=&latency=3000&reliability=7500&sort=reliability&desc=true&pnum=4#table

http://www.xroxy.com/proxylist.php?port=&type=Anonymous&ssl=&country=&latency=5000&reliability=7500&sort=reliability&desc=true&pnum=0#table

http://www.xroxy.com/proxylist.php?port=&type=Anonymous&ssl=&country=&latency=5000&reliability=7500&sort=reliability&desc=true&pnum=1#table

http://www.xroxy.com/proxylist.php?port=&type=Anonymous&ssl=&country=&latency=5000&reliability=7500&sort=reliability&desc=true&pnum=2#table

http://www.xroxy.com/proxylist.php?port=&type=Anonymous&ssl=&country=&latency=5000&reliability=7500&sort=reliability&desc=true&pnum=3#table

http://www.xroxy.com/proxylist.php?port=&type=Anonymous&ssl=&country=&latency=5000&reliability=7500&sort=reliability&desc=true&pnum=4#table

http://www.xroxy.com/proxylist.php?port=&type=Anonymous&ssl=&country=&latency=5000&reliability=9000&sort=reliability&desc=true&pnum=0#table

http://www.xroxy.com/proxylist.php?port=&type=Anonymous&ssl=&country=&latency=5000&reliability=9000&sort=reliability&desc=true&pnum=1#table

http://www.xroxy.com/proxylist.php?port=&type=Anonymous&ssl=&country=&latency=5000&reliability=9000&sort=reliability&desc=true&pnum=2#table

http://www.xroxy.com/proxylist.php?port=&type=Anonymous&ssl=&country=&latency=5000&reliability=9000&sort=reliability&desc=true&pnum=3#table

http://www.xroxy.com/proxylist.php?port=&type=Anonymous&ssl=&country=&latency=5000&reliability=9000&sort=reliability&desc=true&pnum=4#table

http://www.xroxy.com/proxylist.php?port=&type=&ssl=&country=&latency=&reliability=&sort=reliability&desc=true&pnum=0#table

http://www.xroxy.com/proxylist.php?port=&type=&ssl=&country=&latency=&reliability=&sort=reliability&desc=true&pnum=1#table

http://www.xroxy.com/proxylist.php?port=&type=&ssl=&country=&latency=&reliability=&sort=reliability&desc=true&pnum=2#table

http://www.xroxy.com/proxylist.php?port=&type=&ssl=&country=&latency=&reliability=&sort=reliability&desc=true&pnum=3#table

http://www.xroxy.com/proxylist.php?port=&type=&ssl=&country=&latency=&reliability=&sort=reliability&desc=true&pnum=4#table

http://www.xroxy.com/proxylist.php?port=&type=All_http&ssl=&country=&latency=3000&reliability=&sort=reliability&desc=true&pnum=0#table

http://www.xroxy.com/proxylist.php?port=&type=All_http&ssl=&country=&latency=3000&reliability=&sort=reliability&desc=true&pnum=1#table

http://www.xroxy.com/proxylist.php?port=&type=All_http&ssl=&country=&latency=3000&reliability=&sort=reliability&desc=true&pnum=2#table

http://www.xroxy.com/proxylist.php?port=&type=All_http&ssl=&country=&latency=3000&reliability=&sort=reliability&desc=true&pnum=3#table

http://www.xroxy.com/proxylist.php?port=&type=All_http&ssl=&country=&latency=3000&reliability=&sort=reliability&desc=true&pnum=4#table

http://www.xroxy.com/proxylist.php?port=&type=Anonymous&ssl=&country=&latency=3000&reliability=&sort=reliability&desc=true&pnum=0#table

http://www.xroxy.com/proxylist.php?port=&type=Anonymous&ssl=&country=&latency=3000&reliability=&sort=reliability&desc=true&pnum=1#table

http://www.xroxy.com/proxylist.php?port=&type=Anonymous&ssl=&country=&latency=3000&reliability=&sort=reliability&desc=true&pnum=2#table

http://www.xroxy.com/proxylist.php?port=&type=Anonymous&ssl=&country=&latency=3000&reliability=&sort=reliability&desc=true&pnum=3#table

http://www.xroxy.com/proxylist.php?port=&type=Anonymous&ssl=&country=&latency=3000&reliability=&sort=reliability&desc=true&pnum=4#table

http://www.proxz.com/proxy_list_high_anonymous_16.html

http://www.proxz.com/proxy_list_high_anonymous_15.html

http://www.proxz.com/proxy_list_high_anonymous_14.html

http://www.proxz.com/proxy_list_high_anonymous_13.html

http://www.proxz.com/proxy_list_high_anonymous_12.html

http://www.proxz.com/proxy_list_high_anonymous_11.html

http://www.proxz.com/proxy_list_high_anonymous_10.html

http://www.proxz.com/proxy_list_high_anonymous_9.html

http://www.proxz.com/proxy_list_high_anonymous_8.html

http://www.proxz.com/proxy_list_high_anonymous_7.html

http://www.proxz.com/proxy_list_high_anonymous_6.html

http://www.proxz.com/proxy_list_high_anonymous_5.html

http://www.proxz.com/proxy_list_high_anonymous_4.html

http://www.proxz.com/proxy_list_high_anonymous_3.html

http://www.proxz.com/proxy_list_high_anonymous_2.html

http://www.proxz.com/proxy_list_high_anonymous_1.html

http://www.proxz.com/proxy_list_high_anonymous_0.html

http://www.proxz.com/proxy_list_port_std_14.html

http://www.proxz.com/proxy_list_port_std_13.html

http://www.proxz.com/proxy_list_port_std_12.html

http://www.proxz.com/proxy_list_port_std_11.html

http://www.proxz.com/proxy_list_port_std_10.html

http://www.proxz.com/proxy_list_port_std_9.html

http://www.proxz.com/proxy_list_port_std_8.html

http://www.proxz.com/proxy_list_port_std_7.html

http://www.proxz.com/proxy_list_port_std_6.html

http://www.proxz.com/proxy_list_port_std_5.html

http://www.proxz.com/proxy_list_port_std_4.html

http://www.proxz.com/proxy_list_port_std_3.html

http://www.proxz.com/proxy_list_port_std_2.html

http://www.proxz.com/proxy_list_port_std_1.html

http://www.proxz.com/proxy_list_port_std_0.html

http://www.proxz.com/proxy_list_ca_1.html

http://www.proxz.com/proxy_list_ca_0.html

http://www.proxz.com/proxy_list_uk_1.html

http://www.proxz.com/proxy_list_uk_0.html

http://www.proxz.com/proxy_list_anonymous_us_4.html

http://www.proxz.com/proxy_list_anonymous_us_3.html

http://www.proxz.com/proxy_list_anonymous_us_2.html

http://www.proxz.com/proxy_list_anonymous_us_1.html

http://www.proxz.com/proxy_list_anonymous_us_0.html

http://www.my-proxy.com/list/proxy.php?list=10

http://www.my-proxy.com/list/proxy.php?list=9

http://www.my-proxy.com/list/proxy.php?list=8

http://www.my-proxy.com/list/proxy.php?list=7

http://www.my-proxy.com/list/proxy.php?list=6

http://www.my-proxy.com/list/proxy.php?list=5

http://www.my-proxy.com/list/proxy.php?list=4

http://www.my-proxy.com/list/proxy.php?list=3

http://www.my-proxy.com/list/proxy.php?list=2

http://www.my-proxy.com/list/proxy.php?list=1

http://www.my-proxy.com/list/proxy.php

http://www.my-proxy.com/list/proxy.php?list=s1

http://www.my-proxy.com/list/proxy.php?list=s2

http://www.my-proxy.com/list/proxy.php?list=s3

http://www.checkedproxylists.com/

http://www.showlist.info/index.php

http://www.showlist.info/index.php?act=list&port=&type=&country=&page=1

http://www.showlist.info/index.php?act=list&port=&type=&country=&page=2

http://www.showlist.info/index.php?act=list&port=&type=&country=&page=3

http://www.showlist.info/index.php?act=list&port=&type=&country=&page=4

http://www.freshproxylist.org/index.php

http://proxy.antipalivo.ru/

http://www.proxylists.net/

http://www.proxylists.net/http_highanon.txt

http://www.proxylists.net/proxylist.shtml?HTTP

http://www.atomintersoft.com/products/alive-proxy/proxy-list

http://www.atomintersoft.com/products/alive-proxy/proxy-list/Free_Open_Public_HTTP_Proxies_sorted_by_countries

http://www.atomintersoft.com/transparent_proxy_list

http://www.atomintersoft.com/anonymous_proxy_list

http://www.atomintersoft.com/high_anonymity_elite_proxy_list

http://labs.involutive.com/anon/?anon_type=1&page=1

http://labs.involutive.com/anon/?anon_type=1&page=0

http://proxylist.sakura.ne.jp/index.htm?pages=12

http://proxylist.sakura.ne.jp/index.htm?pages=11

http://proxylist.sakura.ne.jp/index.htm?pages=10

я не узнаю этот форум :)

дело не в форуме, а в разделе :))

Добро победит зло! поставит на колени и убьет выстрелом в голову!

добавление:

скрипт proxy_get работает достаточно быстро, а вот

proxy_check.php кажется, что зависает, но на самом деле работает.

много тормозных или вообще не рабочих прикси дают отклик до 4 секунд, поэтому максимальное время работы скрипта может быть до ([количество прокси]*4) секунд

1. Регулярки для поиска прокси:

  "#(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}|[\w\-]{2,20}\.\w{2,4})\:(\d{2,6})#i",
  "#<td[^>]*>\s*(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}|[\w\-]{2,20}\.\w{2,4})\s*</td>\s*<td[^>]*>\s*(\d{2,6})\s*</td>#si",
  "#\<\/td\>\<td\snowrap\>([\w.-]+?)\<\/td\>\<td\>(\d+)\<\/td\>\<td\>#si",
  "#<td>([\w.-]+?)<\/td><td>(\d+)<\/td><td>#si",
  "#\s<td>(\d+\.\d+\.\d+\.\d+)<\/td>\D+\s<td>(\d+)<\/td>#si",
  "#bgcolor=\#ffffff>([\w.-]+?)<\/td><td\swidth=\"25%\"\snowrap\sbgcolor=\#ffffff>(\d+)<\/td>#si",
  "#<li><a\shref=\"\/cgi-bin\/fp.pl.*?>(.+?)<\/a>#si",
  "#\sheight=17>([\w.-]+\.\w+)<\/td>\D+\sheight=17>(\d+)<\/td>#si",
  "#\>(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})\<\/div\>\<\/td\>\s*<td\>\d{2,6}\<\/td\>#si",
  "#\>(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}):\s*(\d{2,6})\<\/td\>#si"

2. Для того, чтоб скрипт не вылетал на 500 ошибке и чтоб не вис нужно добавить перед концом цикла (перед фигурной скобкой) у обеих файлов:

@ob_flush(); @flush();

3. В проверке прокси не учтено следующее:

- CURLOPT_PROXYPORT

- CURLOPT_PROXYTYPE (значения, которые может принимать: CURLPROXY_SOCKS4, CURLPROXY_SOCKS5, CURLPROXY_HTTP)

- CURLOPT_PROXYUSERPWD (пусто должно быть)

Если всё это сделайте, то будет норм скрипт, а так - даже не похоже на "пять минут на колене" =)

я только http парсил, поэтому не учитывал, а за остальное спасибо :)