Hot Downloads

Welcome, Guest
Username: Password: Remember me

TOPIC: Anonymous wget

Anonymous wget 12 years 2 months ago #5424

  • biund
  • biund's Avatar
  • Offline
  • New Member
  • Posts: 3
  • Karma: 0
I would like to rip some content from a site using wget, by gathering a bunch of generated pages containing the data that interests me. Each page is of just a few KB.

I am thinking of running a perl script doing basically the following.
[code:1]
for $i ( 1 .. 5000 ) {
system("wget -O $i.htm http://thesite.com/content/?id=$i");
sleep 5; # just a few secs not to kill the server
}[/code:1]

I would like to proceed by running the script, but for various reasons I'd like to do it anonymously i.e. making the target site unaware of the origin of the requests.

Any ideas about that might be arranged?
The administrator has disabled public write access.

Re: Anonymous wget 12 years 2 months ago #5427

  • nske
  • nske's Avatar
  • Offline
  • Expert Member
  • Posts: 613
  • Karma: 0
you can set the $http_proxy enviromental var to an anonymous proxy's address and use wget with --proxy=on parameter.
The administrator has disabled public write access.

Re: Anonymous wget 12 years 2 months ago #5429

  • biund
  • biund's Avatar
  • Offline
  • New Member
  • Posts: 3
  • Karma: 0
Thanks.

Any hints on where I can find a truly anonymous proxy server appreciated.
The administrator has disabled public write access.

Re: Anonymous wget 12 years 2 months ago #5430

  • nske
  • nske's Avatar
  • Offline
  • Expert Member
  • Posts: 613
  • Karma: 0
there are many in www.stayinvisible.com as well as a service to check whether they are indeed anonymous ;)
The administrator has disabled public write access.

Re: Anonymous wget 12 years 2 months ago #5441

  • biund
  • biund's Avatar
  • Offline
  • New Member
  • Posts: 3
  • Karma: 0
Thanks a lot, nske. Amazing :shock:

Exactly what I was looking for. stayinvisible.com seems just right - others I've seen not as good.
The administrator has disabled public write access.
Time to create page: 0.078 seconds

CCENT/CCNA

Cisco Routers

  • SSL WebVPN
  • Securing Routers
  • Policy Based Routing
  • Router on-a-Stick

VPN Security

  • Understand DMVPN
  • GRE/IPSec Configuration
  • Site-to-Site IPSec VPN
  • IPSec Modes

Cisco Help

  • VPN Client Windows 8
  • VPN Client Windows 7
  • CCP Display Problem
  • Cisco Support App.

Windows 2012

  • New Features
  • Licensing
  • Hyper-V / VDI
  • Install Hyper-V

Linux

  • File Permissions
  • Webmin
  • Groups - Users
  • Samba Setup