How do i set up a web crawler for youtube? or anything else i just need google to get confused about what i want lol.
I don't fully understand the question.
Do you want a webcrawler to get all the links on a certain URL (which would be a lot if you crawl YouTube).
Or do you want something like htaccess redirects?
uhh ... this is way over my head. I heard wendell talk about it in a tek episode.
You want to set up a web crawler but you don't know what for?
He wants something to search random shit so that google doesn't know what is a real search and what is.
3 Likes
You got a data cap on your connection OP? Cause this could get messy fast.
Wait what? What is web crawler?
How is this hard for people... He wants something to search and look at web pages so that people observing his connection can't tell what he is actually looking at because of all the junk data.
NAH, 300 MBPS. think i'll be good? Then again it is time warner lol.
Okay so this is an option, but you're gonna need a drive to dump shit to. Use HTTrack and set it to start downloading a page. You're gonna need to change some settings so that it won't actually download the page or you're gonna run out of space in a few hours and you're going to need to set it to go everywhere. That should confuse most of the internet. I would be careful with this though as it wouldn't know better than to look at every child porn website in existence on it's path to view the entire internet.
yeah and i'm sick and tired of youtube constantly recommending videos i want to watch, it gets me paranoid for no reason lol. BTW i have absolutely have nothing to hide and I don't know if you guys can relate but i just get weird-ed out by all this privacy invasion stuff going on.
1 Like
Dude chill we get it. If I had the time I would set up something like this myself. Maybe we can start an effort to build it within the community.
is there a way to filter that out? or like have it only search up to top 50 websites on the net? sorry im a noob with code stuff please be patient with me .
I personally use Ghostery, Adblock Plus and Adblock Fast to remove as much as I can.
There is. It's config settings. In order to keep it from downloading or going to something though I think you would need to know where it is to begin with. You can at least set depth settings for how many links deep it will go through before it stops.
1 Like
i got this cool addon called random agent spoofer it like spoofs your browser and pc screen res everytime. also does a bunch of other things. Check it out.
What browser?
Firefox
Github project maybe?
TekCrawler; Confuse the World!
3 Likes