KenPC
1
I came across this script to use when downloading packages off of repos. Problem is, when coming across larger files, it slows to a crrawwllllll
Is there a way to edit this to skip files that take longer than say 60 seconds to download and prinline what it skipped at the end?
Or if you know why it slows down on larger files and can find a way to make it not do that, that would be amazing.
Please ignore the auto formatting
$page = 'http://URL/Goes/here/'
$uri = [System.Uri]($page)
$listing = Invoke-WebRequest -Uri $uri
$uris = ForEach($link in $listing.Links.href) {
$link_uri = [System.URI]($link)
If($link_uri.IsAbsoluteUri) {
$link_uri
} Else {
If([System.Uri]::TryCreate($uri,$link,[ref]$link_uri)) {
$link_uri
}
}
}
ForEach($item in $uris) {
If( $uri.Authority -eq $item.Authority ) {
$file = $item.segments[-1]
Invoke-WebRequest -Uri $item -OutFile $file
}
}
I was working on a script yesterday and had to input a wait-job into it. I'm not a PS guru by any means, I just get by when I need one.
Maybe something like this?
$timeoutSeconds = 120
$get-repo = Start-Job -ScriptBlock
{
$page = 'http://URL/Goes/here/'
$uri = [System.Uri]($page)
$listing = Invoke-WebRequest -Uri $uri
$uris = ForEach($link in $listing.Links.href) {
$link_uri = [System.URI]($link)
If($link_uri.IsAbsoluteUri) {
$link_uri
} Else {
If([System.Uri]::TryCreate($uri,$link,[ref]$link_uri)) {
$link_uri
}
}
}
ForEach($item in $uris) {
If( $uri.Authority -eq $item.Authority ) {
$file = $item.segments[-1]
Invoke-WebRequest -Uri $item -OutFile $file
}
}
Wait-Job $get-repo -Timeout $timeoutSeconds | out-null
if ($get-repo.State -eq "Completed") { "done!" }
elseif ($get-repo.State -eq "Running") { "interrupted" }
else { "???" }
KenPC
3
Doesn't run
I'm trying to DL this repo
http://mirror.centos.org/centos-6/6.8/os/x86_64/Packages/
Eden
4
Not tested it but change
Invoke-WebRequest -Uri $item -OutFile $file
to
Invoke-WebRequest -TimeoutSec 60 -Uri $item -OutFile $file
(youd still need code to catch the uri that timedout)
Reference: https://msdn.microsoft.com/en-us/powershell/reference/5.1/microsoft.powershell.utility/invoke-webrequest
to do code blocks suround in
```lang-name
code
```
Doesn't surprise me, nothing I ever type out runs the first time. What version of powershell are you using?
Eden
6
The invoke-webrequest page says it should throw a WebException , so you should be able to catch that and get the uri
This one ran for me.
$timeoutSeconds = 120
$j = Start-Job -ScriptBlock{
$page = 'http://mirror.centos.org/centos-6/6.8/os/x86_64/Packages/'
$uri = [System.Uri]($page)
$listing = Invoke-WebRequest -Uri $uri
$uris = ForEach($link in $listing.Links.href) {
$link_uri = [System.URI]($link)
If($link_uri.IsAbsoluteUri) {
$link_uri
} Else {
If([System.Uri]::TryCreate($uri,$link,[ref]$link_uri)) {
$link_uri
}
}
}
ForEach($item in $uris) {
If( $uri.Authority -eq $item.Authority ) {
$file = $item.segments[-1]
Invoke-WebRequest -Uri $item -OutFile $file
}
}
Wait-Job $j -Timeout $timeoutSeconds | out-null
if ($j.State -eq "Completed") { "done!" }
elseif ($j.State -eq "Running") { "interrupted" }
else { "???" }
Err, actually it just errored out after 120 seconds. You may be able to fix it if @Eden code doesn't work though.
KenPC
9
weird, it just opens and closes for me
+ $j = Start-Job -ScriptBlock{
+ ~
Missing closing '}' in statement block or type definition.
+ CategoryInfo : ParserError: (:) [], ParentContainsErrorRecordException
+ FullyQualifiedErrorId : MissingEndCurlyBrace
I missed a curly brace:
$timeoutSeconds = 15
$j = Start-Job -ScriptBlock{
$page = 'http://mirror.centos.org/centos-6/6.8/os/x86_64/Packages/'
$uri = [System.Uri]($page)
$listing = Invoke-WebRequest -Uri $uri
$uris = ForEach($link in $listing.Links.href) {
$link_uri = [System.URI]($link)
If($link_uri.IsAbsoluteUri) {
$link_uri
} Else {
If([System.Uri]::TryCreate($uri,$link,[ref]$link_uri)) {
$link_uri
}
}
}
}
ForEach($item in $uris) {
If( $uri.Authority -eq $item.Authority ) {
$file = $item.segments[-1]
Invoke-WebRequest -Uri $item -OutFile $file
}
}
Wait-Job $j -Timeout $timeoutSeconds | out-null
if ($j.State -eq "Completed") { "done!" }
elseif ($j.State -eq "Running") { "interrupted" }
else { "???" }
Change the 15 to whatever you want at the top btw. I put 15 for testing.
1 Like
KenPC
11
Looks good, not really working right now but that may be due to my piss poor connection at work right now.
Although, for your exit code, it is only up for a second and then the window closes. Is there a possible wait for user input before closing?
KenPC
13
does it start downloading files for you if you try it?