Any Powershell gurus?

I came across this script to use when downloading packages off of repos. Problem is, when coming across larger files, it slows to a crrawwllllll

Is there a way to edit this to skip files that take longer than say 60 seconds to download and prinline what it skipped at the end?

Or if you know why it slows down on larger files and can find a way to make it not do that, that would be amazing. :slight_smile:

Please ignore the auto formatting :confused:

$page = 'http://URL/Goes/here/'
$uri = [System.Uri]($page)
$listing = Invoke-WebRequest -Uri $uri

$uris = ForEach($link in $listing.Links.href) {
    $link_uri = [System.URI]($link)

    If($link_uri.IsAbsoluteUri) {
        $link_uri
    } Else {
        If([System.Uri]::TryCreate($uri,$link,[ref]$link_uri)) {
            $link_uri
        }
    }
}

ForEach($item in $uris) {
    If( $uri.Authority -eq $item.Authority ) {
        $file = $item.segments[-1]
        Invoke-WebRequest -Uri $item -OutFile $file
    }
}

I was working on a script yesterday and had to input a wait-job into it. I'm not a PS guru by any means, I just get by when I need one.

Maybe something like this?

$timeoutSeconds = 120
$get-repo = Start-Job -ScriptBlock

 {
    $page = 'http://URL/Goes/here/'
$uri = [System.Uri]($page)
$listing = Invoke-WebRequest -Uri $uri

$uris = ForEach($link in $listing.Links.href) {
    $link_uri = [System.URI]($link)

    If($link_uri.IsAbsoluteUri) {
        $link_uri
    } Else {
        If([System.Uri]::TryCreate($uri,$link,[ref]$link_uri)) {
            $link_uri
        }
    }
}

ForEach($item in $uris) {
    If( $uri.Authority -eq $item.Authority ) {
        $file = $item.segments[-1]
        Invoke-WebRequest -Uri $item -OutFile $file
    }
}

Wait-Job $get-repo -Timeout $timeoutSeconds | out-null
if ($get-repo.State -eq "Completed") { "done!" }
elseif ($get-repo.State -eq "Running") { "interrupted" }
else { "???" }

Doesn't run :confused:

I'm trying to DL this repo

http://mirror.centos.org/centos-6/6.8/os/x86_64/Packages/

Not tested it but change

Invoke-WebRequest -Uri $item -OutFile $file

to

Invoke-WebRequest -TimeoutSec 60 -Uri $item -OutFile $file

(youd still need code to catch the uri that timedout)

Reference: https://msdn.microsoft.com/en-us/powershell/reference/5.1/microsoft.powershell.utility/invoke-webrequest

to do code blocks suround in

```lang-name
code
```

Doesn't surprise me, nothing I ever type out runs the first time. What version of powershell are you using?

The invoke-webrequest page says it should throw a WebException , so you should be able to catch that and get the uri

5.0.10586.672

This one ran for me.

$timeoutSeconds = 120
$j = Start-Job -ScriptBlock{
    $page = 'http://mirror.centos.org/centos-6/6.8/os/x86_64/Packages/'
$uri = [System.Uri]($page)
$listing = Invoke-WebRequest -Uri $uri

$uris = ForEach($link in $listing.Links.href) {
    $link_uri = [System.URI]($link)

    If($link_uri.IsAbsoluteUri) {
        $link_uri
    } Else {
        If([System.Uri]::TryCreate($uri,$link,[ref]$link_uri)) {
            $link_uri
        }
    }
}

ForEach($item in $uris) {
    If( $uri.Authority -eq $item.Authority ) {
        $file = $item.segments[-1]
        Invoke-WebRequest -Uri $item -OutFile $file
    }
}

Wait-Job $j -Timeout $timeoutSeconds | out-null
if ($j.State -eq "Completed") { "done!" }
elseif ($j.State -eq "Running") { "interrupted" }
else { "???" }

Err, actually it just errored out after 120 seconds. You may be able to fix it if @Eden code doesn't work though.

weird, it just opens and closes for me

+ $j = Start-Job -ScriptBlock{
+                            ~
Missing closing '}' in statement block or type definition.
    + CategoryInfo          : ParserError: (:) [], ParentContainsErrorRecordException
    + FullyQualifiedErrorId : MissingEndCurlyBrace

I missed a curly brace:

 $timeoutSeconds = 15
$j = Start-Job -ScriptBlock{
    $page = 'http://mirror.centos.org/centos-6/6.8/os/x86_64/Packages/'
$uri = [System.Uri]($page)
$listing = Invoke-WebRequest -Uri $uri

$uris = ForEach($link in $listing.Links.href) {
    $link_uri = [System.URI]($link)

    If($link_uri.IsAbsoluteUri) {
        $link_uri
    } Else {
        If([System.Uri]::TryCreate($uri,$link,[ref]$link_uri)) {
            $link_uri
        }
    }
}
}
ForEach($item in $uris) {
    If( $uri.Authority -eq $item.Authority ) {
        $file = $item.segments[-1]
        Invoke-WebRequest -Uri $item -OutFile $file
    }
}

Wait-Job $j -Timeout $timeoutSeconds | out-null
if ($j.State -eq "Completed") { "done!" }
elseif ($j.State -eq "Running") { "interrupted" }
else { "???" }

Change the 15 to whatever you want at the top btw. I put 15 for testing.

1 Like

Looks good, not really working right now but that may be due to my piss poor connection at work right now.

Although, for your exit code, it is only up for a second and then the window closes. Is there a possible wait for user input before closing?

I couldn't tell you.

does it start downloading files for you if you try it?