copy file + progress bar

[rlangi05]

Hi guys... I want to copy a file from one folder to another but I want a progress bar to show the progress of the transfer. Im using VB .NET 2003. Can anyone help me with a link or sample code.

Thanks.

 

[JohnH]

I declared the filestreams outside of the sub

Outside the sub? Why do you need them there?

in the RunWorkerCompleted function, I compare size of the old and new files and made sure the progressbar received a value of 100 upon success

That is also not necessary, you already have the information if operation was cancelled or errored, if neither then operation logically must be complete.

Do you think it is the monitoring of the byte progress that makes the action slower?

Not much, but it would preserve system resources better. Let's say as illustration you copy a 5mb file and the operation takes 5 seconds, which amounts to 1000kb per second. With progress every 1kb that makes 5000 percentage calculations and reports, 500 per second, each percentage would be posted 50 times. It should go without saying this is lots of redundant information for user, and lots of operations the program has no need to process. If you store previous calculated percentage you could post only each time percentage changes, which would be exactly 100 times. Still that would mean lots of calculations and 20 progress reports per second, which in my opinion is insignificant for user. If progress was updated once per second only 5 reports would be made here (20%,40%,60% etc). And if we imagine a file 10 times that size then progress would be posted for 2%,4%,6% each second. When updating the ProgressBar with a larger value span for example from 20% to 40% it animates the change, and it would appear as a dynamic change to the user. The Progressbar also has animation when value doesn't change, at least in Vista, so user can see something is happening even if should take longer time between changes.

As I explained earlier, what would make this operation very slow is to read and write one by one byte to the disk. The IO controller operates much more efficiently when grabbing a bunch of bytes in one go. Windows XP local file copy for example does this in 64kb chunks. Vista was greatly improved in this regard and does files smaller than 1mb in one go, files up to 2mb in two chunks (1mb buffer), and larger files in 2mb chunks.

 

[BGM]

Outside the sub? Why do you need them there?

Because if I declare them inside the sub, Intellisense tells me, via little green squiggly lines in sr and sw in the Finally clause, "variable sr has been used before it has assigned a value. A null reference exception could result at runtime." If I was more sure of my programming skills, I might think I could ignore the warnings given by intellisense.

If operation was cancelled or errored, if neither then operation logically must be complete.

Then it is safe to assume the integrity of the copied file?

To copy more than one byte at a time, after studying a bit (ha ha), I think I could use write instead of writebyte, perhaps something like this below? 65536 is 64kb, I think. I read that if you are copying to a network location, it is not recommended to use greater than that. I might try this read/write part, but what I have so far isn't right.

Dim len As Long = sr.Length - 1 'counts from zero...
Dim byteData(len) As byte
sr.Read(byteData, 65536, len)
sw.Write(byteData,65536,len)

 

[JohnH]

Because if I declare them inside the sub, Intellisense tells me, via little green squiggly lines in sr and sw in the Finally clause, "variable sr has been used before it has assigned a value. A null reference exception could result at runtime." If I was more sure of my programming skills, I might think I could ignore the warnings given by intellisense.

You can assign Nothing to those variables initially to prevent that warning. Also remember later that these variables might actually be null references if no object was assigned to them.

Then it is safe to assume the integrity of the copied file?

Absolutely, if there was a problem an exception would be thrown.

Dim byteData(len) As byte

Declare the array the size of your decided buffer size, not the size of the file. Remember that space is allocated for an array whether it is used or not.

I might try this read/write part, but what I have so far isn't right.

FileStream.Read Method
offset parameter: where to start putting data in the array, if you are going to fill this array you must start at index 0.
Also notice the return value, this is vital information that you must use when you are writing the read data to the target.

 

[BGM]

Hey! I think we got it! This seems to copy the file okay, and it is MUCH faster!

If i = CopyBufferSize or len-i < CopyBufferSize then

Does this look right? I want to copy if i matches the BufferSize, or, if the chunk is smaller than the BufferSize, copy it anyway?

    Private Sub FileCopyWorker_DoWork(ByVal sender As Object, ByVal e As System.ComponentModel.DoWorkEventArgs) Handles FileCopyWorker.DoWork
    whattempdestinationfile = GetDirFromPath(whatdestinationfile) & "temp"
        dim fi As Fileinfo = nothing
        dim sr As FileStream = nothing
        dim sw As filestream = nothing
        Const CopyBufferSizeAs Integer = 64 * 1024
        Try
            MessengerLogger.Info("Please wait whilst I upload " & whatoriginationfile)
            fi = New IO.FileInfo(whattempdestinationfile)
            sr = New IO.FileStream(whatoriginationfile, IO.FileMode.Open) 'source file
            sw = New IO.FileStream(whattempdestinationfile, IO.FileMode.Create) 'target file, defaults overwrite

            Dim len As Long = sr.Length - 1 'starts at 0, we need to offset
            Dim byteData(CopyBufferSize) As byte
            Dim i As long
            Dim bytesRead As long
        
            For i = 0 To len

                If FileCopyWorker.CancellationPending Then
                    FileStatus = false
                    MessengerLogger.Warn("You cancelled the transfer at: " & progress & "%.")
                    Exit For
                End If
                
                If i = CopyBufferSize or len-i < CopyBufferSize then
                    bytesRead = sr.Read(byteData, 0, CopyBufferSize)
                    'sr.Read(byteData, 0, len)
                    sw.Write(byteData, 0, bytesRead)
                End if

                 If i = len then FileStatus = true

                'This if clause is designed only to monitor for the progress bar progress
                If i Mod 100000 = 0 Then 'only update UI every 100 Kb copied
                    progress = i * 100 / len
                    FileCopyWorker.ReportProgress(progress)
                    Application.DoEvents()
                End If
            Next

        Catch ex As Exception
            MessengerLogger.Error("There was an error in the procedure: " & ex.Message & " :CopyFileWithProgress.")
            FileStatus = false
        Finally
            If Not (sr is Nothing) then sr.close
            If Not (sw is Nothing) then sw.close
        End Try
    End Sub

 

[JohnH]

You don't need a For loop, only a Do loop that you exit when Read method returns 0 (meaning end of stream is reached).
Progress can be calculated from source stream Position/Length (*100 for percentage).

 

[BGM]

Mr. John - Okay, how does this look? I tried it on a 37mb file and it was so fast the progressbar didn't even have time to work. Is that right? I checked the copied file, and the file size is correct.

    Private Sub FileCopyWorker_DoWork(ByVal sender As Object, ByVal e As System.ComponentModel.DoWorkEventArgs) Handles FileCopyWorker.DoWork
    whattempdestinationfile = GetDirFromPath(whatdestinationfile) & "temp"
        dim fi As Fileinfo = nothing
        dim sr As FileStream = nothing
        dim sw As filestream = nothing
        Const CopyBufferSize As Integer = 64 * 1024
        Try
            MessengerLogger.Info("Please wait whilst I upload " & whatoriginationfile)
            fi = New IO.FileInfo(whattempdestinationfile)
            sr = New IO.FileStream(whatoriginationfile, IO.FileMode.Open) 'source file
            sw = New IO.FileStream(whattempdestinationfile, IO.FileMode.Create) 'target file, defaults overwrite

            Dim len As Long = sr.Length - 1 'starts at 0, we need to offset
            Dim byteData(CopyBufferSize) As byte
            Dim bytesRead As long
        
            Do
                If FileCopyWorker.CancellationPending Then
                    FileStatus = false
                    MessengerLogger.Warn("You cancelled the transfer at: " & progress & "%.")
                    Exit Do
                End If
                
                bytesRead = sr.Read(byteData, 0, CopyBufferSize)
                sw.Write(byteData, 0, bytesRead)

                If sw.Position Mod CopyBufferSize= 0 Then
                    progress = sw.Position / len * 100
                    FileCopyWorker.ReportProgress(progress)
                    Application.DoEvents()
                End If

                If bytesRead = 0 then 
                    FileStatus = true
                    Exit Do
                End If
            Loop

        Catch ex As Exception
            MessengerLogger.Error("There was an error in the procedure: " & ex.Message & " :CopyFileWithProgress.")
            FileStatus = false
        Finally
            If Not(sr is Nothing) then sr.close
            If Not(sw is Nothing) then sw.close
        End Try
    End Sub

 

[JohnH]

tried it on a 37mb file and it was so fast the progressbar didn't even have time to work.

Local file copy may very well be that fast, just a few comments:

dim fi As Fileinfo = nothing
fi = New IO.FileInfo(whattempdestinationfile)

Never used.

Dim len As Long = sr.Length - 1
progress = sw.Position / len * 100

Just one byte off. Will only be a problem if file size is 1 byte. DivideByZeroException.

Dim byteData(CopyBufferSize) As byte

Here you declare the array one byte larger than you intend to and ever use. Not a big deal, but it looks like you don't know arrays or VB syntax too well.

If sw.Position Mod CopyBufferSize= 0 Then

Since you read CopyBufferSize each time this will always be true, so you are in effect reporting progress each iteration. 37gb / 64kb = 578 or so, meaning you report each percentage number 5-6 times in this case.

Using a FileStatus boolean variable give no additional value in my opinion. What you should do with CancellationPending is to set e.Cancel and exit, from there you have all information you need in RunWorkerCompleted event to report completion status.

Apart from that, good progressing!

Why is much of the code casing wrong by the way?

 

[BGM]

Thanks again!

I removed the extra lines that you pointed out. I actually caught a few already since I made the post.

Should I do this, then?:
Dim byteData(CopyBufferSize -1 ) As byte
If I do, then it fixes this, right?:
If sw.Position Mod CopyBufferSize= 0 Then...

progress = sw.Position / len * 100
Actually, this is used - it provides the progress value for the progressbar.

Since you read CopyBufferSize each time this will always be true, so you are in effect reporting progress each iteration. 37gb / 64kb = 578 or so, meaning you report each percentage number 5-6 times.

Yes, I know that, actually :smile:; I tried a few other things in that spot, but that seemed the best,as most of the files I will be dealing with are 16mb+, but I'd like to think of something better.

Using a FileStatus boolean variable give no additional value in my opinion. What you should do with CancellationPending is to set e.Cancel and exit, from there you have all information you need in RunWorkerCompleted event to report completion status.

Well, in my RunWorkerCompleted function, I need to know whether the copy-process failed or not. If the user cancelled the operation, I want the progressbar to stop at the percentage where it stopped. If the cancel operation returns false, then the RunWorkerCompleted will provide the right notifications for the user.

    Private Sub FileCopyWorker_RunWorkerCompleted(ByVal sender As Object, ByVal e As System.ComponentModel.RunWorkerCompletedEventArgs) Handles FileCopyWorker.RunWorkerCompleted
        If filestatus then
            'MessengerLogger.Info("Almost there...")
            File.Move(whattempdestinationfile, whatdestinationfile)
            MessengerLogger.Info(msg_finished)
            whatprogressbar.Value = 100
            'MakeThreadedNotifications() 'that is, send an email to notify someone
        Else
            MessengerLogger.Warn(msg_toobad)
            File.Delete(whattempdestinationfile)
        End If
        whatstartbutton.Enabled = True
        whatcancelbutton.Enabled = False
    End Sub

...it looks like you don't know arrays or VB syntax too well.

You are quite right. I am used to coding in BCX, powerpro, and autohotkey. This is my first full-fledged vb.net application, although I've hacked through a bit of asp.net. Aside from that, it is one of my common mistakes to overrun arrays... One language starts them a 1, another starts at 0, and, in order to keep track I have to be in a routine. But I like .net - it is so powerful. I think I am getting a little better, however (with thanks to certain people...)

Why is much of the code casing wrong by the way?

A couple of reasons, actually, first, I find the casing to be an extra thing to type and keep track of, so I often use all lower case letters. Then I wish I had them camelcased because it looks nicer and is easier to read. Then I can't remember which letters were camelcased. Besides that, VS doesn't seem to care, not like Javascript. BCX is case-sensitive, so I use mostly all lowercase. But although VS doesn't seem to care much, it "corrects" my typing most, but not all of the time, which results in the inconsistency.

 

[JohnH]

Well, in my RunWorkerCompleted function, I need to know whether the copy-process failed or not.

Which information is known and available without that variable.

progress = sw.Position / len * 100
Actually, this is used - it provides the progress value for the progressbar.

I was not commenting that, I was commenting the fact that you have the computer frequently check some logic that will always equate to True in your code.

...it looks like you don't know arrays or VB syntax too well.

fair enough, I was just pointing out the minor mistake

Why is much of the code casing wrong by the way?

I was actually referring to code keywords that VS formats automatically. You were giving the impression of posting unresolved "notepad" code.

 

[BGM]

Which information is known and available without that variable.

How is that? You mentioned using "e" of the worker. I don't get that, however. I admit, I am still struggling with understanding the backgroundworker.

As I am copying a file from a local drive to a network drive, I decided to transfer it with the filename of "temp" so that nobody would bother it whilst I am copying. Problem is that I need to rename the file after the copy is complete. I have been using file.move for this, but that takes nearly as long as copying it. Is there a better way of renaming "\\remoteddrive\temp" to "\\remotedrive\myfile.ext"? If I do the file.move in the run_completed sub, then the user has to wait another long time for this to finish, and it looks like the process is hanging.

 

[JohnH]

You mentioned using "e" of the worker. I don't get that

All events that conform to the design guidelines have two parameters, 'sender' which refers to the class instance that raised the event, and 'e' that is a EventArgs derive and provides additional event information. EventArgs class has no properties, but if you see an event that uses a derived class, for example DoWorkEventArgs, it always has additional properties that is relevant to the event handling.

I have been using file.move for this, but that takes nearly as long as copying it

I don't know why that is slow for you, it should only result in a simple request to target computer to remap its file table. That is what is happening when I do that, the operation only takes a few millisecond to complete.

 

[BGM]

Okay, so I see that if I debug print for e, there are a few properties to access.

Debug.Print(e.Result.ToString)

But if I debug.print e.result, it throws a NullReferenceException. If I don't add the .ToString, then it just prints nothing. Can I set e to something manually, or is it automatic?

 

[JohnH]

I am still struggling with understanding the backgroundworker

I recommend you read the documentation for that class, any part of if you want to know about: BackgroundWorker Class

Can I set e to something manually, or is it automatic?

Read for example the DoWork event topic.