webbrowser eccentricity

[Ultrawhack]

Hi,

I manipulate my webbrowser object to navigate to a login page, & auto login using username, password from a database.

Though I test for documentcompleted, sometimes the page does not seem to load completely and the login click event fails.

'SUBMIT
hElement3 = WebBrowser1.Document.GetElementById("Login")
hElement3.InvokeMember("click")

Does anyone have a workaround for documentcompleted ?

Thanks !

 

[JohnH]

I don't know if you have any experience with the AxWebBrowser from earlier VB version (it's still available to use in .Net 2), with that there was some standard code to determine when page was considered fully loaded, that code was "If e.pDisp Is AxWebBrowser1.Application Then ...". With the new webbrowser some thing seems to be missing, or perhaps our old habits just haven't found a new way around the old problems yet.. The code mentioned is something you can't do with the new WebBrowser controls DocumentCompleted event. So I have done some analyzing of the events to see what happens and if there could be some way to determine on equal basis as with the AxWebBrowser when a document is fully loaded. Remember the browser is usually loading several parts of a webpage that are considered separate 'documents' and this is not only frames. What I have done is counting the events Navigating/Navigated/DocumentCompleted, also calculated separately (+Navigating-DocCompleted=Remainding), and checked the IsBusy property and even DOM documents ReadyState. When each of the three events occured I logged the current state/count of all parameters listed above with timestamp - thinking as a measure you could also include a Timer for short timeout assurance that browser is finished loading. Have a look at the log, there are some interesting info. The webpage that was loading was a common 'any' one. I have marked one line in the log, that was the event that the old AxWB reported finished with the pDisp-Application check, but still that one counted 8 completes total too. That line provides an easy measurement to tell that you may access DOM, just count up Navigating and subtract DocumentCompleted on the same counter. When counter reaches 0 in DocumentCompleted event you can go. Now here is the full log:

07.07.2006 03:04:47 Navigating=1 Navigated=0 Completed=0 Remainding=1 IsBusy=False ReadyState=Uninitialized
07.07.2006 03:04:47 Navigating=2 Navigated=0 Completed=0 Remainding=2 IsBusy=True ReadyState=Loading
07.07.2006 03:04:47 Navigating=2 Navigated=1 Completed=0 Remainding=2 IsBusy=True ReadyState=Loading
07.07.2006 03:04:47 Navigating=2 Navigated=2 Completed=0 Remainding=2 IsBusy=True ReadyState=Interactive
07.07.2006 03:04:47 Navigating=2 Navigated=2 Completed=1 Remainding=1 IsBusy=False ReadyState=Interactive
[COLOR=darkgreen][B]07.07.2006 03:04:47 Navigating=2 Navigated=2 Completed=2 [COLOR=red]Remainding=0[/COLOR] IsBusy=False ReadyState=Complete[/B][/COLOR]
07.07.2006 03:04:47 Navigating=3 Navigated=2 Completed=2 Remainding=1 IsBusy=False ReadyState=Complete
07.07.2006 03:04:47 Navigating=3 Navigated=3 Completed=2 Remainding=1 IsBusy=True ReadyState=Complete
07.07.2006 03:04:48 Navigating=4 Navigated=3 Completed=2 Remainding=2 IsBusy=True ReadyState=Complete
07.07.2006 03:04:48 Navigating=5 Navigated=3 Completed=2 Remainding=3 IsBusy=True ReadyState=Complete
07.07.2006 03:04:48 Navigating=6 Navigated=3 Completed=2 Remainding=4 IsBusy=True ReadyState=Complete
07.07.2006 03:04:48 Navigating=7 Navigated=3 Completed=2 Remainding=5 IsBusy=True ReadyState=Complete
07.07.2006 03:04:48 Navigating=8 Navigated=3 Completed=2 Remainding=6 IsBusy=True ReadyState=Complete
07.07.2006 03:04:49 Navigating=8 Navigated=4 Completed=2 Remainding=6 IsBusy=True ReadyState=Complete
07.07.2006 03:04:49 Navigating=8 Navigated=5 Completed=2 Remainding=6 IsBusy=True ReadyState=Complete
07.07.2006 03:04:49 Navigating=8 Navigated=6 Completed=2 Remainding=6 IsBusy=True ReadyState=Complete
07.07.2006 03:04:49 Navigating=8 Navigated=7 Completed=2 Remainding=6 IsBusy=True ReadyState=Complete
07.07.2006 03:04:49 Navigating=8 Navigated=8 Completed=2 Remainding=6 IsBusy=True ReadyState=Complete
07.07.2006 03:04:49 Navigating=8 Navigated=8 Completed=3 Remainding=5 IsBusy=False ReadyState=Complete
07.07.2006 03:04:49 Navigating=8 Navigated=8 Completed=4 Remainding=4 IsBusy=False ReadyState=Complete
07.07.2006 03:04:49 Navigating=8 Navigated=8 Completed=5 Remainding=3 IsBusy=False ReadyState=Complete
07.07.2006 03:04:49 Navigating=8 Navigated=8 Completed=6 Remainding=2 IsBusy=False ReadyState=Complete
07.07.2006 03:04:49 Navigating=8 Navigated=8 Completed=7 Remainding=1 IsBusy=False ReadyState=Complete
07.07.2006 03:04:49 Navigating=8 Navigated=8 Completed=8 Remainding=0 IsBusy=False ReadyState=Complete

If you need to be 'picky' I see no other way than to do the counting plus add a timer that checks timeout after DocumentCompleted, say for instance 1 second, if there still should be pending 'remainders' or similar flags. Also note that sometimes the events Navigating/Navigated/DocumentCompleted don't add up properly, I've seen this when testing some webpages, and once in a while when loading same page several times in succession there seem to be a glitch in occurance of events. So beware

::Edit:: I think now that what the old Ax browser provided for complete-functionality was similar to simply check ReadyState. You see that still is the same line highlighted. So you could go with WebBrowser1.ReadyState=Complete in DocumentCompleted event

 

[Ultrawhack]

Thanks for the exhaustive explanation.

In VB2005Express I placed this code in documentcompleted event
If WebBrowser1.ReadyState = WebBrowserReadyState.Complete Then
bBusy = False
End If

I test for bbusy before login automation.

While (bBusy)
Application.DoEvents()
End While

When I clear browser cache - it fails.

After I login manually - thereafter it works every time.

 

[JohnH]

You said first that code makes the form take long to appear, and that is because form don't appear until Load event handler is finished, where you do this while loop to stop it until ReadyState=Complete - get it? Don't do that, have a look at the example below.

Another thing to remember is that there may be more document completes after ReadyState completes (and stays complete). So you use a boolean to be able do your work only once.

About clearing cache that would also in some cases clear the stored user/pass, which it seems to do here. You can check the source of that login page and find the elements you have to fill in by Id/name and do that through DOM code too. I've included two code line as examples of this.

Dim bDone As Boolean = False
 
Private Sub Form1_Load(ByVal sender As System.Object, ByVal e As System.EventArgs) _
Handles MyBase.Load
  WebBrowser1.Navigate("www.somepage.com")
End Sub
 
Private Sub WebBrowser1_DocumentCompleted(ByVal sender As System.Object, _
ByVal e As System.Windows.Forms.WebBrowserDocumentCompletedEventArgs) _
Handles WebBrowser1.DocumentCompleted
  If WebBrowser1.ReadyState = WebBrowserReadyState.Complete And bDone = False Then
    bDone = True
    DoOnceAfterDocComplete()
  End If
End Sub
 
Sub DoOnceAfterDocComplete()
  WebBrowser1.Document.GetElementById("User").InnerText = "User"
  WebBrowser1.Document.All.GetElementsByName("Email")(0).InnerText = "email"
  WebBrowser1.Document.GetElementById("Login").InvokeMember("click")
End Sub

Set bDone back to initial False if you need to load another page and also want this action to happen again (once on load).

 

[Ultrawhack]

Thanks JohnH - that works flawlessly. After initial autologin I need to move to another page.

1. Using your code, where do I insert webbrowser1.navigate("www.mynextpage.com")
to take the browser to the next page after login ??

2. When form loads, user can see the login page being automated. How can I hide the webbrowser as it is logging in and display a message "Logging in..." until the webbrowser is mynextpage ?

 

[JohnH]

I would place a Picturebox over the webbrowser and display and nice picture with "loading.." message. Then remove Picturebox after second page is loaded. Here's modified code, see the logic?

Dim bDone As Boolean = False
Dim bDoToo As Boolean = False
 
Private Sub Form1_Load(ByVal sender As System.Object, ByVal e As System.EventArgs) _
Handles MyBase.Load
  WebBrowser1.Navigate([URL="http://www.somepage.com/login.htm"]www.somepage.com/login.htm[/URL])
End Sub
 
Private Sub WebBrowser1_DocumentCompleted(ByVal sender As System.Object, _
ByVal e As System.Windows.Forms.WebBrowserDocumentCompletedEventArgs) _
Handles WebBrowser1.DocumentCompleted
  If WebBrowser1.ReadyState = WebBrowserReadyState.Complete And bDone = False Then
    bDone = True 'close first completed automation
    DoOnceAfterDocComplete()
  End If
  If WebBrowser1.ReadyState = WebBrowserReadyState.Complete And bDoToo = True Then
    bDoToo = False 'close second automation again
    DoOnceAfterDocComplete2()
  End If
End Sub
 
Sub DoOnceAfterDocComplete()
  WebBrowser1.Document.GetElementById("User").InnerText = "User"
  WebBrowser1.Document.All.GetElementsByName("Email")(0).InnerText = "email"
  WebBrowser1.Document.GetElementById("Login").InvokeMember("click")
  WebBrowser1.Navigate("www.somepage.com/afterlogin.htm")
  bDoToo = True 'open for second completed automation 
End Sub
 
Sub DoOnceAfterDocComplete2()
  Me.Controls.Remove(PictureBox1)
End Sub

I don't know if you have to wait for a intermediate page when you click login button before you can navigate to afterlogin.htm page. I'm sure you can figure this out.

 

[Ultrawhack]

Thanks. For me, once I use the above code the loginpage fails and it does not move to afterlogin.htm but picturebox1 gets removed.

Your original code works flawlessly. I suspect the webbrowser is not waiting to make sure loginpage is fully loaded before firing
WebBrowser1.Navigate(www.somepage.com/afterlogin.htm)

Any other ideas?

 

[JohnH]

Usually when you fill login info and click login button you are brought to new page or page simply is refreshed. So use the logic from the to examples and make it three completes instead. First complete you do automation and click login. Then you wait for second complete which is the "after-login", when that happens you may willingly navigate the browser on to 'second stage.

 

[Ultrawhack]

I think I've tried it a 100 different ways. It works once and then I clear cache & cookies to test it again and it fails.

 

[JohnH]

Like I said, they store user/pass in cookie, when you clear cache this info is deleted. You have to fill in the user/pass fields with automation as in code example (by getting input field by id or name and setting the innertext value). And when you click login button after you have put user/pass info there, you have to wait for page to postback or navigate to other page for the user/pass to again be stored in cookie, only after that you may automate the browser to navigate to other page. That makes three passes on the documentcompleted event to handle. It only takes three booleans to control that behaviour, as examplified in first and second codes.

 

[JohnH]

I'm afraid that is illegal, refer to the terms of service http://docs.yahoo.com/info/terms/ (esp. 6i and 17)
All public services like this disallow these kinds of actions.
Neither me or any other forum user is allowed to help you with this.
Nor am I interested in creating an Yahoo account, or clearing my internet cache.

I will though, reveal that what you ask for (in similar context) basically is very easy, and all the answers you seek are already given in previous posts. If you decide to personally pursue your goals, I suggest reading through the posts and code again and sit down for a few days to think it over. It is very important that you understand the funtionality of the Boolean data type, it may have either one of two values - True or False - and you are in total control of it at all times. As I have been trying to tell you several times you only need one Boolean for each pass/step in documentcompleted, try to visualize these three boolean values as gates. You need to enter each gate in a specific order, when you've entered a gate you have to close it again to stop the hordes following.

 

[Ultrawhack]

I did not realise that I was violating any rules. Did not mean to ask anyone on these forums to do that. Most sites have this login format and I just chose a common one at random.

 

[CarcaBot]

you can do it by installing roboform application.
roboform is the top-rated Password Manager and Web Form Filler that completely automates password entering and form filling. Roboform is a push-button web form filler with some serious Artificial Intelligence (AI) inside. It allows you to define your personal profile or "usual" responses to a web form. This information is then saved, and when you need to fill a form, just click "Fill Forms" button, and form is filled out. Works as an add-on to M'zoft Internet Explorer, Mozilla, Firefox or Netscape web browsers.