I have a web application project which require to navigate 100's of webpage and scrape data. Looks like at some point the htmldocument become null and unable to set url for navigate, even though the match rule satisfy.
So i have three webpages to navigate and get data. I have interrogated the first page and supplied further url to webpage with Navigate method. The match rule I provide as webpage host, so I was expecting the webpage should match and proceed. Looks like after 10 or 15th iteration I am getting error that HTMLDocument is null. I do follow the wait for create true path. But still on navigate, it fails.
Another way to navigate a web adapter is to interrogate the web browser control. This may cause other problems, but in my experience, it has been reliable. To interrogate, you'll need to do the following;
Update the IgnoreMainBrowser property on the adapter to False.
Navigate to a web page that you have not interrogated.
Uncheck Create Global Webpage from the Interrogation form.
Interrogate some control on the page.
You should see a new hierarchy in your adapter.
Delete all controls below the WebBrowser control.
Update any match rules for controls in this hierarchy so that they match any page you might navigate to.
Generally, you can just delete any Text Match Rules.
This control has a Navigate method that you can use on any page to Navigate the browser.
That would indicate that you have not made the match rules on the WebBrowser control or its parent controls generic. As Thomas mentioned, make sure that you remove any Text based Match Rules. Navigate by hand to each page and your WebBrowser should remain matched. If not, check the first control in the hierarchy that does not match - that will be the one that needs its Match Rule to be fixed.
If your web page navigates to itself (based on your match rules they are all the same page) you must first wait for the page to be destroyed before waiting for it to be created. When you execute the navigate the page is created and your isCreated test is probably taking place before it is destroyed. Your automation should look like this (add appropriate error handling for the fail situations).