Wait until page is loaded with Selenium WebDriver for Python

I want to scrape all the data of a page implemented by a infinite scroll. The following python code works. for i in range(100): driver.execute_script(“window.scrollTo(0, document.body.scrollHeight);”) time.sleep(5) This means every time I scroll down to the bottom, I need to wait 5 seconds, which is generally enough for the page to finish loading the newly … Read more

Wait for page load in Selenium

How do you make Selenium 2.0 wait for the page to load? 48 Answers 48 You can also check pageloaded using following code IWait<IWebDriver> wait = new OpenQA.Selenium.Support.UI.WebDriverWait(driver, TimeSpan.FromSeconds(30.00)); wait.Until(driver1 => ((IJavaScriptExecutor)driver).ExecuteScript(“return document.readyState”).Equals(“complete”));

How to select a drop-down menu value with Selenium using Python?

I need to select an element from a drop-down menu. For example: <select id=”fruits01″ class=”select” name=”fruits”> <option value=”0″>Choose your fruits:</option> <option value=”1″>Banana</option> <option value=”2″>Mango</option> </select> 1) First I have to click on it. I do this: inputElementFruits = driver.find_element_by_xpath(“//select[id=’fruits’]”).click() 2) After that I have to select the good element, lets say Mango. I tried to … Read more

Error message: “‘chromedriver’ executable needs to be available in the path”

I am using selenium with python and have downloaded the chromedriver for my windows computer from this site: http://chromedriver.storage.googleapis.com/index.html?path=2.15/ After downloading the zip file, I unpacked the zip file to my downloads folder. Then I put the path to the executable binary (C:\Users\michael\Downloads\chromedriver_win32) into the Environment Variable “Path”. However, when I run the following code: … Read more

How do I find an element that contains specific text in Selenium WebDriver (Python)?

I’m trying to test a complicated JavaScript interface with Selenium (using the Python interface, and across multiple browsers). I have a number of buttons of the form: <div>My Button</div> I’d like to be able to search for buttons based on “My Button” (or non-case-sensitive, partial matches such as “my button” or “button”). I’m finding this … Read more

Is there a way to get element by XPath using JavaScript in Selenium WebDriver?

I am looking for something like: getElementByXpath(//html[1]/body[1]/div[1]).innerHTML I need to get the innerHTML of elements using JS (to use that in Selenium WebDriver/Java, since WebDriver can’t find it itself), but how? I could use ID attribute, but not all elements have ID attribute. [FIXED] I am using jsoup to get it done in Java. That … Read more

Headless Browser and scraping – solutions [closed]

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers. Want to improve this question? Update the question so it’s on-topic for Stack Overflow. Closed 7 years ago. Improve this question I’m trying to put list of possible solutions for browser automatic tests suits and headless browser platforms capable of … Read more

Debugging “Element is not clickable at point” error

I see this only in Chrome. The full error message reads: “org.openqa.selenium.WebDriverException: Element is not clickable at point (411, 675). Other element would receive the click: …” The element that ‘would receive the click’ is to the side of the element in question, not on top of it and not overlapping it, not moving around … Read more

Can a website detect when you are using Selenium with chromedriver?

I’ve been testing out Selenium with Chromedriver and I noticed that some pages can detect that you’re using Selenium even though there’s no automation at all. Even when I’m just browsing manually just using Chrome through Selenium and Xephyr I often get a page saying that suspicious activity was detected. I’ve checked my user agent, … Read more