Octoparse is an easy-to-use web scraping tool allowing you to build your own crawler, no coding required.
Octoparse can assist by bypassing the majority of anti-scraping technologies. This is done by extending wait times, automatically switching the browser, and more. In some cases, an IP proxy network is still required. For example, many large retail sites have strict restrictions on abnormal access, in this case, it is recommended to rotate the IP address at a certain frequency.
How to integrate Octoparse with Hydraproxy
- Download and install Octoparse. You can do that by accessing the following link https://www.octoparse.com/download/
2. Open Octoparse and create a custom task or select from the predefined task templates.
3. Type in your target domain/s in the website field.
4. Click settings icon on the top left corner to set up IP proxy.
5. Go to Anti-blocking settings.
6. Check the box “Access websites via proxies”
7. Click on configure button. I will automatically check the “Use my own proxies” box.
8. Enter the proxy or the proxy list then click Confirm button. If you need help placing an order please click here: https://hydraproxy.wpengine.com/how-to-order/
9. After you save the settings you can now Run the task.
10. Once the task has finished you can export data in your prefered format.
If you still have any questions you can also check the video tutorial below