The system completes multi-step forms on sites with bot protection, for example government portals in Russia and other countries. One example is registration on a Spanish government services portal protected by F5 Distributed Cloud Bot Defense.
What we did
Our own headless browser and proxy
Full control over the browser environment and address rotation.
Step-by-step forms
Filling in wizards and submitting forms.
Parallel workers
Parsing and form submission run in separate processes and scale independently.
API
a page screenshot, fetching HTML, queuing a task, and serving the next task from the queue.
Debugging over VNC
You can connect to the browser and see where a scenario stopped.

Stack
Python, Selenium, Scrapy, a custom headless browser build and proxy, VNC.
8 927 874 48 31