Dashboard Order Programming FAQ
Login

 What is scraping ?
 How to export CSV data ?
 What keywords work best for scraping?
 Why to use a scraping service?
 What does a "!" during scraping mean ?

Frequently Asked Questions





Keywords and Key-phrases

Each of our jobs may contain up to 100000 keywords, however it is recommended to use multiple smaller sized jobs instead.
A keyword may contain up to 250 characters, it's strongly recommended to use shorter keyword strings than this.

Especially when working with Google the keyword length and keyword complexity can cause issues during scraping.

Google prevents scraping with a self-learning system. The more "organic" your keywords look the more likely scraping will work without issues.
When the detection system is triggered, keywords of similar structure might be rejected, in some cases we have seen temporary world-wide bans.


Especially when using "intitle:", "site:" and similar special search operators keywords can be detected by Google.

 

Bing is often a good alternative, similar search operators can be used with Bing just as with Google.

 

Google search operators: https://support.google.com/websearch/answer/2466433?hl=en
Bing search operators: https://msdn.microsoft.com/en-us/library/ff795662.aspx