Dashboard Order Developer FAQ
Login

 Introduction

 API call structure
 Scrape Google Search
 Scraping Bing Search
 Keyword Analytics
 Keyword rank tracker


 Source Code download

Basic API information

Minimal API link: https://scraping.services/api?user=_USER@ACCOUNT_&credentials=_CREDENTIALS_
API login: _USER@ACCOUNT_
API key: _CREDENTIALS_

Keyword rank tracker

The full functionality seen within our official keyword rank tracking webtool can be accessed through the API.

This ranges from getting projects and their details, creating new projects and modifying them up to controlling recalculation, immediate samples

and retrieving all reporting data.

It is even possible to display all or single reporting charts on your own website, they are responsive and will stay interactive.

 

As usual there are multiple methods to supply the API variables, in these examples we use a "json" object to supply required parameters.

The json object can be passed through GET as well as POST, for simplification we use the GET method.

 

Simultanous API access is permitted.

 

 

API Tasks

get_trackers: 

https://scraping.services/api?user=_USER@ACCOUNT_&credentials=_CREDENTIALS_&job_type=website_tracker&type=json&task=get_trackers

All existing tracker jobs within the account will be returned as a json object.

To get detailed information the API task 'get_tracker' is available.

 

Response example:

{
"exceptions": [],
"answer": {
"tracker_jobs": {
"Demo-job": "Demo-job",
"Demonstration": "Demonstration",
"dailytest": "dailytest",
}
}
}

get_tracker:

https://scraping.services/api?user=_USER@ACCOUNT_&credentials=_CREDENTIALS_&job_type=website_tracker&type=json&task=get_tracker&json={%22jobname%22:%22Demo-job%22}

Example response of an unconfigured job:

{
"exceptions": [],
"answer": {
"tracker_job": {
"jobname": "Demo-job",
"job_type": "website_tracker",
"language": "English",
"country": "United States",
"keywords": [],
"sample_rate": "hold",
"sample_depth": "1000",
"website_competition_hosts": [],
"website_alias_hosts": [],
"active": "0",
"samples_total": "0",
"samples_calculated": "0",
"current_status": "INCOMPLETE"
}
}
}

Example response of a fully configured job:

{
    "exceptions": [],
    "answer": {
        "tracker_job": {
            "jobname": "Demonstration",
            "job_type": "website_tracker",
            "language": "English",
            "country": "United States",
            "keywords": [
                "seo",
                "search engine optimization",
                "buy seo"
            ],
            "sample_rate": "daily",
            "sample_depth": "500",
            "website_competition_hosts": [
                "*moz.com*"
            ],
            "website_alias_hosts": [
                "https:\/\/www.reddit.com\/r\/SEO*",
                "https:\/\/en.wikipedia.org\/wiki\/Search_engine_optimization"
            ],
            "active": "1",
            "samples_total": "27",
            "samples_calculated": "27",
            "current_status": "ACTIVE"
        }
    }
}
  • jobname
    the unique name of the tracker project/job
  • language
    the language for this job, all results from Google/Bing will be scraped for users with this language
  • country
    the country for this job, all results from Google/Bing will be scraped for users within this country
  • keywords
    a json array with all configured keywords
  • sample_rate
    the sample_rate, this is the interval within new samples for all keywords will be scraped.
    Available intervals:
    • hold
    • hourly
    • daily
    • weekly
    • monthly
  • sample_depth
    the sample_depth is a number from 100 to 1000 and defines how many results will be scraped for each keyword.
    the available numbers are available in full 100 steps (100,200,300...1000)
  • website_competition_hosts
    an optional json array with competition hosts/websites
    hosts are usually configured with wildcards (*) to match more easily.
    Example: 'https://*website.com*' this will search all keywords with https URLs that contain the string 'website.com' 
  • website_alias_hosts
    a json array with your own host(s)/website(s)
    hosts are usually configured with wildcards (*) to match more easily.
    Example: 'https://*website.com*' this will search all keywords with https URLs that contain the string 'website.com' 
  • active
    0 or 1, shows if a project is completely configured (keywords, hosts, sample_rate)
  • samples_total
    the total number of samples that have been scraped.
    a job with a sampling rate of 'hourly' will provide 24 samples a day.
  • samples_calculated
    the number of samples that have been calculated, this setting usually matches the samples_total counter.
    the setting will temporarily show a lower number after using the 'tracker_recalculate' api call until all samples have been recalculated.
  • current_status
    • INCOMPLETE - the job is not completely configured for use
    • HOLD - the job is configured and automated scraping is stopped
    • ACTIVE - the job is configured and automated scraping (sample_rate) is configured other than 'hold'
    • PENDING - the job is about to start a scraping run
    • SCRAPING - the job is actively scraping, this will take from 1 up to 30 minutes

 

remove_tracker:

https://scraping.services/api?user=_USER@ACCOUNT_&credentials=_CREDENTIALS_&job_type=website_tracker&type=json&task=remove_tracker&json={%22jobname%22:%22Demo-job%22}

The selected project will be deleted, all collected data will automatically be purged within 24 hours.

Example response:

{
"exceptions": [],
"answer": [
true
]
}

 

create_tracker:

https://scraping.services/api?user=_USER@ACCOUNT_&credentials=_CREDENTIALS_&job_type=website_tracker&type=json&task=create_tracker&json={%22jobname%22:%22Demo-job%22}

A new project is created using the specified 'jobname'. 

It is recommended to add a 'locale' at the end of jobnames, for example 'Website-enUS'.

 

Example response:

{
"exceptions": [],
"answer": {
"tracker_job": {
"jobname": "Demo-job",
"job_type": "website_tracker"
}
}
}

 

edit_tracker:

https://scraping.services/api?user=_USER@ACCOUNT_&credentials=_CREDENTIALS_&job_type=website_tracker&type=json&task=edit_tracker&json={"jobname":"Demo-job","language":"English","country":"United%20States","keywords":["keyword%201","keyword%202","keyword%203"],"sample_rate":"hourly","sample_depth":"100","website_competition_hosts":["*competition.com/path*"],"website_alias_hosts":["*my.host.name*","http://www.website.com/shop"]}}

 

Example json object:

{
"jobname": "Demo-job",
"language": "English",
"country": "United%20States",
"keywords": ["keyword%201", "keyword%202", "keyword%203"],
"sample_rate": "hourly",
"sample_depth": "100",
"website_competition_hosts": ["*competition.com/path*"],
"website_alias_hosts": ["*my.host.name*", "http://www.website.com/shop"]
}

Example response:

{
    "exceptions": [],
    "answer": {
        "tracker_job": {
            "jobname": "Demo-job",
            "job_type": "website_tracker"
        }
    }
}

 

json variables:

  • jobname
    A unique jobname, it's recommended to add a 'locale' at the end to support multiple language configurations for the same project.
    Example: 'website-enUS','website-ptBR','website-deDE'
  • language
    the language for this job, all results from Google/Bing will be scraped for users with this language
  • country
    the country for this job, all results from Google/Bing will be scraped for users within this country
  • keywords
    A json array of keywords
    • hold - this setting is default and will stop any automated scraping activity, no automated samples will be taken.
    • hourly
    • daily
    • weekly
    • monthly
  • sample_rate
    The selected sample rate.
    Any sample_rate that differs from 'hold' will immediately start scraping a job once it is configured completely.
  • sample_depth
    the sample_depth is a number from 100 to 1000 and defines how many results will be scraped for each keyword.
    the available numbers are available in full 100 steps (100,200,300...1000)
  • website_competition_hosts [optional]
    an optional json array with competition hosts/websites
    hosts are usually configured with wildcards (*) to match more easily.
    Example: 'https://*website.com*' this will search all keywords with https URLs that contain the string 'website.com' 
  • website_alias_hosts
    a json array with your own host(s)/website(s)
    hosts are usually configured with wildcards (*) to match more easily.
    Example: 'https://*website.com*' this will search all keywords with https URLs that contain the string 'website.com' 

Available countries and languages:

 

 

tracker_immediate_sample:

https://scraping.services/api?user=_USER@ACCOUNT_&credentials=_CREDENTIALS_&job_type=website_tracker&type=json&task=tracker_immediate_sample&json={%22jobname%22:%22Demo-job%22}

This API call requests an immediate sample to be taken. It is possible to rake a fresh sample once every full hour.

It is not required to use this call after creating a new project, a new project will always cause an immediate sample.

 

 

tracker_recalculate:

https://scraping.services/api?user=_USER@ACCOUNT_&credentials=_CREDENTIALS_&job_type=website_tracker&type=json&task=tracker_recalculate&json={%22jobname%22:%22Demo-job%22,%22timespan%22:%22Day%22,%22timespan_position%22:%220%22}

This API call will recalculate all samples within the selected timespan.

A recalculation is required if website_alias_hosts or website_competition_hosts was modified to affect the changed in reports.

A recalculation is not required if keywords were changed or if historic reporting data is not relevant anymore.

The process of calculating rankings for all samples, keywords and hosts can take a few minutes, depending on the amount of samples available.

Recalculations cost 0.1 Credits per sample.

 

json variables:

  • jobname
  • timespan
    The 'timespan' specifies which samples will be recalculated,
    • Day - last 2 days
    • Week - last 1 week
    • Month - last 1 month
    • 6-month - last 6 months
    • Year - last 1 year
    • All - all samples
  • timespan_position [default 0]
    A timespan_position of '1' would shift the timespan into the past.
    For example: timespan_position of 0 and timespan 'Day' will affect all samples within the past 2 days.
                        timespan_position of 1 and timespan 'Day' will affect all samples younger than 4 days and older than 2 days.
                        timespan_position of 2 and timespan 'Day' will affect all samples younger than 6 days and older than 4 days.


 

get_tracker_report:

https://scraping.services/api?user=_USER@ACCOUNT_&credentials=_CREDENTIALS_&job_type=website_tracker&type=json&task=get_tracker_report&json={%22jobname%22:%22Demonstration%22,%22report%22:%22Keyword%20rankings%22,%22timespan%22:%22Week%22,%22timespan_position%22:%220%22,%22keywords%22:[%22search%20engine%20optimization%22],%22website_alias_hosts%22:[%22https://www.reddit.com/r/SEO*%22,%22https://en.wikipedia.org/wiki/Search_engine_optimization%22],%22single_chart%22:%22keyword_ranking%22,%22search_engine%22:%22google%22}

This API call is used to retrieve reporting data, it allows to either get the raw data of any available report or the responsive charts as html/js code.
It is possible to select charts by name as well as by search engine. Charts can be embedded into your own website.

The raw data can be used to generate all tables and charts visible on the website

 

Return types:

  • 'html' and 'html-embedded'

    When using 'html' or 'html-embedded' the API will return graphical reports without data.

    HTML code as well as required JavaScript code will be embedded.

    You only need to have ChartJS included to display the original and fully functional reports.

    <script src="https://cdnjs.cloudflare.com/ajax/libs/Chart.js/2.6.0/Chart.bundle.min.js" type="text/javascript"></script>

    When using 'html-embedded' you need to place the report within a <div> tag, the size of the <div> tag will define the chart size.

    Charts are interactive and responsive, they will automatically expand or shrink in size. 

  • 'json'
    the json return type will return a large json object including all reporting raw data as well as html and javascript codes for visual charts.
    The data included can be used to completely build reports and tables similar as on our Dashboard.

 

 

Example request json object:
{
"jobname": "Demonstration",
"report": "Keyword rankings",
"timespan": "Week",
"timespan_position": "0",
"keywords": ["search engine optimization"],
"website_alias_hosts": ["https://www.reddit.com/r/SEO*", "https://en.wikipedia.org/wiki/Search_engine_optimization"],
"single_chart": "keyword_ranking",
"search_engine": "google"
}

 

Json variables:

  • jobname
  • report
    The selected report, it is recommended to take a look at our webtool to see possible information gained from each report.
    • 'Full keyword analysis'
      Data: Trending data, report data (all keywords)
      Charts: none
      Require: website_alias_hosts [1 - 20]
    • 'General overview'
      Data: Trending data, report data
      Charts: 'ranking_overview', 'num_keywords_ranked', 'average_ranking'
      Require: website_alias_hosts [1 - 20]
    • 'Keyword rankings'
      Data: Trending data, report data
      Charts: 'keyword_ranking'
      Require: website_alias_hosts [1 - 20], keywords [1 - 20]
    • 'Creative analysis'
      Data: Full keyword table, one keyword SEM details
      Charts: none
      Require: keywords [ 1 ]
    • 'Competition analysis'
      Data: report data
      Charts: 'keyword_ranking'
      Require: website_alias_hosts [1 - 20], website_competition_hosts [1 - 20], keywords [ 1 ]
  • timespan
    The 'timespan' specifies which samples will be recalculated,
    • 'Day' - last 2 days
    • 'Week' - last 1 week
    • 'Month' - last 1 month
    • '6-month' - last 6 months
    • 'Year' - last 1 year
    • 'All' - all samples
  • timespan_position [default 0]
    A timespan_position of '1' would shift the timespan into the past.
    For example: timespan_position of 0 and timespan 'Day' will affect all samples within the past 2 days.
                        timespan_position of 1 and timespan 'Day' will affect all samples younger than 4 days and older than 2 days.
                        timespan_position of 2 and timespan 'Day' will affect all samples younger than 6 days and older than 4 days.
  • keywords [optional]
    A json array with keywords, each keyword has to exactly match a configured keyword.
    Some reports require keywords, some permit only one keyword or require none.
  • website_alias_hosts [optional]
    A json array with hosts, each host has to match a configured host exactly.
    Some reports require hosts
  • website_competition_hosts [optional]
    A json array with hosts, each host has to match a configured host exactly.
    Some reports require hosts
  • single_chart [optional]
    By default all available charts are returned, this option allows to select one specific chart
    For available chart names refer to the 'report' variable description above.
  • search_engine [optional]
    set to : 'google' or 'bing' 
    By default all search engine results are returned.

Example request json object:

{
"jobname": "Demonstration",
"report": "Keyword rankings",
"timespan": "Week",
"timespan_position": "0",
"keywords": ["search engine optimization"],
"website_alias_hosts": ["https://www.reddit.com/r/SEO*", "https://en.wikipedia.org/wiki/Search_engine_optimization"],
"single_chart": "keyword_ranking",
"search_engine": "google"
}

 

Example JSON response: