Tools LinkedIn Pricing API FAQ
Login

Google Bing   KW Analytics   Tracker LinkedIn  Email reverse lookup   Profile Billing

Introduction to Bing SERP scraping per API

Our Bing scraping API is based on HTTP(s) POST or GET calls to our API server. All API requests are self contained and do not require to keep a session or cookies.

Each API call costs 0.01 Credits to prevent overusage (about $1 per 1000 calls) in addition to possible additional charges.

Our Google and Bing scraping APIs are very similar, you can use the same code for both types of scraping.

Not a developer?
We offer web based tools to scrape Bing and export results without any API integration requirements.
Scrape bing search by web-tool

API documentation

We use cURL and URL examples during most of the documentation.
We also provide PHP and Python source code examples to help you get started quickly.

Quick start

Steps to scrape keywords and export results

  1. create_job
    Create a job with a distinct name
  2. modify_job
    Populate the job with keywords, configure optional settings
  3. start_job
    Start the job, scraping will begin
  4. get_job
    Poll for finalization, optional
  5. Export
    Export all or all finished results in JSON or CSV format

Base URL
URL
                            https://scraping.services/api?user=_USER@ACCOUNT_&credentials=_CREDENTIALS_

                            # This base URL contains the credentials of your account
API credentials

URL: https://scraping.services/api
User: _USER@ACCOUNT_
Credentials: _CREDENTIALS_

The API request and response

Request

Let's start by taking a look at a simple request: license_info

The URI is "https://scraping.services/api" with a set of essential GET parameters:

  • user: your registered email address
  • credentials: your private api key
  • job_type: The specifies the API module to access, in our example "info"
  • task: The task specifies the function within the API module to call, in our example info->license_info
  • type: The return format. 'html' and 'json' are available, html is a pretty-printed JSON object formated for HTML inclusion
  • human_readable: by defining this parameter the JSON type data will be pretty printed without any HTML entities

Some API calls require more complicated structures to be passed, for example large keyword arrays.
In these cases the additional variables are supplied as JSON object encoded POST parameter.
It is up to the developer when to use the "json" parameter or when to supply the parameters as GET through the URI.

To see the 'json' parameter being used take a look at modify_job or the source code examples.



Response

The API response always contains two json objects: 'exceptions' and 'answer'.

  • exceptions: Empty on success, filled with one or more exceptions in case of an error.
  • answer: The API response object, in this example it contains information about the current active license

The API response can be converted into a native object or array by almost any programming language.

  • PHP: $array = json_decode($response,true)
  • Java: JSONObject jsonObject = new JSONObject(response);
  • JavaScript: var obj = JSON.parse(response);
  • C# and .NET: dynamic obj = JsonConvert.DeserializeObject(response);
  • Python: json.load(response)

License info request
URL
                            https://scraping.services/api?user=_USER@ACCOUNT_&credentials=_CREDENTIALS_&job_type=info&task=license_info&type=json&human_readable
License info request
cURL
                            curl -G https://scraping.services/api \

                            -d user=_USER@ACCOUNT_ \

                            -d credentials=_CREDENTIALS_ \

                            -d job_type=info \

                            -d task=license_info \

                            -d type=json \

                            -d human_readable
Response
JSON
                            {
  "exceptions": [],
  "answer":
  {
   "license":
    {
      "id": 1,
      "name": "1xsmall",
      "monthly_cost": "14.00",
      "monthly_credits": "100",
      "time_left": "386:31:46",
      "time_left_grace": "482:31:46"
    }
  }
}
Error response
JSON
                            {
  "exceptions": [
    {
      "exception_text": "Wrong username or password",
      "level": 10000,
      "unixtime": 1567743178,
      "error_id": 109
    }
  ],
  "answer": false
}

The scraping 'job'

When scraping Bing for a collection of keywords you first create a 'job', with a limit of 200 unfinished jobs being held in your account.

The job contains all settings required to start the scraping run:

  • Identifier [Alphanumeric]
    The identifier is a unique alphanumeric string, you can use the current time as a simple automated unique string.
  • Results per Keyword [Numeric] (100,200,300 up to 1000, default 100)
    The number of organic results to scrape per keyword. It is recommended to keep this number below 300 as possible for best performance.
  • Priority [Numeric] (0-20, default 1)
    The priority your job will receive among our scraping infrastructure, higher priority will reduce the time required to deliver results but increases the credit cost.
    A priority of 0 results in a significant credit cost reduction but only idle resources will be allocated to your job. This can result in unpredictable time to complete the Job.
    Jobs below 1000 keywords do not benefit from a higher priority than 2, the shortest finialization time is always up to 60 seconds.
  • Localization - Country [Country] (default 'Default')
    The Bing country to scrape results from (see create_job Api documentation)
  • Localization - Language [Language] (default 'English')
    The localized language used during scraping, the language needs to fit to the selected country to receive accurate results.
  • Keywords [Array]
    Your keywords and key phrases to scrape from Bing
A scraping job
JSON
                            "job": {
    "jobname": "B09/06/2019 15:52",
    "created": "2019-09-06 15:53:03",
    "job_type": "scrape_bing_search",
    "max_results": 200,
    "estimated_keywords": 1,
    "priority": 1,
    "finished": "2019-09-06 15:53:53",
    "credit_cost": "0.65",
    "start": 1,
    "progress": 100,
    "total_keywords": 1,
    "total_results": 1,
    "pages": 1,
    "total_pages": 1,
    "estimated_hours_left": "0.00",
    "estimated_hours_left_hr": "0h 0m",
    "country": "Default",
    "language": "English"
}

Api call: get_jobs

This API call will return up to 50 latest jobs. All details about the jobs are included except the keywords,result data and current scraping progress.
To track the progress of a specific job in detail you need to use 'get_job' instead.

The right sided response example shows an account containing two jobs. The latest job was started and is finished, the second job was never started.

The 'progress' variable using get_jobs will only return 0(%) for unfinished or unstarted jobs or 100(%) for finished jobs.

API request
URL
                            https://scraping.services/api?user=_USER@ACCOUNT_&credentials=_CREDENTIALS_&job_type=scrape_bing_search&task=get_jobs&type=json&human_readable
Response
JSON
                            {
    "exceptions": [],
    "answer": {
        "jobs": [
            {
                "jobname": "B09/06/2019 15:52",
                "created": "2019-09-06 15:53:03",
                "job_type": "scrape_bing_search",
                "max_results": 200,
                "estimated_keywords": 1,
                "priority": 1,
                "finished": "2019-09-06 15:53:53",
                "credit_cost": "0.65",
                "start": 1,
                "progress": 100
            },
            {
                "jobname": "B04/21/2019 15:09",
                "created": "2019-04-21 15:09:26",
                "job_type": "scrape_bing_search",
                "max_results": 1000,
                "estimated_keywords": 1,
                "priority": 1,
                "finished": "0000-00-00 00:00:00",
                "credit_cost": "0.65",
                "start": 0,
                "progress": 0
            }
        ]
    }
}

Api call: get_job

The API call get_job is used to retrieve details about one specific job.

This is used to track the current progress or the estimated finishing time.

In most use cases this call will not be required

The only difference to 'get_jobs' is that you specify the unique 'jobname'

API request
URL
                            https://scraping.services/api?user=_USER@ACCOUNT_&credentials=_CREDENTIALS_&job_type=scrape_bing_search&task=get_job&jobname=_JOBNAME_&type=json&human_readable
API request
cURL
                            curl -G https://scraping.services/api \

                            -d user=_USER@ACCOUNT_ \

                            -d credentials=_CREDENTIALS_ \

                            -d job_type=scrape_bing_search \

                            -d task=get_job \

                            --data-urlencode jobname='B09/06/2019 15:52' \

                            -d type=json \

                            -d human_readable
A scraping job
JSON
                            "job": {
    "jobname": "B09/06/2019 15:52",
    "created": "2019-09-06 15:53:03",
    "job_type": "scrape_bing_search",
    "max_results": 200,
    "estimated_keywords": 1,
    "priority": 1,
    "finished": "2019-09-06 15:53:53",
    "credit_cost": "0.65",
    "start": 1,
    "progress": 100,
    "total_keywords": 1,
    "total_results": 1,
    "pages": 1,
    "total_pages": 1,
    "estimated_hours_left": "0.00",
    "estimated_hours_left_hr": "0h 0m",
    "country": "Default",
    "language": "English"
}
Error response
JSON
                            {
    "exceptions": [
        {
            "exception_text": "Job does not exist",
            "level": 10000,
            "unixtime": 1567786831,
            "error_id": 10013
        }
    ],
    "answer": {
        "job": false
    }
}

Api call: get_keywords

This call is used to retrieve all keywords of a particular job, they will be separated into "finished" and "unfinished"
get_keywords is used to retrieve keywords that have been left unfinished in an aborted job.
Available response types are: html, csv, json

License info request
URL
                            https://scraping.services/api?user=_USER@ACCOUNT_&credentials=_CREDENTIALS_&job_type=scrape_bing_search&task=get_keywords&jobname=_JOBNAME_&type=json&human_readable
Response
JSON
                            {
  "exceptions": [],
  "answer":
  {
    "keywords":
    {
      "jobname": "demo_job",
      "job_type": "scrape_bing_search",
      "keywords_finished": ["keyword1","keyword2","keyword3"],
      "keywords_unfinished": ["keyword4","keyword5","keyword6"],
    }
 }
}

Api call: create_job

To create jobs a JSON encoded array called 'json' needs to be added into the form data.
It is recommended to send all parameters as POST data instead of GET but both methods may be used.

Jobname

The 'jobname' parameter is used to define a unique name for the new job, you may also use a timestamp.
The 'jobname' is used to access and modify the job or export results.

Language and country to use

It is not recommended to use non default 'language' and 'country' parameters, Bing makes scraping keywords in bulk much more difficult on non default languages.

Countries
                            defaultAfghanistanAlbaniaAlgeriaAmerican SamoaAndorraAngolaAnguillaAntarcticaAntigua and BarbudaArgentinaArmeniaArubaAustraliaAustriaAzerbaijanBahamasBahrainBangladeshBarbadosBelarusBelgiumBelizeBeninBermudaBhutanBoliviaBosnia and HerzegovinaBotswanaBouvet IslandBrazilBritish Indian Ocean TerritoryBrunei DarussalamBulgariaBurkina FasoBurundiCambodiaCameroonCanadaCape VerdeCayman IslandsCentral African RepublicChadChileChinaChristmas IslandCocos (Keeling) IslandsColombiaComorosCongoCongo, the Democratic Republic of theCook IslandsCosta RicaCote D'ivoireCroatiaCubaCyprusCzech RepublicDenmarkDjiboutiDominicaDominican RepublicEcuadorEgyptEl SalvadorEquatorial GuineaEritreaEstoniaEthiopiaFalkland Islands (Malvinas)Faroe IslandsFijiFinlandFranceFrench GuianaFrench PolynesiaFrench Southern TerritoriesGabonGambiaGeorgiaGermanyGhanaGibraltarGreeceGreenlandGrenadaGuadeloupeGuamGuatemalaGuineaGuinea-BissauGuyanaHaitiHeard Island and Mcdonald IslandsHoly See (Vatican City State)HondurasHong KongHungaryIcelandIndiaIndonesiaIran, Islamic Republic ofIraqIrelandIsraelItalyJamaicaJapanJordanKazakhstanKenyaKiribatiKorea, Democratic People's Republic ofKorea, Republic ofKuwaitKyrgyzstanLao People's Democratic RepublicLatviaLebanonLesothoLiberiaLibyan Arab JamahiriyaLiechtensteinLithuaniaLuxembourgMacaoMacedonia, the Former Yugosalv Republic ofMadagascarMalawiMalaysiaMaldivesMaliMaltaMarshall IslandsMartiniqueMauritaniaMauritiusMayotteMexicoMicronesia, Federated States ofMoldova, Republic ofMonacoMongoliaMontserratMoroccoMozambiqueMyanmarNamibiaNauruNepalNetherlandsNetherlands AntillesNew CaledoniaNew ZealandNicaraguaNigerNigeriaNiueNorfolk IslandNorthern Mariana IslandsNorwayOmanPakistanPalauPalestinian Territory, OccupiedPanamaPapua New GuineaParaguayPeruPhilippinesPitcairnPolandPortugalPuerto RicoQatarReunionRomaniaRussian FederationRwandaSaint HelenaSaint Kitts and NevisSaint LuciaSaint Pierre and MiquelonSaint Vincent and the GrenadinesSamoaSan MarinoSao Tome and PrincipeSaudi ArabiaSenegalSerbia and MontenegroSeychellesSierra LeoneSingaporeSlovakiaSloveniaSolomon IslandsSomaliaSouth AfricaSouth Georgia and the South Sandwich IslandsSpainSri LankaSudanSurinameSvalbard and Jan MayenSwazilandSwedenSwitzerlandSyrian Arab RepublicTaiwan, Province of ChinaTajikistanTanzania, United Republic ofThailandTimor-LesteTogoTokelauTongaTrinidad and TobagoTunisiaTurkeyTurkmenistanTurks and Caicos IslandsTuvaluUgandaUkraineUnited Arab EmiratesUnited KingdomUnited StatesUnited States Minor Outlying IslandsUruguayUzbekistanVanuatuVenezuelaViet NamVirgin Islands, BritishVirgin Islands, U.S.Wallis and FutunaWestern SaharaYemenZambia
Languages
                            AfrikaansAlbanianAmharicArabicArmenianAzerbaijaniBasqueBelarusianBengaliBihariBosnianBretonBulgarianCambodianCatalanChinese (Simplified)Chinese (Traditional)CorsicanCroatianCzechDanishDutchEnglishEsperantoEstonianFaroeseFilipinoFinnishFrenchFrisianGalicianGeorgianGermanGreekGuaraniGujaratiHausaHebrewHindiHungarianIcelandicIndonesianInterlinguaIrishItalianJapaneseJavaneseKannadaKazakhKinyarwandaKirundiKoreanKurdishKyrgyzLaothianLatinLatvianLingalaLithuanianMacedonianMalagasyMalayMalayalamMalteseMaoriMarathiMoldavianMongolianMontenegrinNepaliNorwegianNorwegian (Nynorsk)OccitanOriyaOromoPashtoPersianPolishPortuguese (Brazil)Portuguese (Portugal)PunjabiQuechuaRomanianRomanshRussianScots GaelicSerbianSerbo-CroatianSesothoShonaSindhiSinhaleseSlovakSlovenianSomaliSpanishSundaneseSwahiliSwedishTajikTamilTatarTeluguThaiTigrinyaTongaTurkishTurkmenTwiUighurUkrainianUrduUzbekVietnameseWelshXhosaYiddishYorubaZulu

Feel free to contact customer support for more information regarding this topic.

Results per keyword

The 'max_results' parameter defines the number of results to scrape, it's recommended to keep this parameter below 200 for best performance.
Possible values are 50,100,150, .. 1000

The correct method for scraping large amounts of data on one keyword is to multiply the keyword with additional words.

Scraping priority and speed

The 'priority' parameter defines the resources our backend will spend on your job, in most cases priority 1 is recommended.
Priority 0 is a standby setting at significantly reduced Credit cost, jobs started with priority 0 are not guaranteed to finished or finish within any timeframe as they use only spare resources.
Priority higher than 1 is not recommended for jobs with less than 1000 keywords.

The JSON parameter to define a job
JSON
                            {
    "job_type" : "scrape_bing_search",
    "language" : "English",
    "country" : "default",
    "max_results" : 100,
    "jobname" : "demo_job",
    "priority" : 1
}
Request (POST)
cURL
                            curl -X POST https://scraping.services/api \

                            -d user=_USER@ACCOUNT_ \

                            -d credentials=_CREDENTIALS_ \

                            -d job_type=scrape_bing_search \

                            -d task=create_job \

                            --data-urlencode json='
{
    "job_type" : "scrape_bing_search",
    "language" : "English",
    "country" : "default",
    "max_results" : 100,
    "jobname" : "demo_job",
    "priority" : 1
}' \

                            -d type=json \

                            -d human_readable
Response
JSON
                            {
    "exceptions": [],
    "answer": {
        "job": {
            "jobname": "demo_job",
            "start": 0,
            "job_type": "scrape_bing_search"
        }
    }
}

Api call: remove_job

The API call may be used to remove a job from your account, only jobs that have not been started can be removed.

Our support team can provide removal rights on finished jobs if required.

API request
URL
                            https://scraping.services/api?user=_USER@ACCOUNT_&credentials=_CREDENTIALS_&job_type=scrape_bing_search&task=remove_job&jobname=_JOBNAME_&type=json&human_readable
Response
JSON
                            {
  "exceptions": [],
  "answer":
  [
    true
  ]
}

Api call: modify_job

After create_job the job needs to be populated with keywords, modify_job is used to append or replace keywords within an existing job.

Similar to 'create_job', a JSON encoded object is appended including the keywords.
'keyword_operation' may be set to 'replace' or 'append'

A Job may be modified, extended or deleted until it has been started.

The JSON parameter to modify a job
JSON
                            {
  "job_type" : "scrape_bing_search",
  "language" : "English",
  "country" : "default",
  "max_results" : 200,
  "jobname" : "demo_job",
  "priority" : 2,
  "keywords" :
  [
    "keyword 1",
    "keyword 2",
    "keyword 3"
  ],
  "keyword_operation" : "replace"
}
Request (POST)
cURL
                            curl -X POST https://scraping.services/api \

                            -d user=_USER@ACCOUNT_ \

                            -d credentials=_CREDENTIALS_ \

                            -d job_type=scrape_bing_search \

                            -d task=modify_job \

                            --data-urlencode json='
{
  "job_type" : "scrape_bing_search",
  "language" : "English",
  "country" : "default",
  "max_results" : 200,
  "jobname" : "demo_job",
  "priority" : 2,
  "keywords" :
  [
    "keyword 1",
    "keyword 2",
    "keyword 3"
  ],
  "keyword_operation" : "replace"
}' \

                            -d type=json \

                            -d human_readable
Response
JSON
                            {
  "exceptions": [],
  "answer":
  {
    "job":
    {
      "jobname": "demo_job",
      "max_results": 200,
      "priority": 2,
      "job_type": "scrape_bing_search",
      "estimated_keywords": 3,
      "credit_cost": 0.9
    }
  }
}

Api call: start_job

Any created job that contains keywords may be started at any time.
Once started, the job is locked from further modification and will immediately deduct the credit cost from your wallet.
Such an action is non refundable and can not be interrupted anymore.

Once a job was started it can be monitored using the "get_job" API call to see how many keywords are already finished.
It is possible to download the keyword results using "get_results" while a job is running.
Scraped keyword results are stored for 14 days, once this timespan has passed the detail information will be removed.

API request
URL
                            https://scraping.services/api?user=_USER@ACCOUNT_&credentials=_CREDENTIALS_&job_type=scrape_bing_search&task=start_job&jobname=_JOBNAME_&type=json&human_readable
Response
JSON
                            {
  "exceptions": [],
  "answer":
  {
    "job":
    {
      "jobname": "demo_job",
      "start": "1",
      "job_type": "scrape_bing_search",
      "credit_cost": "0.90"
    }
  }
}

Api call: get_results (deprecated)

This API call is deprecated since 2018, consider using the more flexible Export API

Using this call keyword results are downloaded in pages, each page contains 50 keyword results to prevent JSON parsing memory issues.
The estimated download size is between 1 and 10 MB per page.

The optional parameter 'result_per_keyword_quantity' is an method to limit the amount of results per keyword.
The optional parameter 'page_quantity' is an method to adjust the number of keyword results from 50 to a higher or lower value.
(50 keywords at 100+ results/keyword, 5000 kewords at 1 results/keyword). each 50 keywords are internally generating an API call (0.01 Credits charge)

The API response object contains an array "keyword_results" with up to 50 keyword results.
Each keyword_result contains information about the keyword as well as all organic results and advertisement results, again in form of two arrays.

The return object

  • keyword_results: is an array of objects, each object is one keyword
    • count_organic: the number of organic (normal) results for this keyword
    • count_creative: the number of advertisements for this keyword
    • language: the language code
    • country: the country code
    • keyword: the keyword
    • resultstats: the number of results reported for each page (array)
    • results_organic:
      • url: The URL
      • title: The title
      • description: The result description
      • page: The page of this keyword (by default we scrape 100 keywords per page)
  • results_creative: an array identical to results_organic
  • page: the current page of results requested (containing up to 50 keywords each containing up to 1000 results)
  • pages_available: the number of total pages currently available
  • pages_expected: the number of total pages expected when the job finished
API request
URL
                            https://scraping.services/api?user=_USER@ACCOUNT_&credentials=_CREDENTIALS_&job_type=scrape_bing_search&task=get_results&jobname=_JOBNAME_&type=json&human_readable
Response
JSON
                            {
  "results":
  {
  "exceptions": [],
  "answer": {
    "jobname": "demo_test_2",
    "job_type": "scrape_bing_search",
    "keyword_results": [
      {
        "count_organic": 101,
        "count_creative": 0,
        "language": "en",
        "country": "Default",
        "keyword": "A Chip on Your Shoulder",
        "resultstats": {
          "1": "799000",
          "2": "650000"
        },
        "results_organic": [
          {
            "url": "https://en.wikipedia.org/wiki/Chip_on_shoulder",
            "title": "Chip on shoulder - Wikipedia",
            "description": "To have a chip on one's shoulder refers to the act of holding a grudge or grievance that readily provokes disputation.",
            "facts": [],
            "page": 1
          },
          {
            "url": "https://en.wikipedia.org/wiki/Chip_on_shoulder",
            "title": "Chip on shoulder - Wikipedia",
            "description": "",
            "facts": [],
            "page": 1
          },
          {
            "url": "https://en.wikipedia.org/wiki/Chip_on_shoulder",
            "title": "Chip on shoulder - Wikipedia",
            "description": "To have a chip on one's shoulder refers to the act of holding a grudge or grievance that readily provokes disputation.",
            "facts": [],
            "page": 1
          },
          {
            "url": "http://dictionary.cambridge.org/us/dictionary/english/have-a-chip-on-your-shoulder",
            "title": "have a chip on your shoulder Definition in the Cambridge English ...",
            "description": "have a chip on your shoulder definition, meaning, what is have a chip on your shoulder: to seem angry all the time because you think you have been treated ...",
            "facts": [],
            "page": 1
          }
        ],
        "results_creative": [],
        "result_knowledge": null
      }
    ],
    "page": 1,
    "pages_expected": 1,
    "pages_available": 1
  }
}
Tools dashboard
Licenses and pricing
Development
Terms of Service
Privacy policy
© 2024 scraping.services
Contact
B2B search engine
B2B examples