Skip to content

saamerm/Bing-search-rank-tracker

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

27 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Bing Keyword Ranking Check with Python

Are you poor and don’t have money to buy an enterprise rank tracker? Well, today is your lucky day. With this python script, a shell script and crontab, you can automate Bing rank checker in a few simple steps. I will explain step by step how to implement this and leave it running daily. One thing to note, currently the script does not use proxies to check for the keyword rankings, so if you are looking to run big sets of keywords, Bing will notice this and will start showing a captcha.

Update: I have updated the script by adding the possibility of choosing what device you want to make the rank check. The two options are Mobile and Desktop. I will still leave the old script here but will change the name to rank_legacy.py.

Update2: Included a keyword.xls file that will run all your keywords from there. No need to add each one of those on the .sh file anymore. I also added a random sleep between queries so that Bing won't catch us. The script now is more simple and easy to use, but they are smarter than that too.

Table of Contents


Installation

Installation of Python robobrowser

pip install robobrowser

After all, dependencies are installed, we can start testing if the script is working fine. Read the notes at the bottom if you are having issues

Running tests

Before running any test, we want to go into the keywords.xls file and add the keywords we want to check the ranks. We can add as many as we wish to, but the more keywords, the higher the chances Bing will block you. (I will soon include the option of using proxies.)

After that, we open the terminal and go to the folder that rank.py is saved and give the script executing rights.

chmod +x rank.py

Now we can call our script followed by two arguments: the website we are looking for and the device we want to check on this can be mobile or desktop.

python3 rank.py [website] [device] 

For example

We want to check the website https://www.uselessthingstobuy.com/ on mobile against the keyword nothing package we need to include the keyword on keywords.xls file and run:

python3 rank.py https://www.uselessthingstobuy.com/ mobile

This will output the keyword, the ranking of the keyword, the URL that is ranking on Bing, the device you chose, and the date we did this rank check.

Make sure that the device is lower case. If you misspell the device or add capital, the script will run using mobile device as default

nothing+package 1 https://www.uselessthingstobuy.com/product/give-nothing-for-the-person-who-has-everything/ mobile 01-07-2020

This will also generate a CSV file in the folder where rank.py is located. This will include all the information the terminal is showing.

For example:

Keyword Rank URL Device Date
nothing+package 1 https://www.uselessthingstobuy.com/product/give-nothing-for-the-person-who-has-everything/ mobile 01-07-2020

Creating a shell script

Now that we tested that rank.py works fine, we will create a shell script that will run our python script.

We create a new .sh file and add the terminal commands we ran before. Since we are running everything out of a keyword.xls file to make everything easier, we can call the script with the URL we want and the device we want to check in our rank.sh file. So forget about adding multiple lines and sleep times. I included the sleep times on the script and they do random number between 1,10 so that Bing wont catch us. So the only thing we need to have is the following:

#! /bin/bash

/usr/bin/python3 /path_to_my_script/rank.py [website] [device] 

After we create the shell script we would need to make the script executable

chmod +x rank.sh

and then to test the shell script. We go to the console and run it using

./rank.sh

This will output 5 csv files and the output file name is composed by the date + [keyword] + [device]

Cronjob

Now that we have our .sh file running fine, we will set up a cronjob that will run every day at 5:00 pm

In the terminal, we type:

crontab -e

Then we will press the letter i to star editing and we will add the following:

0 17 * * * path_to_my_script/rank.sh

after adding this we press esc and add :wq to save.

Contributing and Questions

If you want to contribute or fix anything please feel free to do so.

If you have any question or need help setting this up please open an issue and will try to help.

If you have a werkzeug Error Read this As of February 2020 werkzug upgraded to 1.0.0 and RoboBrowser lazy developers havent fixed. To fix this you need to go to your Robobrowser folder on your computer something like (/Users/yourusername/opt/anaconda3/lib/python3.7/site-packages/robobrowser/) and open browser.py and add change from werkzeug import cached_property to from werkzeug.utils import cached_property

Please take a look at this url for more info: Link to issue

Alternatively you can add these lines import werkzeug werkzeug.cached_property = werkzeug.utils.cached_property

before import robobrowser to solve the issue

If you are having more issues, you can make sure to see if you have the required packages installed by following these steps:

  • pip install pipreqs
  • pipreqs .
  • pip install -r requirements.txt

Additionally you might have a pandas issue, I solved that on my Mac using pip3 install pandas, then I had to install pip3 install xlrd. and now I just run using python3 rank.py https://thefirstprototype.com/ desktop

Since I don't have Microsoft Excel, when I tried to make changes to the keywords file, it only let me save it as a XLSX file. So I changed the code around line 162 to my new keywords.xlsx file. and I needed to do pip3 install openpyxl and then python3 rank.py https://thefirstprototype.com/ desktop worked again

Check if you are blocked

You can try to run python3 rank.py https://thefirstprototype.com/ mobile instead and that will work ~5 times. To check if you are blocked, go to the function def desktop(keyword,sitename,device,useragent): and under the line browser.open(... add a line print(browser) to print the output. If you are blocked, it will say <RoboBrowser url=https://www.bing.com....> in the output window. I'm not sure how to proceed.

Switch it to work wih Google Search

If you are blocked by Bing, you use the Google browser instead. Just change these:

  • links = browser.find_all("li", {"class": "b_algo"}) to this links = browser.find_all("div", {"class": "KJDcUb"})
  • browser.open('https://www.bing.com/search?q=' + keyword) to this browser.open('https://www.google.com/search?q=' + keyword)

To check if you are blocked on Google, go to the function def desktop(keyword,sitename,device,useragent): and under the line browser.open(... add a line print(browser) to print the output, if you haven't already done that above. If you are blocked, it will say <RoboBrowser url=https://www.google.com/sorry/index?continue=https://www.google.com/search%3Fq%3Dthe%2Bfirst_prototype&q=EgRsqoWdGLO1jYcGIhAgHPLrEoXTpalTKCP5PuFaMgFy> in the output window. I went to the link in my browser, I said I am not a robot, and it redirected me to the Google Search. But when I came back to the Robobrowser, it still gave me the same error. I'm at a roadbloack now

About

SEO: Python script + shell script and cronjob to check ranks on a daily basis

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Python 100.0%