Scraping Top Pages in GWT

  1. Set Parameters

    1. Select Report Type – “Top Pages”.
    2. Set “Date Range” with “Start Date” and “End Date”.
    3. Select “Rows” to display – choose maximum = 500.
    4. Select “Filters” – main options under “Search” are All, Web or Mobile – disregard “Location” and “Traffic” filters.

    In practice, for a given Date Range (e. g. One Week from Saturday to Sunday), we generate 2 Filtered (Web vs. Mobile) data-sets.

  2. Generate 2 (Straightforward) Excel Files

    1. Click on “Download This Table” – save to tp-dtt.csv – open in Excel – save to <start date>-<end date><profile>-dtt-tp.xls.
    2. Click on “Download Chart Data” – save to tp-dcd.csv - open in Excel – save to <start date>-<end date>-<profile>-dcd-tp.xls.
    3. E.g. 20140315-20140328-web-dtt-tp.xls.
  3. Build a Macro to Blow-Out the Queries & generate 3rd Excel file

    1. Copy and paste Source Code for Profile=”Web” into (say) source.txt.
    2. Using PowerGrep, re-write source.txt as  iMacros TAG commands:
      PowerGrep - Settings for Collect data from GWT source code
      Where we search on (getElementById)(\u0028\u0022)([^\u0022]+) and replace with/collect to TAG POS=1 TYPE=A ATTR=ID:\3
    3. Copy and paste TAG POS lines from source.txt into iMacros macro template (say) scrape.iim:
      VERSION BUILD=8810214 RECORDER=FX
      TAB T=1
      URL GOTO=https://www.google.com/webmasters/tools/top-search-queries?hl=en&siteUrl=http%3A%2F%2Fwww.site.com%2F&de=20131207&db=20131201&qv=change&type=urls&grid.s=500&prop=ALL&region=&more=true
      TAG POS=1 TYPE=A ATTR=ID:ud_4C057013_C11A7450_F2B4C340
      TAG POS=1 TYPE=A ATTR=ID:ud_7BB0D7E1_944C001F_71BB9CF7
      ...
  4. Blow-Out & Scrape the “Top Pages” Queries

    1. Within iMacros, run scrape.iim to blow-out the queries for Top Pages.
    2. Select, copy and paste the blown-out queries on-page table into Excel (say) tp-scr.xls.
      Page Impressions Change Clicks Change CTR Change Avg. position Change
      www.site.com/
      92,789 0% 10,536 -8% 11% -1 4.2 -0.2
      query01 62,674 7,742
      query01 5,146 723
  5. Repeat Steps 2 – 5 for Filter=”Mobile”.

  6. Leaving us with …

    For Filter=”Web” and Filter=”Mobile” we have generated 3 Excel files:

    • 20140315-20140328-web-dtt-tp.xls - Top Pages - Download This Table – Step 2
    • 20140315-20140328-web-dcd-tp.xls - Top Pages – Download Chart Data – Step 2
    • 20140315-20140328-web-scr-tp.xls - Top Pages – Scraped – Step 4

See Merging GWT \”Top Queries\” for Filter = (All | Web | Mobile)

The following two tabs change content below.

allenpg

Latest posts by allenpg (see all)

Leave a Reply