Tracking Google Ranking Change by Sector - A New Tool to follow the Bot

With the SEO landscape constantly changing, it can be difficult for Search marketers to follow Google’s updates and their impact on each site. In 2014, there were numerous updates spread throughout the year, with more Panda updates than you can count on one hand. 

On April 21, 2015 (yesterday), Google’s latest update went live, changing the reward structure for mobile-optimized sites and mobile search results. This is the biggest update of the year, as claimed by Google, and we were anticipating significant fluctuations in the Search Engine Result Pages (SERPs) within each industry, but so far nothing happened. 

How did we know?

Ayima Pulse, a tool that was launched yesterday to provide market insight to help marketers adjust their SEO strategy based on any and all algorithm changes and fluctuations in real-time.

Ayima Pulse tracks over 54,000 hand-picked, non-branded keywords in 10 unique industries across the UK & US – more than any other equivalent free tool on the market. This provides unprecedented insight into when shifts occur in Google’s algorithm by industry, and the sites affected.

Pulse is showing fluctuations from the past 30 days in the following industries:

• Telecoms
• Travel
• Electronics
• Insurance
• Fashion
• Finance
• Jobs
• Education
• Gambling
• Automotive

an “All keywords” view that combines all the industry reports into a single view of Google’s fluctuations.

Whilst major Panda, Penguin, and mobile updates garner the coverage to alert marketers of changes within the SERPs, day-to-day fluctuations go unpublicised. In turn, attributing a drop in traffic and possibly conversions becomes problematic. Crucially, these unannounced changes may only affect the industry in which YOU operate. Pulse offers a solution to this problem by offering industry focused tracking that’s updated daily.

How Ayima Pulse Works?

At midnight each day, Ayima Pulse takes Google’s top 100 organic search results for the most popular non-branded keywords relevant to the top 10 industries and add them to Ayima Pulse database Site ranking changes are assessed against those from the previous day, proportionately weighting the significance of each move.

Ayima Pulse plots the difference on a sliding volatility scale of 0-100, updating graphs for the whole market and individual industries.

How Can Ayima Pulse Help Marketers?

Very simply, the more ranking fluctuations you see, the higher the volatility rating. Selecting an industry displays the volatility for that sector along with the top 10 highest ranked sites (assessed across relevant keywords).

If you want to see a site’s visibility trend line and percentage share of voice, just select a site from the top 10 or top 100 list. Or if you want to see how you stack up against the competition, you can select up to 5 sites to compare. Plot lines will display the trend against the volatility of that industry.

Sites displaying an increase in visibility on a highly volatile day have been positively affected by changes in the SERPs. Likewise, those displaying reduced visibility would have been adversely affected.

Who are Ayima Pulse's Competitors? 

Before Ayima Pulse, we have been using Algoroo and MOZ Algorthm changes:

What is Algoroo?

Algoroo is a Google algorithm tracking tool developed by DEJAN. The system monitors roughly 17,000 keywords to 100 search positions deep and looks for fluctuations. Both negative and positive movement is added up to create a single SERP flux metric they call "roo". High roo value indicates a high volatility in Google's search results. Low roo value is usually displayed on an ordinary day, unlikely to be affected by any algorithmic changes at Google. Throughout its timeline you may notice a number of detected events which go onto orange or even red. Google doesn't always disclose internal updates and algorithm changes, but when they do Algoroo add it as an annotation on its graph.

MozCast and Google Algorithm Change History By Moz

MozCast is a weather report showing turbulence in the Google algorithm over the previous day (or see the 5-day history on the left). The hotter and stormier the weather, the more Google's rankings changed. Google Algorithm Change history lists all the algorithm changes by date and name (if there is any)

Resources:

• https://www.ayima.com/pulse/
• https://algoroo.com/
• http://mozcast.com/ 
• http://moz.com/google-algorithm-change

Google is so 2010 - Inspired by Andrew Shotland

Andrew Shotland, the author of Local SEO Guide, once said in an interview with SEOBook, "SEO is not exactly Ghandi-type work. So you better enjoy it."

But that is not his best quotation. To me, his masterpiece, and what really affected me is: 

"Leavng Google is like walking away from a bully. Everything I do now in terms of marketing is pro-active and on my terms. There's no rug under me. There's no dictator telling me how to be righteous. Sure, I stand or fall on my own marketing acumen, which is all I asked for anyway. I never wanted to be punished based on a 3rd party's rules. If I mess up, I want to mess up because my target market (the 2nd party) don't want what I have to offer at the price I'm offering it - not because some 3rd party intermediator decides arbitrarily that I'm not good for my target market."

With thousands of websites going online everyday, SEO agencies are also growing in number, and do not be surprised to find one in every corner. It is not a rocket science and if you are lucky and have good sales team you can make fortune "Selling SEO" to small businesses who believe that they can make fortune too being on top of Google SERP for the keyword "buy" or "fix".

The problem with these companies is that they know very well that SEO is for Google not for the customer, and because they are selling keywords not value, they end up losing both Google and the customer and eventually the deceived business owner who believed that the SEO agency can do her/his site some Apracadabra magic to make it rank.

If only SEO is run by smart marketers who know very well how the market is shaped and understand the true meaning of the word value, only then even Google will become obsolete.
Those Google Hypocrites do not only waste their time doing an old fashion dead job, but they also harm their clients' websites with their spam, scraped nonsense content, and over optimized stuffed with keywords tags.

Why I am saying that now, because after google scared the hell out of everybody regarding its Mobilegeddon algorithm change, nothing happened. But of course I am not blaming Google, I am blaming those who decided to make their sites user friendly just because they fear Google, not because they want their visitors to enjoy their stay on their sites. For those people I would say, you deserve it!

301 Redirect Old Domain Without Passing Link Juice or Referral Signals

If you're hit by Google algorithm's Penguin and tried your best to disavow all the "Bad" links coming to your site, but your site has not been recovered yet, then you might be thinking of starting a new website with clean backlinks portfolio and White Hat SEO.

Of course you do not want your visitors to go to the old abandoned site, and of course you cannot 301 redirect the old domain to the new one, or else you will be transferring all the harmful link signals with you.

So, the best technique to do (after you've decided to start a fresh site) is do this simple yet very effective technique:

1- get a new domain name to use as intermediary  (Example: www.oldsite2.com)
2- Add a Robots.txt file and make the root domain (of the intermediary site) Disallowed

User-agent:*
Disallow: / 

3- Redirect (301) the old domain to the intermediary.
4- Permananetly redirect (301) the intermediary to the brand new domain

More to do:

You can also:
1- Add a robots.txt file to the old site to deindex it from search engines (follow step 2)
2- Use Google's URL removal tool and remove all the URLS of the old site.

A Fresh Beginning:

Now it is a new opportunity to start fresh with a new domain, new content, and better strategy.

Short Story Long:

http://searchenginewatch.com/sew/how-to/2355513/youve-been-hit-by-penguin-should-you-start-over-or-try-to-recover
http://searchenginewatch.com/sew/how-to/2384644/can-you-safely-redirect-users-from-a-penguin-hit-site-to-a-new-domain

20 Reasons why Localization is Important to Website Conversion

Thanks to the global reach of the internet, website localization is one of the best things you can do to increase website conversions. By creating a culture- and language-specific version of your website for each demographic market you target, you become a truly international business. All businesses, even small online retailers, can benefit from localization. In fact, you can’t afford not to have localized websites, and here are 20 reasons why.

1. It offers global expansion and increased reach.

Although English is still the predominant language online, other languages, most notably Chinese, Spanish, French, and Arabic, are quickly closing the distance. Offering web content in additional languages and cultures helps you increase your reach and become a respected international business.

2. Localization helps you appeal to multicultural audiences.

Translation helps international visitors find and buy from you, but it doesn’t consider cultural differences and sometimes doesn’t convey your message or brand very well. Localization includes both cultural and linguistic concerns, helping you reach audiences in different cultures much better.

3. It increases web traffic.

Search engines rank websites with localized versions or pages higher than non-localized websites and return your website as a result more often. On top of that, local sites are more likely to link to you when you provide information in the local language. Increasing traffic is one of the three most important things you can do to boost revenue, and more traffic means more sales.

4. You get more traffic from regional and language-specific search engines.

These smaller search engines have much less competition because they’re small and most businesses don’t have localized websites to appear in results. This means it’s much easier for your localized websites to rank higher than your English website. The higher you rank and the more often your website appears in search results, the more traffic and sales you get.

5. Localization increases brand recognition.

When you translate your website into the language and culture of your target market, you show that you respect and value your audience. They in turn are more aware of your business than your English-only competitors because they see your website more often and more easily understand your message.

6. Localization increases website stickiness and sales.

Having a strong localization plan boosts your presence and sales in a targeted area, such as localizing in French and German to increase sales in Europe. Multiple studies have found that when users are presented with a website in their native language, they stay on the site twice as long and are four times more likely to make a purchase from it.

7. It increases overall ROI.

Increased traffic, conversions, and brand awareness also leads to increased trust, credibility, customer loyalty, and satisfaction, in turn leading to more conversions. Localization is also scalable for both your audience and your budget, delivering huge benefits for only a marginal additional cost.

8. Localization maintains low printing and content distribution costs.

Localizing your website increases reach without raising these costs a few ways. First, you can reuse much of the same content across multiple languages; second, translating your website into a new language and culture is scalable; finally, having a web presence costs the same no matter what language or culture. Having a localized website may also eliminate the need for direct mail such as catalogs and brochures in various languages.

9. It is a cost-effective virtual branch office or satellite location.

Instead of building a brick-and-mortar store or renting an office in an international location, your localized websites become those virtual stores by offering information, products, contacts, and everything else you can deliver digitally.

10. Localization lowers customer support costs.

By answering questions and providing information in a target market’s native language and culture, you give customers what they need online in the best format for them, which reduces the need for multilingual phone and chat support.

11. It allows you to target minorities in your own area.

Many countries have large subgroups with their own languages, cultures, and skyrocketing purchasing power, such as the Latino market in the USA. Creating localized websites for these groups helps you solidify your presence and boost sales in your own area.

12. Localization maintains brand image and voice across cultures.

The problem with straight translation is that it doesn’t consider cultural differences and doesn’t always maintain your branding message. Localization is better than translation because it considers communication, sales incentive, design, layout, and programming specific to each culture and area, so you don’t lose the integrity of your brand across languages.

13. You become a local business.

Localizing your website turns you into a local business, which boosts conversions because many people want to buy locally, you get more traffic from local keywords, and you have an easier time building brand awareness.

14. Localization makes your local marketing stronger.

When you have a website specific to a certain area’s language and culture, your local internet marketing efforts (including search engine optimization, directory listings, and social media) benefit from having a local resource to point visitors to.

15. It makes you more trustworthy and credible.

By using the area’s local slang, idioms, metaphors, and figures of speech, you can communicate with your target customers more easily and directly, reducing confusion and boosting your own reputation.

16. Localization appeals to more customers.

Most web users don’t buy products online in a language other than their own. By offering them that option, you attract more prospects and close more sales.

17. It means fewer abandoned carts.

Programming can be as much a barrier as language or culture. Localization includes proper programming to prevent backend problems such as forms that make it difficult to input personal and payment info. Fewer problems means more closed sales and higher average order value.

18. Localization makes payment easier.

When you enable local credit cards, shipping and tax codes, and buying practices, your localized websites attract customers that would shop elsewhere otherwise, boosting your ROI, conversions, and revenue.

19. It increases local sales.

Offering products, support, FAQs, and other information in your customers’ native languages makes them more likely to buy from you because they have all the information they need in a format they understand to make an informed purchase.

20. Localization increases revenue.

Most consumers care more about language than price. So even if they know they can find a product cheaper somewhere else, they are more likely to buy from you at full price if you have a localized website for them.

SEO Joke: Search Engines and the Happy Birthday card

I don't' know if it is funny or sad. But if this is how SEO works, it would be S.A.D.

The Birthday Card SEO Joke Analogy 

Updates on The Knowledge Graph Results on Google SERP

Did Google move some of the knowledge graph results down the page to throw a duplicate ad graph above it?!

How to make the internet a better place with SEO? by Matt cutts

Does Google Consider SEO to be spam?
A 3 years old video by Mat Cutts that answers a question people still ask.

And the answer is yet the same and will still be the same:

SEO, Stands for Search Engine Optimization, is about trying to make sure that your pages are well represented to search engines. 

How?

• By making sure the site crawlable (Robots, sitemaps, etc.)
• Finding the Right keywords (KW and competitors research)
• Usability and design (titles, headers, landing pages, content, multimedia)
• Speed 
• Responsiveness (is it mobile and tablet friendly or not)
• Update rate (How frequent the site adds new content, and how valuable and informative it is)? 

Read more: Search Engine optimization Starter Guide

Filter Smartphones search terms in GWMTs

Here is how To know which search terms your visitors used to find your website from their mobile phones, by changing the Search Queries report from the default Web filter, you can display impressions, clicks, click-through rate, and average position for your keywords as they appeared in Google search engine results on phones.

Use rel="alternate" hreflang="x" annotations to Serve the Correct Language or Regional URL to Searchers!

The rel='alternate' attribute enables you to tell search engines that a web page is available in different language versions. For example, you could add the following to the head section of a web page if that page is available in English, German and French:

<link rel=”alternate” href=”http://en.example.com” hreflang=”en” />
<link rel=”alternate” href=”http://de.example.com” hreflang=”de” />
<link rel=”alternate” href=”http://.fr.example.com” hreflang=”fr” />

All other languages can be directed to the default version of your website:

<link rel=”alternate” href=”http://example.com” hreflang=”x-default” />

From Google Webmasters:

Some example scenarios where rel="alternate" hreflang="x" is recommended:

• You keep the main content in a single language and translate only the template, such as the navigation and footer. Pages that feature user-generated content like a forums typically do this.
• Your content has small regional variations with similar content in a single language. For example, you might have English-language content targeted to the US, GB, and Ireland.
• Your site content is fully translated. For example, you have both German and English versions of each page.

Using language annotations

Imagine you have an English language page hosted at http://www.example.com/, with a Spanish alternative at http://es.example.com/. You can indicate to Google that the Spanish URL is the Spanish-language equivalent of the English page in one of three ways:

• HTML link element in header. In the HTML <head> section of http://www.example.com/, add a link element pointing to the Spanish version of that webpage at http://es.example.com/, like this:

  <link rel="alternate" hreflang="es" href="http://es.example.com/" />

• HTTP header. If you publish non-HTML files (like PDFs), you can use anHTTP header to indicate a different language version of a URL:

  Link: <http://es.example.com/>; rel="alternate"; hreflang="es"

• Sitemap. Instead of using markup, you can submit language version information in a Sitemap.

If you have multiple language versions of a URL, each language page must identify all language versions, including itself.  For example, if your site provides content in French, English, and Spanish, the Spanish version must include a rel="alternate" hreflang="x" link for itself in addition to links to the French and English versions. Similarly, the English and French versions must each include the same references to the French, English, and Spanish versions.

You can specify multi-language URLs in the same domain as a given URL, or use URLs from a different domain.

It's a good idea to provide a generic URL for geographically unspecified users if you have several alternate URLs targeted at users with the same language, but in different locales. For example, you may have specific URLs for English speakers in Ireland (en-ie), Canada (en-ca), and Australia (en-au), but want all other English speakers to see your generic English (en) page, and everyone else to see the homepage. In this case you should specify the generic English-language (en) page for searchers in, say, the UK. You can annotate this cluster of pages using a Sitemap file or using HTML link tags like this:

<link rel="alternate" href="http://example.com/en-ie" hreflang="en-ie" />
<link rel="alternate" href="http://example.com/en-ca" hreflang="en-ca" />
<link rel="alternate" href="http://example.com/en-au" hreflang="en-au" />
<link rel="alternate" href="http://example.com/en" hreflang="en" />

For language/country selectors or auto-redirecting homepages, you should add an annotation for the hreflang value "x-default" as well:
<link rel="alternate" href="http://example.com/" hreflang="x-default" />

Supported language values

The value of the hreflang attribute identifies the language (in ISO 639-1 format) and optionally the region (in ISO 3166-1 Alpha 2 format) of an alternate URL. For example:

• de: German content, independent of region
• en-GB: English content, for GB users
• de-ES: German content, for users in Spain

Do not specify a country code by itself! Google does not automatically derive the language from the country code. You can specify a language code by itself if you want to simplify your tagging.  Adding the country code after the language to restrict the page to a specific region.  Examples:

• be: Belarusian language, independent of region (not Belgium French)
• nl-be: Dutch for Belgium
• fr-be: French for Belgium 

For language script variations, the proper script is derived from the country. For example, when using zh-TW for users zh-TW, the language script is automatically derived (in this example: Chinese-Traditional). You can also specify the script itself explicitly using ISO 15924, like this:

• zh-Hant: Chinese (Traditional)
• zh-Hans: Chinese (Simplified)

Alternatively, you can also specify a combination of script and region—for example, usezh-Hans-TW to specify Chinese (Simplified) for Taiwanese users.

Finally, the reserved value "x-default" is used for indicating language selectors/redirectors which are not specific to one language or region, e.g. your homepage showing a clickable map of the world.

Common Mistakes

Important: Make sure that your provided hreflang value is actually valid. Take special care in regard to the two most common mistakes:

In general you are advised to sign up with your site to Webmaster Tools. This enables you to receive messages in regard to wrong annotations.

Example configuration: Annotations in action

Example Widgets, Inc has a website that serves users in the USA, Great Britain, and Germany. The following URLs contain substantially the same content, but with regional variations:

• http://www.example.com/ Default page that doesn't target any language or locale; may have selectors to let users pick their language and region.
• http://en.example.com/page.html English-language homepage. Contains information about fees for shipping internationally from the USA.
• http://en-gb.example.com/page.html English-language; displays prices in pounds sterling.
• http://en-us.example.com/page.html English-language; displays prices in US dollars.
• http://de.example.com/seite.html German-language version of the content

rel="alternate" hreflang="x" is used as a page level, not a site level, and you need to mark up each set of pages, including the home page, as appropriate. You can specify as many content variations and language/regional clusters as you need.

To indicate to Google that you want the German version of the page to be served to searchers using Google in German, the en-us version to searchers using google.com in English, and the en-gb version to searchers using google.co.uk in English, userel="alternate" hreflang="x" to identify alternate language versions.

Update the HTML of each URL in the set by adding a set of rel="alternate" hreflang="x" link elements. For the default page that doesn’t target any specific language or locale, add rel="alternate" hreflang="x-default":

<link rel="alternate" hreflang="x-default" href="http://www.example.com/" />
<link rel="alternate" hreflang="en-gb" href="http://en-gb.example.com/page.html" />
<link rel="alternate" hreflang="en-us" href="http://en-us.example.com/page.html" />
<link rel="alternate" hreflang="en" href="http://en.example.com/page.html" />
<link rel="alternate" hreflang="de" href="http://de.example.com/seite.html" />

This markup tells Google's algorithm to consider all of these pages as alternate versions of each other.

Search Engines Ranking Factors - Survey and Correlation Data by MOZ

Every two years, Moz surveys the opinions of dozens of the world's brightest search marketers and runs correlation studies to better understand the workings of search engine algorithms. They gather this data to gain insight into the factors that may help—or hurt—a website's visibility in search engines. Note that these factors are not "proof" of what search engines use to rank websites, but simply show the characteristics of web pages that tend to rank higher.

SEO during site downtime or Maintenance

If you take down your website temporarily, you must inform search engines such as Google. As you could read above, this is done by utilizing the HTTP status code: 503 Service Unavailable, that informs the search engines that the server is temporarily unavailable. To do this one must first create a file that returns a 503 status code on the server. When the search engine sees this, it will understand the situation. This can be done by copying the four lines below into Notepad (or the like) and saving it as 503.php. You must then place this file in the root of your server.

The first two lines tell us that it is a 503 status code, and the last line is used to tell when the website is expected to be online again. Google understands this message, so it is possible to tell Google when to visit the website again. You must either provide a number (seconds) or a date. If you live in Denmark like I do and you expect to return on the 5th of January 2012, at 14:00, you must put down:

Notice that I wrote 13:00:00 in the code, even though I wrote 14:00:00 above. This is due to the fact that the time must be provided in GMT/UTC, which is, in my case, 1 hour behind local time.

But it is not enough to just put a 503 message on your server. You will receive visitors (Google included) from many different sources and to all sorts of pages of your website. They must all be redirected to the message explaining that the website is temporarily closed.

On an Apache/Linux server, this can be easily solved by using a .htaccess file to redirect all the pages towards the 503.php file. The .htaccess file is often used for 301 redirects, but that is not our purpose here. We will use a 302 redirect. You may have been previously warned about using this sort of redirect, and for good reason. It can do a great deal of damage if not used correctly. But in this case, it must be used, and in fact a 301 redirect would be detrimental in its place.

Save the 6 following lines as a .htaccess file and place it in the root of your server as well.

The 'R' in the last line indicates that this is a 302 redirect. R is 302 by default. To create a 301 redirect, it would have said [R=301, L]. The clever thing about this file, however, is that we can give ourselves access to the site and simultaneously show everyone else a 503 message. Let’s say you have the following IP address: 12.345.678.910. You then put the numbers in line 4 as shown below:

When you have placed the two files (503.php and .htaccess) on your server, you’re done. You now have peace and quiet to tinker with your website, as long as you leave those two files in the root of your server – and if Google visits, they’ll know that the site will be back later, and you’ve even let them know when to try again.

But what about passing on the message to your visitors?

How to tell your visitors that the website is only closed temporarily.

With a few additions to the 503.php file, which we made just before, we can pass on a message to visitors:

Source: http://moz.com/blog/how-to-handle-downtime-during-site-maintenance 

Google Algorithms Pets in a nutshell

Are you still getting confused between the 3 algorithms of Google; Panda, Penguin, and the Hummingbird?

which is which and which does what?

here is a simple diagram showing you the function and basic information on each:

Source: http://www.link-assistant.com/news/key-google-updates.html 

Best Analytics Solution and Page Tracking for Chrome and Firefox

   Take advantage of the power of page tagging and track important pages that indicate your business KPIs, such as the thank you page after a purchase is made. I suggest that you list all the important URLs you wish to target.

   You can use a Tool Called WASP (or Web Analytics Solution Profiler) to check if a tag was correctly inserted in your pages. When you run this extension, you can see below all the tags that are inserted in your pages and the scripts related to each.

   Wasp allows you to do quality assurance of your web analytics tags faster than ever. Test and debug your tags, save time, see detected values, find missing tags and beacons, detect double tagging, capture Flash events and export results.

   You can download it for Chrome or Firefox. 

Here are the Chrome features: http://webanalyticssolutionprofiler.com/chrome/features

and the Firefox Features are also here: http://webanalyticssolutionprofiler.com/firefox/features 

Link Audit Formula: What to Do if Google Detected a Pattern of Artificial or Unnatural Links Pointing to Your Site?

Even if you do not do any kind of Black hat SEO, or involved in any unnatural link building schemes, you may find a message from Google telling you that you have been penalized. 

I once got this message from Google Webmaster Tools:

Google has detected a pattern of artificial or unnatural links pointing to your site. Buying links or participating in link schemes in order to manipulate PageRank are violations of Google's Webmaster Guidelines.

As a result, Google has applied a manual spam action to gouverneur.com/. There may be other actions on your site or parts of your site.

---

Recommended action

• Use the Links to Your Site feature in Webmaster Tools to download a list of links to your site.
• Ensure that unnatural links pointing to your site are removed.
• When these changes are made, and you are satisfied that links to your site follow Google's Webmaster Guidelines, submit a reconsideration request. If you're unable to remove links pointing to your site, please provide as much detail as possible in your reconsideration request.
• For an updated list of manual actions currently applied to your site, visit the Manual Actions page. If no manual actions are listed, there is no longer a need to file a reconsideration request.

If we determine your site is no longer in violation of our guidelines, we'll revoke the manual action.

If you have any questions about how to resolve this issue, please visit the Webmaster Help Forum.

If you got it too and do not know what to do, I believe it is time for a Deep Link Audit.

Let's Start:

Collecting the Link Data

To get a complete backlink profile, you will need a paid subscription to a backlink checker. Everyone seems to have a “favorite”, but any of the “Big 4” SEOmoz, Majestic SEO, Link Research Tools or Ahrefs will do the job.

We will be focusing on the following link characteristics:

·         The URL of the page linking to you

·         The URL on your site that is being linked to

·         The IP of the URL linking to you

·         The anchor text used

·         The Percentage (Mix) of Anchor text

·         The follow/nofollow status of the link

·         A measure (rank) of the link’s trust & authority

To begin, enter the URL to audit into the backlink tool. Next, export the data into a CSV file. Sort in ascending value (low to high) by domain/trust/Moz/Cemper whatever rank. In theory this will provide you with a list of links in the order of weakest to strongest. I say “In Theory” as some of the weakest links may be harmless, and some powerful paid links may be killing you. There is no pure algorithmic solution. To do a link audit correctly, requires a manual review.

Analyzing the Link Data

Links that need to be reviewed and considered for removal are the following:

Links that appear on a domain that isn't indexed in Google.

This usually signals a quality problem. A quick way to test for this is to run a “site” command:

Example: “Site:haveresults.com”

sometimes a perfectly good site isn’t indexed, because of a bad robots.txt, like:

User-agent: *
Disallow: /

This usually happens when a website leaves the development stage, but the robots.txt isn’t changed to allow the search engines to crawl the site. That’s why a manual review is important.

Links that appear on a website with a malware or virus warning.

This is pretty self explanatory.

Links that appear on the same page as spammy, unrelated links.

Run the Google Search Command: inurl:links sex,viagra,payday loans and you can find unlimited hacked pages, too.

Links that appear on a page with Google PageRank that is gray bar or zero.

This usually signals poor quality or low trust, but it could also indicate a new page that hasn’t been updated in the PR bar. Gray PR is not the same as PR 0 (zero). The graybar is sometimes a quality indicator, but doesn’t necessarily mean that the site is penalized or de-indexed. Many low quality, made for SEO directories, have a gray bar or PR 0.

Links coming from link networks.

Link networks are a group of websites with common registrars, common IPs, common C-blocks, common DNS, common analytics and/or common affiliate code. Chances are, if a group of websites shares a common ip, you will also find some of the other characteristics of a link network, so that’s where I look first. If using Ahrefs, you would navigate to Domain reports>yourwebsite.com>IPs and get a report like this:

Then Drill down to Domain reports>yourwebsite.com>referring domains, to discover a crappy network

Sitewide Links – especially blogroll and footer links.

Most are unnatural and none pass the juice that they once did.

Watch for exceptions to the rule: After a manual review, I am able to determine that in this case, the first sitewide link found in the tool is natural and there is no need to remove it:. Just one more example of why human intervention is necessary to get a link audit right.

Paid links.

If you are attempting to recover from a manual penalty, every paid link must be removed. No exception. The Google spam team spends all day every day rooting out paid links. After a while, spotting a paid link becomes second nature. That juicy link that you are certain that you can slip by Google will stick out like a sore thumb to the trained eye and will only prolong the agony of a manual penalty.

Beyond specific link types, which could be considered “suspicious”, there are new link rules that need to be reviewed and adhered to in a Post Penguin era.

Post-Penguin Link Audit Considerations

Keep in mind that Penguin is just the latest anti link spam algorithm rolled out by Google. They are hammering websites built on link schemes and rewarding sites with a natural backlink profile. A natural profile contains an assortment of link types, pointing to a website. Your audit should turn up a good mix of:

·         Brand links: Variations include: Your Domain, YourDomain.com, www.YourDomain.com, YourDomain.

·         Exact-match anchor text keyword links: These anchor text links should point to the most appropriate page on the website (the one you are optimizing).

·         Partial-match keyword links: It’s important not to over-optimize with exact match keywords, otherwise you could trip a phrase based filter.

·         Generic Links: Like “Read More” or “Click Here.” Keep in mind that good content should fill this need with little if any work required on your part.

·         Page title links: Some of your links should be the same as your page title.

There are some good tools on the market like Link Detox and Remove’em to help you with link audits and even link removals. The key takeaway is that no matter what tool you are using, a human review is going to be necessary to “get it right.” Leaving it to metrics alone is a formula for failure.

What follows is a step-by-step, tactical walkthrough of exactly how to perform a link profile audit, and how to figure out which links should be removed and/or disavowed.

What you’ll need:

• Scrapebox (A tool every SEO must have in their arsenal)
• Proxies for Scrapebox (optional, recommended. I recommend going for the “Bonanza” package from the “Exclusive Proxies” section.)
• Microsoft Excel

Find Your Anchor Text Ratio

To get started, we need to analyze the most important signal that Google’s Penguin algorithm looks for: over-optimization of anchor text.

Step 1: Get a list of your website’s inbound links and put the list in your Excel spreadsheet. You can get this information from the following sources:

• Google Webmaster Tools
• Majestic SEO
• SEOMoz‘s Open Site Explorer
• Ahrefs

For the most complete information, try to combine data from all four sources. However, I recommend just using the data from your Google Webmaster Tools account. It’s free, and usually about as thorough as you’ll get from the other sources. Plus, it’s straight from Google. For this walkthrough, we’ll assume you’re using the list from your Webmaster Tools account.

Note: To get a list of your inbound links from Google Webmaster Tools, follow the steps below:

1. Login to Google Webmaster Tools
2. Click your Website
3. Click “Traffic” on the left navigation
4. Click “Links to your site”
5. Click “Who links the most”
6. Click “Download latest links”

Step 2: Run your list of links through Scrapebox to get the anchor text of each link. For a detailed walkthrough of how to set up Scrapebox, load proxies, etc., please see my post on how to use Scrapebox to find guest blogging opportunities. Depending on how long your list of links is, and how many proxies you’re using, this step could take a long time.

For lists of links that are 1,000 or less, it shouldn’t take more than 10 minutes. But several nights ago, I ran a report on a list of links that was over 43,000, and I had to let Scrapebox run over night in order to complete.

Step 3: Export the report to Excel on your desktop. You may need to open and re-save the file after you export it, because for some reason it often corrupts immediately after export. Opening and re-saving the spreadsheet should fix it.

Step 4: Within your spreadsheet, sort your columns as such:

• Column A: Source URL
• Column B: Destination URL
• Column C: Anchor Text
• Column D: Found?

Step 5: Sort column D by alphabetical order and remove all rows in which column D’s value is anything other than “Found.” You’ll likely see lots of “Not Found,” “Error 404″ and such from the Scrapebox output, which should be removed.

Step 6: Delete Column D (it’s no longer necessary).

Step 7: Add a new Column D with header “Number of Anchor Occurrences.”

Step 8: In cell D2, enter the following formula: =COUNTIF($C$2:$C$6633,C2).

Note: Change “6633″ in the above formula to whatever the number of the last row of your data set is.

Step 9: Apply this formula to all rows in column D by clicking in cell D2 and then clicking the box in the lower-right of the cell, and dragging it down the entire length of Column D. You’ll now have a list of the number of occurrences of each anchor text in the spreadsheet.

Step 10: Open a new tab (or worksheet) within your spreadsheet and paste in the data from Columns C and D.

Step 11: That data will still contain the formulas in the cells, so we need to remove that. To do so, copy/paste the data from columns C and D into notepad. Then, re-copy and paste it back into your new worksheet. The values for “Number of anchor occurrences” will now be absolute values rather than formulas.

Step 12: Now, it’s time to remove duplicates. Remove duplicates by highlighting your two columns, then going to the “Data” tab in Excel and clicking “Remove Duplicates.” In the ensuing popup box, make sure both columns are checked and then click OK.

Step 13: Add a new column C with header “Percent of Total.”

Step 14: Sort by Column B (“Number of anchor occurrences”) from largest to smallest.

Step 15: Scroll down to the last row containing data, and in column B, in the cell directly below the cell containing the last piece of data, enter the following formula: =SUM(B2:B6633).

This will result in the total number of links.

Note: Change “6633″ in the above formula to whatever the number of the last row of your data set is.

Step 16: In Column C (“Percent of Total”), click in cell C2 and type the following formula: =B2/$B$422.

Note: Change “422″ in the above formula to the number of the row that contains the total number of links, which you created in step 15.

Step 17: Change the format of the values in Column C to “Percentage” with two decimal points. You can do this by highlighting the column, right-clicking, and selecting “Format Cells” then changing the “Category” setting to “Percentage.”

Step 18: Apply this formula to all rows in column C. You should now have a list of percentages of anchor text as a ratio of the entire link profile.

Step 19: Highlight in red any rows in which the anchor text exceeds 2 percent of the overall link profile, EXCEPT the following anchor types:

• Brand anchors
• Naked URLs
• Images (i.e. no anchor text)

The remaining highlighted anchor text is the anchor text for which your inbound link profile is over-optimized.

If you’ve made it this far and found no over-optimized anchor text in your inbound link profile, congratulations! You’re probably not a target of Google Penguin. If you did find over-optimized anchor text, read on.

Analyze Your Referring Domains

Next, it’s time to get a list of referring domains, and gather some metrics on each one so we can determine whether we have any domains that need to be completely disavowed.

Step 20: Copy/paste your list of links into a Notepad file.

Step 21: Load that file into Scrapebox using the “Import URL list” button.

Step 22: Click “Trim to Root”

Step 23: Click “Remove/Filter” then click “Remove Duplicate Domains.”

Step 24: Click “Check PageRank” and “Get Domain PageRank” to get the domain PR of each domain.

Step 25: Export the list of domains using the “Import/Export URLs & PR” button.

Step 26: Copy/paste the output from your newly exported file back into your Excel spreadsheet and sort by PR from largest to smallest.

Find Out Which Links and Domains Need to Be Disavowed or Removed

Now, it’s time to figure out which links and domains need to be removed or disavowed.

Step 27: Refer to your list of anchor text percentages. Find the first highlighted anchor (from Step 19) and note what the anchor is.

Step 28: Return to your Scrapebox output with the column that includes anchor text, and sort by anchor text, in alphabetical order.

Step 29: Scroll down the list of anchors until you find the first occurrence of the anchor you noted in step 27.

Step 30: Copy/paste all link URLs containing that anchor into a new worksheet titled “links to disavow.”

Step 31: Repeat steps 27-30 for all anchor texts highlighted in red from Step 19.

Step 32: Refer again to your list of anchor text percentages. Go through each anchor and eyeball any anchors that are completely unrelated to the niche or maliciously and obviously spam (for example, porn, gambling, or viagra-related anchors). Add all links containing these anchors to your “links to disavow” worksheet in addition to a new, separate list.

Step 33: Load your list of links from the “links to disavow” worksheet into Scrapebox and get the domain PageRank of each link.

Step 34: Copy/paste the output from your newly exported file back into your Excel spreadsheet and sort by PR from largest to smallest.

Step 35: Highlight all links with a PR of 4 or below, and all links with malicious or completely unrelated anchor text.

Step 36: Add the highlighted links to your “links to disavow” list. Now, it’s time to figure out which domains to completely disavow.

Step 37: Copy/paste your list of links from Step 33 (your “links to disavow” spreadsheet) into a Notepad file.

Step 38: Load that Notepad file into Scrapebox and repeat steps 20-26.

Step 39: Add all domains with PR 2 or below to your disavow list.

Step 40: Eyeball the remaining domains and highlight any that don’t end in the following extensions (unless you’re sure you don’t want to remove them):

• .com
• .net
• .org

Step 41: Add the highlighted domains to your “links to disavow” list.

You should now have a list that contains the following:

• A list of links that contain anchor text for which your inbound link profile is over-optimized, which reside on a domain that’s PR 4 or less
• A list of links that contain spammy, malicious, or completely unrelated anchor text
• A list of domains that contain links to your website with over-optimized anchor text and are also PR 2 or less
• A list of domains with domain extensions that are not .com, .net or .org

To disavow an entire domain, use the following format:

domain:spamdomain1.com
domain:spamdomain2.com
domain:spamdomain3.com

To disavow individual links from a domain, use the following format:

http://spamdomain4.com/contentA.html

http://spamdomain5.com/contentB.html

http://spamdomain6.com/contentC.html

Your disavow list should look like this:

domain:spamdomain1.com
domain:spamdomain2.com
domain:spamdomain3.com
http://spamdomain4.com/contentA.html

http://spamdomain5.com/contentB.html

http://spamdomain6.com/contentC.html

Step 42: When you’re ready to submit your list of links to disavow, follow Google’s official instructions on how to do so.

Closing Thoughts

• If you have access to the SEOMoz API, feel free to substitute domain authority (DA) as your metric rather than PageRank. This is a more accurate metric to use, but it’s expensive to use it in bulk. In step 35, substitute PR 4 with DA 40 or below. In Step 39, substitute PR 2 with DA 30 or below.

• Why did I choose 2 percent as the threshold for over-optimization? I’ve done at least 50 inbound link profile audits, and in my experience, the sweet spot appears to be about 2 percent.  The 2 percent figure is purely based on my hands-on experience in the field working with real clients who were penalized by Google Penguin.

• How did I come up with the specific PR and DA thresholds for disavowal? Again, this is based purely on my experience in the field. There’s no textbook that’ll tell you the “right” number(s) or even metrics to use.

Source: 
http://www.searchenginejournal.com/how-to-know-which-links-to-disavow-in-google/50709/
http://searchenginewatch.com/article/2207168/How-to-Conduct-a-Link-Audit