A common annoyance for web users is when websites require browser technologies that are not supported by their device. When users access such pages, they may see nothing but a blank space or miss out a large portion of the page's contents.
Starting today in our English search results in the US, we will indicate to searchers when our algorithms detect pages that may not work on their devices. For example, Adobe Flash is not supported on iOS devices or on Android versions 4.1 and higher, and a page whose contents are mostly Flash may be noted like this:
Developing modern multi-device websitesFortunately, making websites that work on all modern devices is not that hard: websites can use HTML5 since it is universally supported, sometimes exclusively, by all devices. To help webmasters build websites that work on all types of devices regardless of the type of content they wish to serve, we recently announced two resources:
- Web Fundamentals: a curated source for modern best practices.
- Web Starter Kit: a starter framework supporting the Web Fundamentals best practices out of the box.
As always, if you need more help you can ask a question in our webmaster forum.
Posted by Keita Oda, Software Engineer, and Pierre Far, Webmaster Trends Analyst
Making sure the deployed annotations are usable by search engines can be rather difficult, especially on sites with many pages, and site owners all around the world haven’t been shy telling us about this. Today we're releasing a feature that should make debugging rel-alternate-hreflang annotations much easier.
The Language Targeting section in the International Targeting feature enables you to identify two of the most common issues with hreflang annotations:
- Missing return links: annotations must be confirmed from the pages they are pointing to. If page A links to page B, page B must link back to page A, otherwise the annotations may not be interpreted correctly.
For each error of this kind we report where and when we detected them, as well as where the return link is expected to be.
- Incorrect hreflang values: The value of the hreflang attribute must either be a language code in ISO 639-1 format such as "es", or a combination of language and country code such as "es-AR", where the country code is in ISO 3166-1 Alpha 2 format.
In case our indexing systems detect language or country codes that are not in these formats, we provide example URLs to help you fix them.
Additionally, we've moved the geographic targeting setting to this part of Webmaster Tools, so that you can find all information relevant to international and multilingual targeting in the same place.
We hope you'll find this new feature useful and that it helps you to identify issues with the rel-hreflang-implementation on your site. If you have comments or questions about the feature, please post in our Webmaster Help Forum.
Posted by Gary Illyes, Webmaster Trends
Webmaster level: intermediate-advanced
To crawl, or not to crawl, that is the robots.txt question.
Making and maintaining correct robots.txt files can sometimes be difficult. While most sites have it easy (tip: they often don't even need a robots.txt file!), finding the directives within a large robots.txt file that are or were blocking individual URLs can be quite tricky. To make that easier, we're now announcing an updated robots.txt testing tool in Webmaster Tools.
You can find the updated testing tool in Webmaster Tools within the Crawl section:
Here you'll see the current robots.txt file, and can test new URLs to see whether they're disallowed for crawling. To guide your way through complicated directives, it will highlight the specific one that led to the final decision. You can make changes in the file and test those too, you'll just need to upload the new version of the file to your server afterwards to make the changes take effect. Our developers site has more about robots.txt directives and how the files are processed.
Additionally, you'll be able to review older versions of your robots.txt file, and see when access issues block us from crawling. For example, if Googlebot sees a 500 server error for the robots.txt file, we'll generally pause further crawling of the website.
We hope this updated tool makes it easier for you to test & maintain the robots.txt file. Should you have any questions, or need help with crafting a good set of directives, feel free to drop by our webmaster's help forum!Posted by Asaph Arnon, Webmaster Tools team
Do you have an Android app in addition to your website? You can now connect the two so that users searching from their smartphones and tablets can easily find and reach your app content.
App deep links in search results help your users find your content more easily and re-engage with your app after they’ve installed it. As a site owner, you can show your users the right content at the right time — by connecting pages of your website to the relevant parts of your app you control when your users are directed to your app and when they go to your website.
Hundreds of apps have already implemented app indexing. This week at Google I/O, we’re announcing a set of new features that will make it even easier to set up deep links in your app, connect your site to your app, and keep track of performance and potential errors.Getting started is easy
We’ve greatly simplified the process to get your app deep links indexed. If your app supports HTTP deep linking schemes, here’s what you need to do:
As we index your URLs, we’ll discover and index the app / site connections and may begin to surface app deep links in search results.
We can discover and index your app deep links on our own, but we recommend you publish the deep links. This is also the case if your app only supports a custom deep link scheme. You can publish them in one of the following ways:
- Add a rel=alternate <link> element to specify the corresponding app URI.
You can insert it in the <head> section of each web page, or in your sitemap. Find out how to implement these methods on our developer site.
- Use the App indexing API
There’s one more thing: we’ve added a new feature in Webmaster Tools to help you debug any issues that might arise during app indexing. It will show you what type of errors we’ve detected for the app page-web page pairs, together with example app URIs so you can debug:
We’ll also give you detailed instructions on how to debug each issue, including a QR code for the app deep links, so you can easily open them on your phone or tablet. We’ll send you Webmaster Tools error notifications as well, so you can keep up to date.
Give app indexing a spin, and as always, if you need more help ask questions on the Webmaster help forum.
Posted by Mariya Moeva, Webmaster Trends Analyst
Few topics confuse and scare webmasters more than site moves. To help you avoid surprises, we've created an in-depth guide on how to handle site moves in a Googlebot-friendly way. So what, exactly, is a site move and how do you go about moving a site correctly?
Basics of site movesA site move is, broadly, one of two types of content migrations:
- Site moves without URL changes. Only the underlying infrastructure serving the website is changed without any visible changes to the URL structure. For example, you might move www.example.com to a different hosting provider while keeping the same URLs and site structure on www.example.com.
- Site moves with URL changes. Here, the URLs on the website change in any number of ways:
- The protocol: http://www.example.com to https://www.example.com
- The domain name: example.com to example.net
- The URL paths: http://example.com/page.php?id=1 to http://example.com/widget
Moving to responsive web designA related increasingly-common question we're seeing is how can a website move from having separate mobile URLs or dynamic serving to using responsive web design. To help you implement this configuration change, please see this new page on our smartphones recommendations site.
And, as always, please ask on our Webmaster Help forums if you have more questions.
Posted by Pierre Far and Zineb Ait Bahajji, Webmaster Trends Analysts
Have you ever used Google Search on your smartphone and clicked on a promising-looking result, only to end up on the mobile site’s homepage, with no idea why the page you were hoping to see vanished? This is such a common annoyance that we’ve even seen comics about it. Usually this happens because the website is not properly set up to handle requests from smartphones and sends you to its smartphone homepage—we call this a “faulty redirect”.
We’d like to spare users the frustration of landing on irrelevant pages and help webmasters fix the faulty redirects. Starting today in our English search results in the US, whenever we detect that smartphone users are redirected to a homepage instead of the the page they asked for, we may note it below the result. If you still wish to proceed to the page, you can click “Try anyway”:
And we’re providing advice and resources to help you direct your audience to the pages they want. Here’s a quick rundown:
1. Do a few searches on your own phone (or with a browser set up to act like a smartphone) and see how your site behaves. Simple but effective. :)
2. Check out Webmaster Tools—we’ll send you a message if we detect that any of your site’s pages are redirecting smartphone users to the homepage. We’ll also show you any faulty redirects we detect in the Smartphone Crawl Errors section of Webmaster Tools:
3. Investigate any faulty redirects and fix them. Here’s what you can do:
- Use the example URLs we provide in Webmaster Tools as a starting point to debug exactly where the problem is with your server configuration.
- Set up your server so that it redirects smartphone users to the equivalent URL on your smartphone site.
- If a page on your site doesn’t have a smartphone equivalent, keep users on the desktop page, rather than redirecting them to the smartphone site’s homepage. Doing nothing is better than doing something wrong in this case.
- Try using responsive web design, which serves the same content for desktop and smartphone users.
Posted by Mariya Moeva, Webmaster Trends Analyst
In April, we launched App Indexing in English globally so deep links to your mobile apps could appear in Google Search results on Android everywhere. Today, we’re adding the first publishers with content in other languages: Fairfax Domain, MercadoLibre, Letras.Mus.br, Vagalume, Idealo, L'Equipe, Player.fm, Upcoming, Au Feminin, Marmiton, and chip.de. In the U.S., we now also support some more apps -- Walmart, Tapatalk, and Fancy.
If you’re interested in participating in App Indexing, and your content and implementation are ready, please let us know by filling out this form. As always, you can ask questions on the mobile section of our webmaster forum.
Finally, if you’re headed to Google I/O in June, be sure to check out the session on the “Future of Apps and Search”, where we’ll share some more updates on App Indexing.
Posted by Erik Hendriks, Software Engineer
The Fetch as Google feature in Webmaster Tools provides webmasters with the results of Googlebot attempting to fetch their pages. The server headers and HTML shown are useful to diagnose technical problems and hacking side-effects, but sometimes make double-checking the response hard: Help! What do all of these codes mean? Is this really the same page as I see it in my browser? Where shall we have lunch? We can't help with that last one, but for the rest, we've recently expanded this tool to also show how Googlebot would be able to render the page.
You can find the Fetch as Google feature in the Crawl section of Google Webmaster Tools. After submitting a URL with "Fetch and render," wait for it to be processed (this might take a moment for some pages). Once it's ready, just click on the response row to see the results.
Handling resources blocked by robots.txtGooglebot follows the robots.txt directives for all files that it fetches. If you are disallowing crawling of some of these files (or if they are embedded from a third-party server that's disallowing Googlebot's crawling of them), we won't be able to show them to you in the rendered view. Similarly, if the server fails to respond or returns errors, then we won't be able to use those either (you can find similar issues in the Crawl Errors section of Webmaster Tools). If we run across either of these issues, we'll show them below the preview image.
We recommend making sure Googlebot can access any embedded resource that meaningfully contributes to your site's visible content, or to its layout. That will make Fetch as Google easier for you to use, and will make it possible for Googlebot to find and index that content as well. Some types of content – such as social media buttons, fonts or website-analytics scripts – tend not to meaningfully contribute to the visible content or layout, and can be left disallowed from crawling. For more information, please see our previous blog post on how Google is working to understand the web better.
We hope this update makes it easier for you to diagnose these kinds of issues, and to discover content that's accidentally blocked from crawling. If you have any comments or questions, let us know here or drop by in the webmaster help forum.
Posted by Shimi Salant, Webmaster Tools team
Sometimes things don't go perfectly during rendering, which may negatively impact search results for your site. Here are a few potential issues, and – where possible, – how you can help prevent them from occurring:
- If your web server is unable to handle the volume of crawl requests for resources, it may have a negative impact on our capability to render your pages. If you’d like to ensure that your pages can be rendered by Google, make sure your servers are able to handle crawl requests for resources.
To make things easier to debug, we're currently working on a tool for helping webmasters better understand how Google renders their site. We look forward to making it to available for you in the coming days in Webmaster Tools.
If you have any questions, please feel free to visit our help forum.
Posted by Erik Hendriks and Michael Xu, Software Engineers, and Kazushi Nagayama, Webmaster Trends Analyst
To help developers and webmasters make their pages mobile-friendly, we recently updated PageSpeed Insights with additional recommendations on mobile usability.
Poor usability can diminish the benefits of a fast page load. We know the average mobile page takes more than 7 seconds to load, and by using the PageSpeed Insights tool and following its speed recommendations, you can make your page load much faster. But suppose your fast mobile site loads in just 2 seconds instead of 7 seconds. If mobile users still have to spend another 5 seconds once the page loads to pinch-zoom and scroll the screen before they can start reading the text and interacting with the page, then that site isn’t really fast to use after all. PageSpeed Insights’ new User Experience rules can help you find and fix these usability issues.
These new recommendations currently cover the following areas:
- Configure the viewport: Without a meta-viewport tag, modern mobile browsers will assume your page is not mobile-friendly, and will fall back to a desktop viewport and possibly apply font-boosting, interfering with your intended page layout. Configuring the viewport to width=device-width should be your first step in mobilizing your site.
- Size content to the viewport: Users expect mobile sites to scroll vertically, not horizontally. Once you’ve configured your viewport, make sure your page content fits the width of that viewport, keeping in mind that not all mobile devices are the same width.
- Use legible font sizes: If users have to zoom in just to be able read your article text on their smartphone screen, then your site isn’t mobile-friendly. PageSpeed Insights checks that your site’s text is large enough for most users to read comfortably.
- Size tap targets appropriately: Nothing’s more frustrating than trying to tap a button or link on a phone or tablet touchscreen, and accidentally hitting the wrong one because your finger pad is much bigger than a desktop mouse cursor. Make sure that your mobile site’s touchscreen tap targets are large enough to press easily.
- Avoid plugins: Most smartphones don’t support Flash or other browser plugins, so make sure your mobile site doesn't rely on plugins.
As always, if you have any questions or feedback, please post in our discussion group.
Posted by Matthew Steele and Doantam Phan, PageSpeed Insights team