a protocol to speed up the indexing of web pages on search engines

For several months, Google and the other major search engines have struggled to index all the new pages published on the web. Microsoft and yandex have jointly worked on a new protocol that could help webmasters speed up the time between publishing and indexing: it’s called IndexNow. Here’s what you need to know about it.

What is IndexNow?

IndexNow is a simple way for website owners to instantly notify search engines of the latest content changes on their website.

In its simplest form, IndexNow is a simple ping letting search engines know that a URL and its content has been added, updated or removed, allowing search engines to quickly reflect this change in their results. search, including indexing the URL in their search engine.

Without IndexNow, search engines can take days or even weeks to discover that the content has changed.because search engines don’t often crawl every URL.

With IndexNow, search engines immediately know “URLs that have changed, which helps them prioritize crawling those URLs and thus limit organic crawling to discover new content.”

IndexNow is licensed under the Attribution-ShareAlike Creative Commons license and is supported by Microsoft Bing and Yandex.

Which search engines support IndexNow?

IndexNow having been developed on the initiative of Bing and Yandex, these two world-class search engines use it natively for the indexing of new search results.

Google has not yet joined the IndexNow project although the latter is currently testing the protocol. He could thus join him very soon.

Microsoft and Yandex have released a new WordPress plugin to speed up the indexing of pages and articles published through WordPress

indexnow wordpress
Overview of the WordPress IndexNow plugin from Microsoft Bing

Microsoft Bing has released a new WordPress plugin that makes it easy to index your WordPress blog and site with the IndexNow protocol.

The plugin was released during the Christmas holidays and is available in the official WordPress plugin directory at this address: https://wordpress.org/plugins/indexnow/

It installs in just a few clicks and requires almost no configuration.

IndexNow WordPress plugin, what is it?

The WordPress IndexNow plugin allows automated URL submission of WordPress sites to multiple search engines without having to do anything, in a 100% automated way.

Once installed, the plugin will automatically generate and host the API key associated with your site.

It will detect the creation / update / deletion of pages in WordPress and will automatically submit them in the background to the various search engines in the market, including Yandex and Bing of course.

This ensures that search engines using this protocol will always be informed as quickly as possible of the latest updates and published articles.

For the moment, Google is testing this protocol without having officially joined it.

IndexNow FAQ: answers to frequently asked questions

This is a French translation of the official IndexNow protocol FAQ.

What is the search engine endpoint for submitting URLs for indexing?

IndexNow-enabled search engines immediately share any URLs submitted to all other IndexNow-enabled search engines, so all you need to do is notify an endpoint:

  • IndexerNow: https://api.indexnow.org/indexnow?url=url-changed&key=your-key
  • Microsoft Bing: https://www.bing.com/indexnow?url=url-changed&key=your-key
  • Yandex: https://yandex.com/indexnow?url=url-changed&key=your-key

I submitted a URL, what happens next?

If the search engines “like” your URL, the search engines will attempt to crawl it for the latest content quickly based on their crawl planning logic and crawl quota for your site.

I submitted 10,000 URLs today, what will happen next?

If search engines like your URLs and have sufficient crawl quota for your site, search engines will attempt to crawl some or all of these URLs.

I submitted a URL, but I can’t see the URL indexed?

Using IndexNow ensures that search engines are aware of changes to your website. The use of IndexNow does not guarantee that web pages will be crawled or indexed by search engines.. It may take time for the change to be reflected in the search engines.

I just started using IndexNow, should I publish the URLs that changed last year?

No, you should only publish URLs that have changed (added, updated, or deleted) since the time you start using IndexNow.

Do submitted URLs count against my crawl quota?

Yes, each crawl counts towards your crawl quota. By posting them to IndexNow you are letting search engines know that you care about these URLs, search engines will generally prioritize crawling these URLs over other URLs they know.

Why can’t I see all submitted URLs indexed by search engines?

Search engines can choose not to crawl and index URLs if they do not meet their selection criteria..

Why is my URL indexed on one search engine but not on others?

Search engines may choose not to select a specific URL if it does not meet its selection criteria.

I have a small website which has few web pages. Should I use IndexNow?

Yes, if you want search engines to discover content as soon as it changes, you should use IndexNow. You won’t have to wait several hours or worse weeks to see your changes on the search engines.

Can I submit the same URL multiple times per day?

Avoid submitting the same URL multiple times a day. If the pages change often, then it is best to wait 10 minutes between changes before notifying the search engines. If the pages are constantly updated (examples: time in Waimea, Weather in Tokyo), it is preferable not to use IndexNow for each change.

Can I submit 404 URLs through the API?

Yes, you can submit dead link pages (http 404, http 410) to notify search engines of new dead links.

Can I submit new redirects?

Yes, you can submit newly redirected URLs (eg 301 redirect, 302 redirect, html with meta refresh tag, etc.) to notify search engines that the content has changed.

Can I submit all the URLs for my site?

Use IndexNow to submit only URLs that have been changed (added, updated, or deleted) recently, including all URLs if all URLs have been recently changed. Use sitemaps to let search engines know about all of your URLs. Search engines will visit the sitemaps every few days.

I received an HTTP 429 Too Many Requests response from a search engine, what should I do?

Such an HTTP 429 Too Many Requests response status code indicates that you are sending too many requests in a given amount of time, slow down or try again later.

When should I change my key?

Search engines will attempt to crawl the {key} .txt file only once to verify ownership when they receive a new key. Plus, you don’t need to change your key often.

Can I use multiple keys per host?

Yes, if your websites use different content management systems, each content management system can use its own key; publish various key files to the root of the host.

Can I use one file key for the whole domain?

No, each host in your domain must have its own key. If your site has host-a.example.org and host-b.example.org, you must have a key file for each host.

Can I use the same key on two or more hosts?

Yes, you can reuse the same key on two or more hosts and on two or more domains.

I have a sitemap, do I need IndexNow?

Yes, when sitemaps are an easy way for webmasters to notify search engines of all the pages on their sites that are available for crawling, sitemaps are rarely visited by search engines. With IndexNow, webmasters “don’t” have to wait for search engines to discover and crawl sitemaps, but can notify search engines directly when new content arrives.

See also  New for Core Web Vitals Report in Search Console

Leave a Comment

Your email address will not be published.