What You'll Learn in This Article
8 key topics covered to help you take action.
What Is A Sitemap?
How Important Is A Sitemap For SEO?
Does Your Website Need One? Google's Own Test
What Belongs In It, And What Does Not
How To Submit It
How To Read What Google Reports Back
Crawl Budget: A Worry You Can Probably Drop
Where A Sitemap Sits In The Bigger Job
Best Marketing Malaysia
"We have submitted the sitemap" is one of the first things a web developer tells a client, often in a tone that suggests the SEO is now done. It is not. Finding a page and ranking it are separate steps, and a sitemap only helps with the first.
This guide explains what a sitemap is, whether a Malaysian business website needs one, what belongs in it, how to submit it, and how to read what Google reports back. Every point is checked against Google's own documentation, because much of what is written about sitemaps is out of date.
What Is A Sitemap?
Google's definition is short: a sitemap is a file where you provide information about the pages, videos and other files on your site. Search engines read it to crawl the site more efficiently. In practice it is a plain list of your page addresses, written for machines, and often found at yourdomain.com.my/sitemap.xml.
You may also hear of an "HTML sitemap". That is an ordinary page of links for human visitors, and a different thing. When people talk about a sitemap and SEO, they mean the file.
The file can carry more than addresses. Google says it can also state when a page was last updated and which other language versions of it exist, which is useful if your site runs in English, Bahasa Malaysia and Chinese.
How Important Is A Sitemap For SEO?
It matters for being found. It is not a way to rank higher: every benefit Google lists for a sitemap is about discovering and crawling pages. Google comes to know about a page in one of three ways: it has visited the page before, it follows a link from a page it already knows, or the owner submits a list. The sitemap is that list.
So a sitemap matters as much as the other two routes are failing you. On a small site where every page is in the menu, they are not failing. Google's SEO Starter Guide says of submitting a sitemap that it is not required, and that you should first focus on making sure people know about your site.
Google is just as clear about the limits. A sitemap helps search engines discover the addresses on your site, but it does not promise that everything in it will be crawled and indexed. Its guide to building one calls a submission "merely a hint".
Does Your Website Need One? Google's Own Test
Google's page on sitemaps gives two lists. You might need a sitemap if:
- your site is large, because on a large site it is harder to make sure every page is linked from at least one other;
- your site is new and few other sites link to it;
- your site has a lot of video and images, or appears in Google News.
You might not need one if:
- your site is small, which Google puts at about 500 pages or fewer that you want in search results;
- your pages are comprehensively linked to each other, so that Google can reach every important one by following links from the home page;
- you have few media files or news pages that you want shown in search.
Hold a few Malaysian businesses against that test. A fifteen-page site for an accounting firm in Seremban, with every page in the menu, gains little. An online store in Penang with six thousand products gains a great deal. So does a clinic in Kota Kinabalu whose site went live last month and has no links from anywhere yet.
Even so, Google adds that in most cases a site will benefit from having one. It also notes that if you use a system such as WordPress, Wix or Blogger, it has likely made a sitemap available already. For most owners the real question is not whether to create a sitemap but whether the one they have is right.
What Belongs In It, And What Does Not
Google's instructions for building a sitemap come down to one sentence: include the addresses that you want to see in Google's search results. Write each one in full, starting with https.
That rule also tells you what to leave out: addresses that redirect somewhere else, pages you have marked noindex, duplicate versions of a page, and pages nobody should arrive at from a search, such as the cart, the login page and internal search results. Listing them sends a mixed message, because Google reads every page listed in a sitemap as one you are suggesting as the main version. It calls that a weak signal beside a redirect, but there is no sense in pointing it the wrong way.
Three technical points are worth knowing when you talk to your developer.
- Size. A single sitemap is limited to 50 MB uncompressed or 50,000 addresses. A bigger site splits its list into several files and submits them together through a sitemap index file.
- The last-modified date. Google uses it if it is consistently and verifiably accurate. A date that changes every night, whether or not the page did, is not accurate.
- Priority and change frequency. Google ignores both values. If someone offers to "set your priorities" in the sitemap, nothing will come of it.
How To Submit It
- Find it. Type your domain followed by /sitemap.xml into a browser. If nothing appears, ask whoever built the site where it is.
- Submit it in Search Console. Open the Sitemaps report, paste the address of the file and submit.
- Name it in robots.txt as well. Google's instructions give the line to add: the word Sitemap, a colon, and the full address of the file. It can go anywhere in your robots.txt file.
Older guides tell you to "ping" Google with the address of the file. Google announced in June 2023 that the ping endpoint was being retired, and said sitemaps can still be submitted through robots.txt and Search Console.
Then wait. Google says it can take some time to crawl the addresses listed in a sitemap.
How To Read What Google Reports Back
The Sitemaps report gives each file a status. Success means it was loaded and processed with no errors. Has errors means it was fetched, but something in it is wrong. Couldn't fetch means Google could not get the file at all.
The figure beside it, "Discovered pages", is only the number of addresses Google read out of the file. It is not the number indexed. For that, open the Page indexing report and filter it to "All submitted pages". It shows how many of the pages in your sitemap Google has indexed, and gives a reason for each one it has not.
Read the reasons in two groups.
- Page with redirect, URL marked noindex, Not found (404). The sitemap is listing addresses it should not. Fix the list.
- Crawled, or discovered, and currently not indexed. Google knows about the page and has not kept it. The sitemap has done its job, and submitting it again is not the cure. Look at the page itself: is it thin, or nearly the same as another page?
The report's help page makes one more thing plain. Deleting a sitemap from the report does not take your pages out of Google, which will not forget the sitemap or the addresses in it. To keep a page out of search, it points you to a noindex rule or to removing the page.
Crawl Budget: A Worry You Can Probably Drop
Articles about sitemaps like to say they "save crawl budget". Google's guide to crawl budget is written for sites with a million pages or more, or with 10,000 or more pages that change daily. For other sites, it says that keeping the sitemap up to date and checking the Page indexing report regularly is adequate.
Where A Sitemap Sits In The Bigger Job
A sitemap is plumbing. It puts your pages in front of Google. Whether they deserve to be shown is decided by what is on them and by who links to them.
If pages are missing from Google and the sitemap is in order, the cause lies elsewhere, and the nine checks in our guide to doing an SEO audit are the next step. The sturdier fix for pages Google cannot find is links between your own pages, which is the subject of our guide to internal linking. For a large online store, the sitemap is one line among many in our ecommerce SEO checklist.
When It Is Worth Getting Help
Checking a sitemap takes ten minutes. Working out why a third of a large site is not indexed takes rather longer, and that is the kind of question our SEO audit for Malaysian websites answers. The work that follows is described on our SEO services page for Malaysia.
Two things to know before you call. Our SEO starts from RM8,960 a month, and every price is listed in ringgit. And we are a Singapore company: Singapore is where our own rankings and our client results were earned.
On SEO, the offer on this site is Guaranteed SEO Growth On Last Year, Or We Keep Working Free: page-1 keywords or organic clicks, counted in your own Search Console, growing at least 10% by month 3, 30% by month 6 and 60% by month 12 against the same months last year. It has conditions, and they are set out on our guaranteed SEO page. To go through them for your own site, book a free 30-minute call.
Questions Malaysian Owners Ask About Sitemaps
Do I need both an XML sitemap and an HTML sitemap?
For Google, the file is the one that counts. An HTML page of links is for visitors, and a site with clear menus rarely needs it.
How often should a sitemap be updated?
Whenever a page is added, removed or moved. On most website platforms that happens automatically, so check once that it does, and then leave it alone.
Will submitting a sitemap get a new page indexed faster?
It tells Google that the page exists. Google calls a submission a hint, and says crawling can take some time. A link to the new page from your home page helps as well, because Google also finds pages by following links.
Is Best Marketing a Malaysian company?
No. Best Marketing is a Singapore company, registered in Singapore. We work with you over Zoom, and the prices on this site are in ringgit.
