Have you ever landed on a webpage only to see a big error message serving as a dead end to your reading? As a user, you’re left with little direction and close the tab. As the website owner, that has bad news written all over it. While these pages serve a purpose, they can be harmful to your website if misused.
An example of a “page not found” error used in the wrong context is a soft 404 error. These can wreak havoc on your search engine rankings, website traffic, and even credibility.
Below, I’ll define what a soft 404 error is and cover the difference between regular (or hard) 404 errors. Then I’ll explain how to find these errors and resolve them.
What Are Soft 404 Errors?
Soft 404 errors are pages that return a “not found” error code (404), but the content on the page is not actually missing. The page may be empty, or it may contain partial content from other pages on the site. Soft 404s can be caused by misspellings in URLs, by outdated links, or by problems with the server.
When it’s a soft 404 error, the error is conveyed to website visitors but not to Google crawl bots. This means the pages continue to be crawled and indexed on Search Engine Results Pages (SERPs.)
A soft 404 is not an official status; rather, it is one Google creates to alert you. This means Google and other search engines continue to crawl these web pages until you take action.
What Is the Difference Between (Hard) 404 and Soft 404 Errors?
The difference between hard 404 errors and soft 404 errors is the status they display to search engines.
With a hard 404 error, the 404 error code is returned to both page visitors and search engines. This tells both that the page doesn’t exist. Here’s an example of a hard 404 error from the clothing site ModCloth:
With a soft 404 error, the 404 error code is displayed to visitors but not search engines.
Instead, they’ll see a 200 OK status which gives them the green light to crawl. So if you’re seeing soft 404 errors in your Google Search Console, it means that there are pages on your site that may be returning 404 errors even though they actually exist.
Can Soft 404 Errors Impact My Site Ranking?
The short answer is yes; soft 404 errors can impact your website’s ranking on SERPs. To understand why, you need to understand how Google crawls a website.
To prevent strain on a website’s server and reduce the workload for the crawl bots, Google uses something called a crawl budget to determine how much and how often to crawl a site. A crawl budget is the number of pages that Googlebot will crawl on your site during each visit. The crawl budget is composed of crawl rate and crawl demand.
According to Google, the definitions are as follows:
- Crawl Rate is the number of requests per second Googlebot makes to your site when it is crawling it.
- Crawl Demand is how much Google wants to crawl your site based on popularity, how long it’s been since it crawled, etc.
When you have a large number of soft 404 errors on your website, you’re using a considerable amount of your crawl budget. This means fewer of your existing pages are crawled, leading to fewer and poorer rankings in SERPs.
Think of it this way: If your site has 100 pages, but Google determines your crawl budget is 70, then Google will only crawl 70 percent of your pages. If 10 pages are soft 404 errors, there’s a risk that some or all of those pages will fall into the crawl budget of 70. Thus, you lose out on the search engines crawling valuable pages.
What Problems Can Soft 404s Cause?
Aside from lower rankings in SERPs, other problems are associated with soft 404 errors on your website.
For example, if Googlebot crawls a page with a soft 404 error and finds that the content on the page is not actually missing, it may assume that your site is serving up fake 404 errors. This could lead to your site being penalized by Google.
One of the biggest problems is a poor user experience. Since soft 404 URLs still appear in search results, users may be directed to non-existent pages.
If a user clicks on a link to a page that returns a soft 404 error, they may assume that the page doesn’t exist and leave your site. This can hurt your site’s bounce rate and cause users to spend less time on your site.
There can also be implications for the website’s performance and operations.
While 404 error pages don’t take up as much server space as pages with content, they do still take up bandwidth. If Google and other search engines direct traffic to a non-existent page, you’re still taking hits, which can slow website speed and performance.
How to Find (and Fix) Soft 404 Errors
Since soft 404 errors aren’t an official status, they aren’t as obvious to website owners.
This is where a tool like Google Search Console comes in. According to Google, Search Console “help[s] you measure your site’s Search traffic and performance, fix issues, and make your site shine in Google Search results.”
One issue Search Console can help you to resolve is soft 404 errors.
1. Find Soft 404 Errors in Google Search Console
To use Google Search Console, sign up for an account and connect your domain. It’s a simple process that takes only a few minutes to complete.
Once connected and Search Console has had time to collect information, you’ll find soft 404 errors in Google Search Console under Coverage:
It can show in one of two ways: as an error or as excluded (as shown above).
The soft 404s that appear with an error status in Google Search Console are what you want to target first. Those with an excluded status should still be addressed, but Google has stopped crawling those pages for one reason or another.
If you don’t see any soft 404 errors in Google Search Console, you can also use a crawler like Screaming Frog to crawl your site and look for pages that return 404 errors. Just go to the “Response Codes” tab and look for any pages that are returning 404 errors.
2. If the Page Doesn’t Exist: Set Up a Proper 404 or 410 Error
If the soft 404 error you received is real, set up a proper 404 (not found) or 410 (content deleted) error on the server. This ensures website visitors and crawl bots see the correct error.
Is there a difference between the 404 and 410 error codes when it comes to search engines?
According to Google’s John Mueller, not really. Both of these error codes tell Google to stop indexing the URL, which is the outcome you want in either case.
Most Content Management Systems (CMSs) have built-in 404 and 410 error pages. To make the most out of your 404 pages, I recommend adding a few related blog posts or providing a search feature or tool.
On my site, for example, users can analyze their website SEO from my 404 page:
If your CMS doesn’t offer one, you’ll need to configure your server and design the page yourself or with the help of a website developer.
While a personalized 404 error page can take time and money, it’s also a great branding opportunity.
3. If the Page Does Exist: Improve the Content and Reindex
Google is good at finding soft 404 errors, but it’s not perfect. It may incorrectly report a page as a soft 404, even if content exists on the page.
This generally occurs when the content is short or irrelevant.
Assuming the page is necessary, you’ll want to bulk up the content to ensure it’s not mistaken for a soft 404. That doesn’t mean using fluff or filler, but relevant and engaging content that improves the quality of your website.
If your webpage needs bulking, consider adding images, charts and diagrams, and data-driven content you can back up with outside references. These lend credibility to your website, which is especially important if the page was returning a soft 404 error. Finding 404 errors can be beneficial for both the backend and front-facing sides of your website and user experience.
4. If the Page Was Moved: Set Up a 301 Redirect
When combining content pages or moving content around, it’s important to set up a redirect as soon as possible. Otherwise, your website visitors may be directed to a 404 page.
If this results in soft 404 errors, then you’ll be happy to know the fix is simple. You just need to set up a permanent 301 redirect to the newer page on your website.
The majority of CMSs have a built-in redirect tool. Simply enter the old URL and the new one, and the traffic is redirected automatically. This guide walks you through setting up redirects on WordPress.
Once set up, the redirect is communicated to search engine crawlers so they crawl the correct pages.
FAQs
What are soft 404 errors?
Soft 404 errors are “page not found” errors that weren’t properly communicated to search engine crawl bots. The error shows correctly to website visitors but not to crawl bots, which means they continue to crawl those pages, wasting time and resources.
What causes a soft 404?
There are a few different things that can cause a soft 404 error. The most common reason is that the page has been moved to a new location and a redirect hasn’t been set up. Other common causes include poor content on the page and pages that return 404 errors even though they actually exist.
Will fixing soft 404 errors increase my website traffic?
When soft 404 errors are resolved, the crawl bots can focus on a larger percentage of your active content pages. As a result, more content appears in SERPs. In theory, this can increase website traffic.
What is the difference between regular 404 and soft 404 errors?
The difference between a regular 404 and a soft 404 error is what is communicated to search engine crawl bots. With a regular 404, the website server delivers the 404 error to the crawl bot. With a soft 404, the website server delivers a 200 OK status, which means the bots continue to crawl the page.
Will Google penalize me for soft 404 errors?
There’s no official penalty for soft 404 errors, though there are natural consequences if the errors aren’t resolved. These include lower visibility on search engine results pages and a poor user experience.
Does having 404s affect SEO?
404 errors can hurt your site’s ranking if they’re not properly handled. If Googlebot crawls a page with a 404 error, it may assume that the page doesn’t exist and stop crawling your site. As a result, your most important pages may not be getting crawled as often as they should, which can hurt your site’s ranking.
Additionally, if users click on a link to a page that returns a 404 error, they may assume that the page doesn’t exist and leave your site. This can hurt your site’s bounce rate and cause users to spend less time on your site, both of which can negatively impact your site’s ranking.
To avoid these problems, be sure to set up a proper 404 or 410 error if a page doesn’t exist, and be sure to set up a 301 redirect if a page has been moved.
Additionally, make sure that all of your pages have good content that is relevant to what users are looking for. You can perform technical SEO audits throughout the year to ensure your pages are up to par. By taking these steps, you can help ensure that your site’s ranking isn’t hurt by soft 404 errors.
How do I fix soft 404 errors?
If the page doesn’t actually exist, creating a customized 404 page or 410 error will help. A 404 error tells users that the page they’re looking for doesn’t exist, while a 410 error tells users that the page has been permanently removed.
To set up a 404 or 410 error, you’ll need to add a “404.html” or “410.html” file to your site.
If the page does exist but is returning a soft 404 error, you’ll need to improve the content on the page and then reindex it. This can be done by adding more relevant content to the page or by improving the page’s title and meta tags.
If the page was moved to a new location, you’ll need to set up a 301 redirect. A 301 redirect tells users that the page has been permanently moved to a new location. To set up a 301 redirect, you’ll need to add a “301.html” file to your site.
Conclusion
When a page is removed from your website, there are proper ways to signal this removal to both website visitors and search engine crawl bots.
The most common method is setting up a 404 error page. If not configured properly, though, the 404 error may not be communicated to crawl bots. Thus, a soft 404 error occurs.
While a soft 404 error can cause issues with search engine rankings as well as user experience, they are easy to find and resolve.
Depending on the circumstances, you might set up a permanent 404, set up a 301 redirect, or improve the page content and reindex it. These fixes take just a few minutes but can significantly impact website performance.
When was the last time you checked for soft 404 errors?
