When a 404 Not Found should be a 404 Not Found

The ASP.NET standard behavior with custom error pages is dubious, at best. When a page is not found, it does not say so. It says that the page has been moved (302), and then it typically says either that the page now indeed was found (200) at the new location, or it says that the redirection to the new location is wrong (404). Humans will read the content on the displayed page, but that status is wrong and will potentially confuse search engines.

If a page is not found, that page request should return a 404 Not Found, nothing else.

Unfortunately, this is non-trivial to achieve in an EPiServer environment with friendly URL enabled.

The solution

The solution requires quite a few steps. To get this to work right, you need at least to:

1. Configure IIS 404 handling to refer to a page that literally does not exist, for example ThisFileDoesNotExist.aspx.
2. Write code to hook Application_Error in Global.asax.cs or equivalent.
3. Write a special page type to display errors, with a very special constructor.
4. Write a custom HtmlRewriteToExternal class and a custom UrlRewriteProvider.

IIS

Because a request to a friendly URL that does not exist will be passed back to IIS, presumably because ASP.NET will return a status stating that it did not handle the request, IIS must also be configured. Another way is to write your own catch-all HttpHandler, but the problem there is that it's hard to append a catch-all to the HttpHandler chain without duplicating the entire chain in your own web.config.

Anyway, once it gets passed back to IIS, it'll be treated as a 404. To get back to ASP.NET and EPiServer to serve the friendly 404 page, configure IIS with a truly non-existing URL that is mapped to ASP.NET. For example a non-existing .aspx page, "ThisFileDoesNotExist.aspx".

This page will be called with a query string parameter consisting of the error code and a semi-colon followed by the URL encoded original URL, so you can use this in your error handling to determine the actual URL. (For the curious, this behavior was actually the basis for friendly URL handling in EPiServer 4).

Application_Error

Application_Error is called whenever there's an un-handled exception in your application. Here you should do approximately the following:

• Check Server.GetLastError(). If it's a HttpException, get the error code from it and do any special handling you need. Finally you call your friendly error page using Server.Execute (not Server.Transfer, see below), set the Response.StatusCode and you're done. Here is also the place where you can check for the query string parameter indicating that you've come via IIS 404-handling.
• If it's some other error, you probably want to set Response.StatusCode to 500, and then Server.Execute a static HTML page stating that an unexpected error occurred.

There are other equivalent ways to hook the error event, use whatever method appeals.

Server.Transfer/Server.Execute problem

Everything seems straightforward until you try it. In this case, it all blows up in the Server.Execute (the same for Server.Transfer) call with an exception in ProcessRequestInternal. This is apparently due to some dependency in one of the SimplePage page extensions. This might be a bug in EPiServer, or at least fixable, but I have not had the to time to reflect that deeply on the issue. So you need to disable it. Problem is, this is during object construction, so it must be done in the constructor. Even worse - you should probably not disable this page extension in edit mode...

public partial class MyPage : SimplePage
{
  public MyPage() : SimplePage(0, HttpContext.Current.Items["InErrorHandler"] == null ? 0 : PageExtensions.SaveCurrentPage.OptionFlag)
  {
  }
}

To get this to work, you'll have to also set Items["InErrorHandler"] before calling the error page from your Application_Error code. This will disable the troublesome page extension when the page is rendered as the result of an error, but will leave it in place in other case such as when editing the page.

The friendly URL issue

No it's all done, right? Sorry... It'll seem ok until you try a not-found on a friendly URL with for example a language prefix and you'll find all your style sheets gone. This is because the friendly URL rewriter will get confused when trying to rewrite relative URLs (those not starting with http(s): or /) relative to a URL that does not exist. (This is probably an EPiServer bug, also probably originally introduced by yours truly. Sorry about that.)

The quick and easy solution is to make EPiServer rewrite all URLs to be root relative (start with a slash) instead. To do this, you'll have to write a small custom HtmlRewriteToExternal class, and to get that to be used, you'll have to make a small custom UrlRewriteProivder. Something like this:

public class MyFriendlyUrlRewriteProvider : FriendlyUrlRewriteProvider
{
    private class HtmlRewriteToRootRelativeExternal : HtmlRewriteToExternal
    {
        protected override bool HtmlRewriteUrl(UrlBuilder int, UrlBUilder ext, UrlBuilder url, Encoding enc, out object obj)
        {
            bool isModified = false;
            PageReference pr = PermanentLinkUtility.GetPageReference(url);
            isModified = Global.UrlRewriteProvider.ConvertToExternal(url, pr, enc);
            isModified |= url.Rebase(int, ext, UrlBuilder.RebaseKind.RootRelative);
            obj = pr;
            return isModified;
        }
    }
    public override HtmlRewriteToExternal GetHtmlRewriter()
    {
         return new HtmlRewriteToRootRelativeExternal();
    }
}

Don't forget to fixup your web.config to refer to your shiny new FriendlyUrlRewriteProvider.

What was learned

One major thing was an unexpected behavior of Server.Transfer, which is the reason for using Server.Execute. Apparently (possibly not under all circumstances, I have not had the time to really ascertain this), Server.Transfer behaves much like Response.Redirect in that after the handler called by Server.Transfer has finished executing, it short circuits remaining events in the HttpApplication pipeline. This means that PostRequestHandlerExecute, ReleaseRequestState, PostReleaseRequestState, response filtering, UpdateRequestCache and PostUpdateRequestCache events will NOT be raised! This messes up all kinds of things, but most importantly it means that EPiServer rewriting of outgoing HTML (the filter is hooked up in PostRequestHandlerExecute, and is implemented as a filter) will not happen. Too bad.

Recall that if you skip UpdateRequestCache the page won't be eligible for output caching. If you skip ReleaseRequestState, you'll probably loose session state etc. It's simply not a good idea to jump to the end without doing these pit stops.

Disclaimer

These are essentially notes from memory. Details may be wrong, and something may be missing. But it should enable you to get started and finished quicker than I did...

If I missed the obvious trivial solution to the issue, please let me know!