Guide to using Auto Escape

Introduction

Auto Escape is an optional mode of execution in the Template System developed to provide a better defense against cross-site scripting (XSS) in web applications. In this mode, the Template System assumes responsibility for applying the proper escaping modifiers for each variable in your template (and templates it may include). As such, the template developer no longer needs to manually apply escaping modifiers to each variable, a process which is repetitive and error-prone particularly in larger and more complex applications.

In order for the Template System to properly determine the escaping modifiers to apply to variables, it activates a built-in HTML-aware parser during template initialization. The parser scans the template to determines the context in which each variable is being emitted. The scanning is performed at initialization-time only hence does not contribute to processing costs at template expansion time.

Refer to Security Considerations for additional security information on escaping directives and their use according to the context in which content is being expanded.

Current Status

Auto Escape currently supports well HTML and Javascript templates. It also provides very basic support for CSS, JSON and XML templates where it applies the corresponding escaping modifier but without utilizing a parser. We may in the future provide parsers specific to these contexts and modifiers for them that are finer-grained.

Overview

This section provides more background on Auto Escape. Refer to How to Auto Escape for information on how to leverage this mode in your application.

Using template modifiers

In the simplest case, where you only want to use template-modifiers to achieve html-safety (that is, you only use html_escape, url_query_escape, and so forth), you can stop specifying template-modifiers in your template at all. The Auto Escape mode will generate the modifiers on its own and perform the correct escaping.

If you need other types of modifiers, including your own custom modifiers, you can still specify them in your template and they will continue to work just the same. The Auto Escape mode will simply apply the appropriate escaping modifiers after your own.

Other reasons for manually specifying variable modifiers include:

To indicate a variable is safe. You may add the :none modifier as the last modifier for a given variable, in which case the Auto Escape mode will leave it untouched and not apply escaping directives to it. Use only when you are sure the variable is trusted to be safe from XSS.

To select a different escaping modifier from a list of equivalent modifiers. Certain modifiers are equivalent from an escaping point of view in that they would all ensure given content is safe when escaped in a specific context. The Auto Escape mode choses the most common escaping modifier for that use but allows you to indicate a different choice as shown below:

TemplateContext	Primary Modifier	Accepted alternatives
`TC_HTML`	`:html_escape`	`:pre_escape` `:html_escape_with_arg=snippet` `:html_escape_with_arg=attribute` `:html_escape_with_arg=url` `:url_query_escape` `:url_escape_with_arg=html` `:img_src_url_escape_with_arg=html`
`TC_HTML` and `TC_CSS`	`:cleanse_css`	`:url_escape_with_arg=css` `:img_src_url_escape_with_arg=css`
`TC_XML`	`:xml_escape`	`:html_escape` `:html_escape_with_arg=attribute`
`TC_JS`	`:javascript_escape`	`:url_escape_with_arg=javascript` `:img_src_url_escape_with_arg=javascript`
`TC_JSON`	`:javascript_escape`	`:json_escape`

To add additional escaping modifiers. The Template System will never remove escaping directives you explicitly specify.

Escaping Contexts

The table belows indicates which modifier Auto-Escape selects to escape a given variable, based on the context of the variable being inserted and, possibly, on more specific information such as whether the variable is inside quotation marks. The table only applies for the well-supported TemplateContext values (currently TC_HTML and TC_JS).

In a few cases, we do not have a modifier that is both safe against XSS and respecting of the semantics of the variable therefore we fail the template initialization. An error is logged indicating the cause of the failure.

Context	HTML Quoted?	Examples	Action Performed
Regular HTML Body and HTML comments	Any	<p>Hello {{USER}}</p>	Escape `USER` using `:html_escape`
In URL attribute: Starts at pos 0	Yes	<a href="{{URL}}">;	Escape `URL` using `:url_escape_with_arg=html`
In URL attribute: Other	Yes	<a href="/foo?q={{QUERY}}">	Escape `QUERY` using `:html_escape`
In URL attribute: Starting at pos 0	No	<form action={{URL}}>	Fail template initialization
In URL attribute: Other	No	<a href=/foo?q={{QUERY}}>My Link</a>	Escape `QUERY` using `:url_query_escape`
In STYLE attribute	Yes	<div style="color:{{COLOR}};">	Escape `COLOR` using `:cleanse_css`
In STYLE attribute	No	<div style=color:{{COLOR}};>	Fail template initialization
In Javascript attribute: String literal	Yes	<a href="url" onclick="doFoo('{{ARG}}');">	Escape `ARG` using `:javascript_escape`
In Javascript attribute: Non-string literal	Yes	<a href="url" onclick="doFoo({{ARG}});">	Escape `ARG` using `:javascript_escape_with_arg=number`
In Javascript attribute: Any	No	<a href="url" onclick=doFoo('{{ARG}}');>	Fail template initialization.
In all other attributes	Yes	<b class="{{CLASS}}">	Escape `CLASS` using `:html_escape`
In all other attributes	No	<table border={{BORDER}}>	Escape `BORDER` using `:html_escape_with_arg=attribute`
In Javascript code: In a string literal	Any	<script>var a = '{{VALUE}}';</script>	Escape `VALUE` using `:javascript_escape`
In Javascript code: Non-string literal	Any	<script>var a = {{VALUE}};</script>	Escape `VALUE` using `:javascript_escape_with_arg=number`
In a <style> tag	Any	<style>font-size={{FONTSIZE}};</style>	Escape `FONTSIZE` using `:cleanse_css`

Comments:

For values of URL-accepting attributes, we apply a different modifier if the variable is at the beginning of the attribute value or not because in the former case we consider the variable to represent a complete URL and hence also check that its scheme is safe (HTTP or HTTPS).
We recommend that you enclose attribute values in quotes in particular:
1. Under some conditions outlined in the table above, when variables are present in attribute values that are not enclosed in quotes, we may fail template initialization because we do not have escaping modifiers that are safe to use.
2. Also the determination of which escaping modifier to apply for unquoted attribute values specifically is likely to change in the future.

The Auto Escape mode is designed to be simple to use and to produce correct correct escaping with minimal input. It does however come with restrictions governing the use of this template system.

Limitations

Templates may only be included in regular HTML text. Including a template within HTML comments or inside an HTML tag is currently not supported. In the example below, HEADER and FOOTER are properly included but TAG and COMMENT are not.
```
          <html><head>{{>HEADER}}</head><body>
          <{{>TAG}}>
          <p>
          
          <p>{{>FOOTER}}
          </body></html>
       
```
If the template system detects that you are including a template in another HTML context, it will log a warning.
Included templates may not cross HTML boundaries. They cannot start with one TemplateContext and end in another. For example, the template below crosses boundaries -- it starts in HTML and ends in Javascript -- and is therefore not a valid included template for auto-escaping.
```
        <html>
          <head>
          </head>
          <body>
            Hello USER.
              <script>
                 var a = 'not a nice template';
       
```
To rectify this violation, continue the template by completing the javascript code and terminating it with a </script> tag.
Error reporting. We currently report errors by logging a warning or an error during initial template initialization and parsing if we suspect something is incorrect. Please check for them, as they indicate an error that may cause auto-escaping to work improperly.
Beware of double escaping. When you use the Auto Escape mode, you surrender all variable escaping to the template system. If you also perform your own escaping in your source code, you may face situations where content was escaped multiple times.

In particular, do not use any of the legacy methods below or others that perform variable escaping.
- SetEscapedValue
- SetEscapedFormatttedValue
- SetEscapedValueAndShowSection
Supported HTML contexts. HTML (TC_HTML) and Javascript (TC_JS) templates are well supported. Other template contexts have only basic support. For these contexts, variables are escaped as follows:

TemplateContext Escaping Applied

TC_JSON :javascript_escape

TC_XML :xml_escape

TC_CSS :cleanse_css
New restrictions on the use of BI_SPACE and BI_NEWLINE. These system variables may not be re-assigned to a semantically different value via dict->SetValue or related methods, and they may not have modifiers attached to them in templates. For instance: it's ok to redefine BI_NEWLINE from \n to \r\n, since html treats them identically. But changing it to hello, world! is not allowed.

TemplateContext	Escaping Applied
TC_JSON	`:javascript_escape`
TC_XML	`:xml_escape`
TC_CSS	`:cleanse_css`

A new filename convention

The Auto Escape feature codifies simple conventions of template filenames:

CSS templates optionally include, in their filename, the keywords style or stylesheet or css.
Javascript templates optionally include, in their filename, the keywords js or javascript.

If we find one of these keywords in the filename but it does not match the template type given, we log a warning.

Jad Boutros