# Template Language

This is a little template language specialized in generating HTML documents.

The language allows web designers to do their job and lets developers put in the dynamic parts. The syntax and terminals are chosen to be readable and allow rendering the templates in the browser as well as in any HTML 5 compatible WYSIWYG editors. Still, when opened in the DSL editor you get the fully featured, statically typed Eclipse editor.

## Overview

A template language works in two modes: Plain text mode, where everything goes directly into the output and the expression mode, where expressions have to be evaluated and the result is inserted into the text. To switch between text mode and expression mode, we use the French quotes `«` and `»`. A document starts in text mode.

The template will be compiled to a Java class with a `generate(params)` method. You can provide additional information like a package declaration, imports and parameters in the preamble inside a template at the beginning of the document.

We provide additional `FOR-ENDFOR` and `IF-ELSE-ENDIF` statements to iterate / branch over fixed text blocks. To distinguish them from the Xbase expressions with the same names, they are in uppercase.

## Running the Example

In the runtime workspace, run the GenerateHtml file as a Java application (Run as… → Java Application from the context menu). This will execute the template MyWebsite and print the result to the console.

## Grammar

This is the grammar of the templates DSL:

    grammar org.xtext.template.Template
       with org.eclipse.xtext.xbase.annotations.XbaseWithAnnotations

    generate template "http://www.xtext.org/template/Template"
    import 'http://www.eclipse.org/xtext/xbase/Xbase' as xbase

    TemplateFile:
    	'<!--''«'
    		('package' package=QualifiedName)?
    		importSection=XImportSection?
    		params+=Parameter*
    		body=RichString;

    Parameter:
    	annotations+=XAnnotation*
    	'param' type=JvmTypeReference? name=ID ('=' defaultexp=XExpression)?;

    RichString returns xbase::XBlockExpression:
    	{RichString}
    	expressions+=RichStringLiteral
    	(expressions+=RichStringPart expressions+=RichStringLiteral)*;

    RichStringLiteral returns xbase::XStringLiteral:
    	{RichStringLiteral} value=TEXT;

    RichStringPart returns xbase::XExpression:
    	XExpressionInsideBlock |
    	RichStringForLoop |
    	RichStringIf;

    RichStringForLoop returns xbase::XForLoopExpression:
    	{RichStringForLoop}
    	"FOR" declaredParam=JvmFormalParameter ':' forExpression=XExpression
    		eachExpression=RichString
    	"ENDFOR";

    RichStringIf returns xbase::XIfExpression:
    	{RichStringIf}
    	"IF" if=XExpression
    		then=RichString
    	(else=RichStringElseIf | "ELSE" else=RichString)?
    	"ENDIF";

    RichStringElseIf returns xbase::XIfExpression:
    	{RichStringIf}
    	"ELSEIF"if=XExpression
    		then=RichString
    	(else=RichStringElseIf | "ELSE" else=RichString)?;

    terminal TEXT : '»' (!'«')* (EOF|'«');

It becomes quite straightforward once you have understood the escaping. Have a look at the last rule TEXT first: It says that a text starts with a closing French quote and ends with an opening quote or the end of the file. By inverting opening and closing quotes we mark up text instead of expressions.

A TemplateFile starts with a comment and switches to the expression mode for the preamble part consisting of the package declaration, the imports and the parameter declaration. The body is a RichString, which is an alternating sequence of RichStringLiterals and RichStringPart. The RichStringLiterals is essentially a text block (in inverted French quotes). The RichStringPart is either an Xbase expression, a RichStringForLoop or a RichStringIf. The latter inherit from the Xbase expressions with the same name to reuse as much of the Xbase infrastructure as possible. The rest should be easy.

## Translation to Java

Each TemplateFile is compiled to a Java class with a `generate` method that takes a lambda expression as a parameter. The lambda expression is called to initialize the template’s properties, by handling the template itself as an argument. This comes along nicely, especially when called from Xtend.

- DSL:

      <!--«
        ...
      »-->

- Java:

      public class MyWebsite {
      ...
        public String generate(final Procedure1<MyWebsite> init) {
          if (init != null)
            init.apply(this);
          String result = generate().toString();
          // remove leading -->
          result = result.replaceAll("^-->\\r?\\n","");
          // trim multi-newline to single newline
          result = result.replaceAll("\\r?\\n\\s*\\r\\n?", System.getProperty("line.separator"));
          return result;

        }
      }

The corresponding code in the TemplateJvmModelInferrer is:

    class TemplateJvmModelInferrer extends AbstractModelInferrer {
    ...
         def dispatch void infer(TemplateFile element,
                                 IJvmDeclaredTypeAcceptor acceptor,
                                 boolean isPreIndexingPhase) {
           val simpleName = element.eResource.URI.trimFileExtension.lastSegment
           val qualifiedName = if(element.getPackage != null)
               element.getPackage + "." + simpleName
             else
               simpleName
        val javaClass = element.toClass(qualifiedName)
           acceptor.accept(javaClass)[
    ...
          // generate a method accepting an initializer lambda expression
          members += element.toMethod("generate", typeRef(String)) [
            parameters += element.toParameter(
              "init", typeRef(Procedures.Procedure1, typeRef(javaClass))
            )
            body = '''
              if (init != null)
                init.apply(this);
              String result = generate().toString();
              // remove leading -->
              result = result.replaceAll("^-->\\r?\\n","");
              // trim multi-newline to single newline
              result = result.replaceAll("\\r?\\n\\s*\\r?\\n",System.getProperty("line.separator"));
              return result;
            '''
    ...

Each Parameter becomes a Java property, i.e. a field with a getter and a setter.

- DSL:

      param title = "No Title"

- Java:

      private String title = "No Title";

      public void setTitle(final String title) {
        this.title = title;
      }

      public String getTitle() {
        return this.title;
      }

In the inferrer, note that we derive the property’s type in three steps: Take the declared one, if there is none, derive it from the initializer and if even that fails, use String as default.

    for (param : element.params) {
      val type = param.type
        ?: param.defaultexp?.inferredType
        ?: typeRef(String)
      members += param.toField(param.name, type) [
        if (param.defaultexp != null)
          initializer = param.defaultexp
      ]
      members += param.toSetter(param.name, type)
      members += param.toGetter(param.name, type)
    }

The body of the template is compiled into a big private `generate()` method. We skip the inferrer code here, as it is straightforward. But we had to extend the compiler to support rich strings and the new `FOR` loop our new control structures. This is described in the next section.

- DSL:

      »<!DOCTYPE html>
      <html lang="en">
      <head>
        <meta charset="utf-8">
        <title>«title»</title>
        <meta name="viewport" content="width=device-width, initial-sca...">
        <meta name="description"
          content="«description»">
        <meta name="author" content="«...

- Java:

      private CharSequence generate() {
        StringBuilder _appendable = new StringBuilder();
        _appendable.append(ObjectExtensions.operator_elvis(
          "-->\n<!DOCTYPE html>\n<html lang=\"en\">\n<head>\n\t<meta...",
          ""));
        _appendable.append(ObjectExtensions.operator_elvis(
          this.title,""));
        // appends galore
      ...

## Extending the Compiler

We have added additional expressions to Xbase, so we have to tell the compiler how to translate them to Java. The XbaseCompiler has a method `doInternalToJavaStatement` that dispatches the compilation with regard to the type of the current expression. This is where we have to hook in with our TemplateCompiler:

    class TemplateCompiler extends XbaseCompiler {

      override protected doInternalToJavaStatement(XExpression expr,
                                                   ITreeAppendable it,
                                                   boolean isReferenced) {
        switch expr {
          RichString : {
    ...   }
          RichStringForLoop : {
    ...   }
          default :
            super.doInternalToJavaStatement(expr, it, isReferenced)
        }
      }
    ...

For a RichString, we declare a variable `_appendable` of type StringBuilder and append the results of all its evaluated expressions. Remember from the grammar that every second expression is a RichStringPart which can be `null`, so we use the ‘elvis operator’ `?:` to insert an empty string in this case.

    RichString : {
      val name = declareVariable(expr, '_appendable')
      newLine
      append('''
        StringBuilder «name» = new StringBuilder();
      ''')
      for (nestedExpression : expr.expressions) {
        nestedExpression.internalToJavaStatement(it, true)
        newLine
        append('''«name».append(ObjectExtensions.operator_elvis(''')
        nestedExpression.internalToJavaExpression(it)
        append(',""));')
      }
    }

As our RichStringLiteral inherits from XStringLiteral, it does not need any special treatment. The same holds for RichStringIf and RichStringElseIif. The RichStringForLoop requires special treatment, because as opposed to the XForLoopExpression that always returns `null`, we want it to return a concatenation of its results. This looks like

    RichStringForLoop : {
      expr.forExpression.internalToJavaStatement(it, true)
      val paramType = typeProvider.getTypeForIdentifiable(expr.declaredParam)
      val name = declareVariable(expr, '_forLoopResult')
      newLine
      append('''
        StringBuilder «name» = new StringBuilder();
        for (final ''')
      serialize(paramType, expr, it);
      append(''' «declareVariable(expr.declaredParam,
          makeJavaIdentifier(expr.declaredParam.name))» : ''')
      internalToJavaExpression(expr.forExpression, it)
      append(") {").increaseIndentation
        expr.eachExpression.internalToJavaStatement(it, true)
        newLine
        append('''«name».append(''')
        expr.eachExpression.internalToJavaExpression(it)
        append(');')
      decreaseIndentation.newLine.append("}")
    }

The compiler now knows how to handle the new expressions in a statement context. In addition, we have to teach it to compile them in an expression context. This is what the second method does:

    override protected internalToConvertedExpression(XExpression obj,
                                                     ITreeAppendable it) {
      if (hasName(obj))
        append(getName(obj))
      else
        super.internalToConvertedExpression(obj, it)
    }

As usual, we have to bind our TemplateCompiler in the TemplateRuntimeModule in order to be picked up as the XbaseCompiler in the context of our language.

## Type Computation

The type system has to know how to determine the types of our new expressions. This is the job of the TemplateTypeComputer: RichString becomes a StringBuilder. As opposed to its super type XForLoopExpression a RichStringForLoop is of type StringBuilder as well. The for-loop’s body is expected to have a type, as the results must be concatenatable, which is different from Xbase’s for-loop.

    class TemplateTypeComputer extends XbaseWithAnnotationsTypeComputer {

    	def dispatch computeTypes(RichString expression, ITypeComputationState state) {
    		super._computeTypes(expression as XBlockExpression, state)
    		state.acceptActualType(getTypeForName(StringBuilder, state))
    	}

    	def dispatch computeTypes(RichStringForLoop expression, ITypeComputationState state) {
    		super._computeTypes(expression as XForLoopExpression, state)
    		state.acceptActualType(getTypeForName(StringBuilder, state))
    	}
    }

Like the compiler, we have to bind this implementation in our runtime module as well.

## Value Converter

The RichStringLiterals still have the French quotes around their values. As we do not want to see them in the output, we have implemented the TemplateValueConverterService and bound it in the runtime module.

## Content Assist

The French quotes are not easy to type on every keyboard. We have adapted content assist to insert them when the cursor is inside a TEXT terminal:

    public class TemplateProposalProvider
        extends AbstractTemplateProposalProvider {
      @Override
      public void complete_TEXT(EObject model,
                                RuleCall ruleCall,
                                ContentAssistContext context,
          ICompletionProposalAcceptor acceptor) {
        acceptor.accept(new CompletionProposal("«»",
            context.getOffset(), 0, 1));
      }
    }

## Syntax Highlighting

Sometimes it is hard to see whether you are in text mode or in expression mode. To give the user better feedback, we have changed the way the text is highlighted. This customization consists of two parts: Add new highlighting styles in the TemplateHighlightingConfiguration and apply them to the text in the TemplateHighlightingCalculator. As this is rather extensively covered in the Xtext documentation, we skip a deeper explanation here.

Next Chapter: Little Tortoise