Text Composers

Text Composition

To fully understand the  TextComposerLib library’s capabilities, we first need to understand the text composition classes located under the TextComposerLib.Text namespace. These classes are designed to as text construction and manipulation components suitable for code generation as the main application. The text composers can be integrated for many tasks including scripting and template engine design.

1. Concatenate Methods : Simple Text Compositions

The simplest way to compose text is by concatenating several strings into a single string. Several extension methods are created inside the TextComposerLib.Text.TextUtils static class for this purpose. The Concatenate overloaded extension methods can accept a sequence of items and join them into a single string with various forms:

  • The items can be joined directly in the same sequence they are presented.
  • The items can be joined with a separator string between each two successive items.
  • The items can be joined with a separator and a prefix and suffix strings added for the whole joined string.
  • The items can be joined with a separator, a final prefix and suffix, and an item prefix and suffix that surrounds each item in the sequence.

The JoinPairs extension methods work on two string sequences to zip them together into another sequence of strings in various forms:

  • The corresponding items are joined directly.
  • The corresponding items are joined with a separator string in the middle.
  • The corresponding items are joined with a separator and prefix and suffix strings are added per item in the final string sequence.

Concatenate Methods

Example40

The shown code will produce this output based on the Concatenate method:

Example40

To see an example for the JoinPairs method, the shown code will produce this output:

2. Structured Composers : Composing from Data Structures

In many cases a simple concatenation of strings is not sufficient. We often need to order the strings in some way as they come and then concatenate them in their current order, which can also change over time. One way to order items is by using simple data structures like lists, stacks, queues, and priority queues. A set of structured text composers can be used for this task. All structured text composers can be found under the TextComposerLib.Text.Structured namespace.

Structured Composers

One such structured text composer is called ListComposer.  This composer is a list that can be populated with items like any .NET list structure. At any time the items in the list can be used to generate concatenated text from the individual items. Each item can have arbitrary prefix and\or suffix strings independent of the other items. The concatenated items can be separated by an arbitrary string in the final composed text. The final composed text can have arbitrary prefix and\or suffix strings of its own.

The same idea behind ListComposer is implemented in 3 other structured composers based on stacks, queues, and priority queues as shown in the class diagram. All structured composers implement the common IStructuredComposer interface that specifies the separator string, the final prefix and suffix strings, and the active prefix and suffix item strings that are automatically inserted when adding new items. Structured composers are data structures containing  StructuredTextItem objects that can be populated with such items like expected, and at any time can be used to generate concatenated text from the stored items just like ListComposer. All structured composers have extension methods similar to the  ComposeToList method of the ListComposer that can be found under the StructuredUtils static class.

One final structured composer is the SequenceComposer that can successively append IEnumerable<string> sequences acting as the items to be composed into the final text. The main difference between this composer and the others is the use of the ActiveItemPrefix and ActiveItemSuffix properties. In the SequenceComposer items prefix and suffix strings are common among all items whereas in the other composers all items added before changing the ActiveItemPrefix and ActiveItemSuffix properties of the composer will retain their prefix and suffix strings because they are stored with the item in a  StructuredTextItem object.

Example40

To generate the string  {'1', '2', '3', '4', '5'} we can use the list composer as in this C# code snippet:

We can also use the extension method  ComposeToList to reduce the required code:

Example40

To show how structured text that varies over time can be generated, the shown code can be used to compose this multi-line string:

The first elements 1, 2, and 3 are added to a stack composer using the ComposeToStack method with active item prefix and suffix strings set to a single quote '. Then the other items a, b, and c are added after changing the active item prefix and suffix strings into <  and >. Inside the while loop, the current state of the stack is concatenated into a string and added as an item to a list composer and the stack is popped.

3. MappingComposer : Composing by Transforming

The mapping composer, under the TextComposerLib.Text.Mapped namespace, works by dividing its input string into segments of text, stored in the MappingComposerSegment class. The segmentation is based on specially marked text inside the string. Then a transformation function or lookup dictionary can be used to transform the segments’ text into new text. The final composed text is re-assembled again from the transformed text of the segments in the same order of their presence in the original input string. Two methods of marking special text can be used, as can be seen in the SegmentMarkingMethod enumeration:

  • Delimited: Where a pair of fixed left and right string delimiters specify the special text segments. In this case any text inside the two delimiters can be marked without restrictions.
  • Identified: Where a fixed prefix string followed by an identifier name specifies the special text. In this case only legal identifier names are used as marked text. An identifier is any sequence of alpha-numeric characters or underscores not starting with a number.

In the shown class diagram for the MappingComposer we can see several TransformUsing methods, under the MappedUtils static class, that perform the required transformations on the text segments.

MappingComposer

Example40

The best way to explain the operation of this particular composer is by following an example. Let us have the string:

The following code will make a simple substitution to replace @a by firstVar and @b by 5:

And we get the following output text:

Example40

As a more advanced example, let us have the text:

And we wish to transform this into a C# String.Format() method call string, perhaps as part of a code generation process, like this:

In other words, we need to detect all sub-strings delimited by $ $  (like $i$  and $id$  in this example) and replace them by sub-strings of the form {n} where n is an index marking the appearance order of this sub-string ( {0}  for $i$  and {1}  for $id$  in this example). The following code solves this problem:

The SetDelimitedText()  method takes the initial text of the composer and the two delimiters. Inside this method, the mapping composer initially divides the input string into the following unique segments:

Note that the same segment may appear several time in the original text (for example $i$  appears 3 times).

Then the TransformByIndexUsing()  method is called taking a transforming function of type Func<int, string>; it applies the given transformation  index => "{" + index + "}" to each delimited segment (i.e. only the text of the segments $i$  and $id$  are transformed using their order of first appearance). The final text of the composer is then given by reconstructing all the segments in the same order but with replacing delimited segments text by their transformed text as follows:

We then use the initial text of each delimited segment to form a list that is passed into a Concatenate()  method to build the final desired string by concatenating the initial text of the delimited segments (i.e. the two strings "i" and "id"):

4. LinearComposer : Composing Text Line-by-Line

The idea behind the LinearComposer is that we add text line by line and once we are on a new line we can’t modify any of the old lines anymore. This method of composing text is very useful when we are creating code files in a linear fashion that require consistent indentation and formatting. The LinearComposer has many useful methods to ensure control over the addition of new text lines. We can also add several line headers inside this composer to be added each time a new line is created to hold additional automatic text like date and time of line creation, line number, the progress of time as a stopwatch, etc.

LinearComposer

The LinearComposer class has several Append methods each having a specific usefulness:

  • Append : This method adds the given text to the active line of the composer and does not finish the line to go to a new line.
  • AppendLine : This method adds the given text to the active line and finalizes it and creates a new empty line for later appending of new text.
  • AppendNewLine : This method finalizes the current line, adds a new empty line, and then it appends the given text to the new line.
  • AppendAtNewLine : This method tests if the current line is new (i.e. empty), if the line is not new it finalizes the line and creates a new line. It then appends the given text to the new line.
  • AppendLineAtNewLine : Just like AppendAtNewLine but finalizes the new line after appending the given text.

A common feature of all the append methods is that when the given text is actually a multi-line string it is separated automatically into lines and each line is added and finalized except for the last one. This ensures the consistent addition of line headers, especially the indentation line header, as will be seen in the following examples.

An important feature of this composer is that we can easily create a new type of line headers by inheriting from the LcLineHeader class and using the AddLineHeader method of the composer.

Example40

The shown code can produce this block of text lines:

5. ParametricComposer : Composing by Substitution

One very useful text composer in TextComposerLib is the ParametricComposer under the TextComposerLib.Text.Parametric namespace. In this composer we define a text template containing named parameters. Any single parameter can appear several times anywhere in the template. A substitution process is done to replace the parameters by string values to produce the final text. A parameter is implicitly defined using delimiters of arbitrary choice per template. Combined with other composers, we can use the ParametricComposer to generate almost any desired text files with predefined structure.

ParametricComposer

Another related class is the ParametricComposerCollection class that can be used to store and generate text from several templates at once. In addition, the ParametricComposerCollection class can be used to load the full definitions of a set of templates by parsing a text file containing the definitions. A user interface for this operation is present inside the TextComposerLib.Text.Parametric.UI namespace to make designing such templates easy for the user. The main windows form of the interface is the FormTemplateComposerEditor form.

Example40

The shown code will generate this block of multi-line text:

  • The first 4 lines create the ParametricComposer with left and right parameter delimiters < and >; then it sets AlignMultiLineParameterValues = true to ensure correct handling of parameters with multi-line text value.
  • The SetTemplateText method changes the base text of the template and parses all parameters automatically from the given text and delimiters. Here we have the 4 parameters class_name (used 2 times in the template text), base_class_name, constructor_code, and constructor_code.
  • The following lines of code set each parameter’s value to a given string.
  • The final code line generates the text by substituting values for parameters in the original template text.

We can use several alternative methods to assign values to template parameters and generate the desired text. All the shown code blocks result in identical behavior. Note that in code piece 4 we call the GenerateUsing method to assign values to template parameters in the order each parameter first appears in the template. This eliminates the need for specifying the name of each parameter as in the GenerateText and SetParametersValues methods.

Example40

This example give an illustration of integrating several text composers to get the required results so it deserves special attention:

  • After creating the ParametricComposerCollection in the first line of code, a call to the Parse method is done to parse a text containing the definition of two templates: declare_property and declare_class. We will comment on the structure of this text shortly.
  • Next we create the propertiesComposer object (a list text composer) and initialize it with 3 items generated from the declare_property with different parameter values through the GenerateUsing method of the ParametricComposer class. The propertiesComposer object will be used for generating text shortly.
  • We do similar tasks for two other list composers: paramsComposer and codeComposer.
  • Finally we access the declare_class template inside the ParametricComposerCollection and use the 3 list composers to generate values for the 3 parameters: properties, params, and code.

Further investigation of the parsed templates definition text is due. The given templates definition text is shown.

  • The first command in this text must be a delimiters command. Here we define the left and right delimiters used in the following template definitions. We can add a delimiters command before defining a set of templates to change the used delimiters for all following templates. The two delimiters are any series of characters separated by a white space.
  • To define a new template we must use the begin <template name>  and end <template name> pair. the <template name> must match exactly in the begin\ end pair to mark the beginning and end of template text. Each begin and end must be on a separate line and the template text will be any text in between these two lines.
  • We can use C-like single line comments to put comments anywhere outside templates definitions as in the example. If we put a single line comment inside the template text it will be considered a part of the template definition and will be used in the generated text.

6. RegionComposer : Composing by Text Injection

Under the TextComposerLib.Text.Region namespace we find several classes to create and manipulate text templates that can be used for code injection. The idea is to treat text in the template as a list of separate lines and group the lines, using line markers, into two kinds of text regions:

  • Fixed text regions where no change or text injection takes place.
  • Slot regions where we can add or generate the injected text under one or more slot tags. Each slot region has a string, called the tag string, that can guide the generation process.

The text lines inside each slot region is further grouped into tags that can be:

  • Fixed text tags having text lines that remain unchanged in the final text.
  • Slot tags where we can generate text based on a tag string associated with each tag.

So this is a two-level template engine that enable text injection under the slot tags of the slot regions. The classes holding this information are:

  • The RcTemplate class holding the main information of the text template. This class is the top-level of the construction.
  • The RcFixedRegion and RcSlotRegion classes holding information about text regions in the templates. These two classes are the second level of the construction.
  • The RcFixedTag and RcSlotTag classes holding information about tags inside slot regions in the template. This is the third and final level of the construction.

We can construct a template, to be used for text generation later, in these ways:

  • Create a RcTemplate object and use its methods to add fixed and slot regions as desired. Then in each slot region define fixed and slot tags. In the root RcTemplate object define markers for the regions and tags to distinguish their lines from each other.
  • Parse a given text with appropriately marked lines into an RcTemplate object to create its regions and tags from text. This is probably the way we go in code injection and template engines.
  • Inherit from the RcTemplateProcessor class and implement its abstract Visit() methods to manipulate the structure of an RcTemplate object as desired.

Finally the RegionComposer class contains an internal RcTemplate member and several methods to update its structure and generate text inside its slot tags.

RegionComposer Classes and Interfaces

The line markers are arbitrary strings that can be distinguished from each other to mark the meaning of each line. The line markers are:

  • Slot region Begin and End line markers are used to mark slot regions inside the text templates. Any text outside these markers is considered part of a fixed text region.
  • Fixed tag marker is used to mark the beginning of fixed text inside slot regions.
  • Slot tag marker is used to mark a tag with some associated tag string. Any non-marked text immediately after a slot tag marker is replaced during text generation.
  • Begin and End join slot tags markers are used when the slot tag string spans multiple lines them we delimit the string with these two marker lines.

Example40

 

WordPress Appliance - Powered by TurnKey Linux