sql-server-samples/samples/manage/sql-assessment-api/docs/Reference/DataTransformation/parse.md

Parse

The regular expression parser extracts data from substrings of variable values that correspond to the named groups specified in regular expressions.

Parameters

Parameter	Required	Type	Description
map	Required	Map	Sets regular expressions for variables.
flatten	Optional	bool	Enables combining fields from subsequent rows.
join	Optional	string	Enables joining of strings from all rows and sets the separator.

Map

Map is a JSON object setting regular expression. Each map property defines a regular expression for a variable of the same name. Named groups in regular expressions define substrings to be returned as new variables. A new variable name consists of original.group where original is the name of the parsed variable and group is the regular expression group name.

When a regular expression has more than one match, a new row is generated for every match. To pack partial matches into a single row, use flattening or joining.

NOTE: The Explicit captures option improves the morph memory footprint and performance. Use 'x' after the closing slash to enable this option: /(?<int>\d+)/x.

The following examples demonstrate transformation results:

Example 1

Input:

@a	@b
"There are 12 chairs"	"The chairs are blue"
"Found 5 files"	"Red status"
"Number of Napoleons: 6"	"Black pearl"

Morph:

{
    "type": "parse",
    "map" : {
        "a": "/(?<amount>\\d+)(\\s*(?<kind>\\w+?)s?\\b)?/x",
        "b": "/(?<color>red|green|blue|yellow)/ix"
    }
}

Output:

@a	@a.amount	@a.kind	@b	@b.color
"There are 12 chairs"	"12"	"chair"	"The chairs are blue"	"blue"
"Found 5 files"	"5"	"file"	"Red status"	"Red"
"Number of Napoleons: 6"	"15"	-	"Black pearl"	–

Example 2

Input:

@a	@b
"12 chairs and 1 priest"	"The chairs are blue"
"40 pages in 5 files"	"Red status"
"Number of Napoleons: 6"	"Black pearl"

Morph:

{
    "type": "parse",
    "map" : {
        "a": "/(?<amount>\\d+)(\\s*(?<kind>\\w+?)s?\b)?/x",
        "b": "/(?<color>red|green|blue|yellow)/ix"
    }
}

Output:

@a	@a.amount	@a.kind	@b	@b.color
"12 chairs and 1 priest"	"12"	"chair"	"The chairs are blue"	"blue"
"12 chairs and 1 priest"	"1"	"priest"	"The chairs are blue"	"blue"
"40 pages in 5 files"	"40"	"page"	"Red status"	"Red"
"40 pages in 5 files"	"5"	"file"	"Red status"	"Red"
"Number of Napoleons: 6"	"6"	–	"Black pearl"	-

Flattening

Flattening enables combining fields from sequential rows into one in cases when a data row occupies more than one line.

Flattening works only for fields generated by the parsing process; other data is removed.

Example 3

Raw output data:

@a.f1	@a.f2	@a.f3	@other
11	-	-	"extra"
-	-	13	-
-	12	-	-
21	22	23	"data"
-	32	33	"will"
31	-	-	-
41	-	-	"be"
-	42	-	-
51	-	-	"ignored"

After flattening:

@a.f1	@a.f2	@a.f3	@other
11	12	13	-
21	22	23	-
31	32	33	-
41	42	-	-
51	-	-	-

Joining strings

This option joins all rows into one. Such a new row contains string equivalents of field values joined with the specified delimiter.

Joining works only for variables mentioned in Map; other data is removed.

In the following example, join = ", ".

Input:

@a	@b	@c	@notMentioned
42	"Lorem"	-	"extra"
13	"ipsum"	true	"data"
0	"dolor"	false	"ignored"

Joined:

@a	@b	@c	@notMentioned
"42, 13, 0"	"Lorem, ipsum, dolor"	"true, false"	-