From: Fabien Potencier Date: Wed, 7 Dec 2011 09:09:34 +0000 (+0100) Subject: merged branch arnaud-lb/str-interpolation (PR #515) X-Git-Url: http://git.silmor.de/gitweb/?a=commitdiff_plain;h=1036f90e79aaff0e948261a4c297f16008c88b32;p=web%2Fkonrad%2Ftwig.git merged branch arnaud-lb/str-interpolation (PR #515) Commits ------- d41d10c simplification 2dbb420 moved hard-coded regexes 61986c1 shortcut for non-interpolated double quoted strings 062ed5c regex optimization 0e63abb integration tests for interpolated strings 773fff5 parser tests for interpolated strings 098634e parser support for interpolated strings c8bba24 lexer tests for interpolated strings a203686 lexer support for interpolated strings 0bd63dd stack lexer states Discussion ---------- String interpolation syntax This adds support for string interpolation: ``` {{ "foo #{ any expression here } bar" }} ``` I find string interpolation to be often more readable than concatenation, and doesn't have the precedence ambiguity of the concatenation operator: ``` Interpolation: {{ "foo #{bar} baz" }} {{ "foo #{1+2} baz" }} {{ "foo #{1+2} baz"|escape }} Concatenation: {{ "foo "~bar~" baz" }} {{ "foo "~(1+2)~" baz" }} extra parentheses {{ ("foo "~(1+2)~" baz")|escape }} more extra parentheses ``` --------------------------------------------------------------------------- by nikic at 2011/11/12 12:41:54 -0800 Really nice idea. We should discuss the exact syntax though :) Dart for example uses `${expr}` for interpolation or `$name` if the expression is just a name. I think it makes sense to have a special short syntax for the name case, because it is the most common one. --------------------------------------------------------------------------- by stof at 2011/11/12 13:06:51 -0800 @nikic could you comment on the PR instead of commenting on each commit individually ? it would make it easier to see what is concerned in the PR. --------------------------------------------------------------------------- by arnaud-lb at 2011/11/12 14:22:14 -0800 @nikic Thanks for your review, I'll take care of your remarks. For the syntax, I have no preference. I've taken this one because it has the same semantics. In Ruby and Coffeescript the `#{...}` allows any expression between the brackets. In PHP the expression in the `${...}` must start with a variable name. As you mentioned Dart's `${...}` seems to allow any expression. --------------------------------------------------------------------------- by nikic at 2011/11/12 15:03:48 -0800 @arnaud-lb If Ruby and Coffeescript use this, the #{...} syntax should be okay. Especially as ${...} might be confusing for PHP developers as it has the semantics of variable variables in PHP. @stof Thanks, will do so next time. --------------------------------------------------------------------------- by damianb at 2011/11/12 15:38:31 -0800 @stof he doesn't need to - this is something github handles already. in the PR view, beside the "@nikic commented" bits, there's a set of links that shows where exactly the comment was in the format of commit sha, filename, and line inside the diff. You can see for yourself right here, upper left: ![pr snapshot](http://i42.tinypic.com/zl6s82.png) --------------------------------------------------------------------------- by stof at 2011/11/13 01:20:38 -0800 @damianb The comment appears, but not the context in which the comment was. We have to follow each link to see it whereas comments done on the PR appear directly with their context. This is why I said it is easier when comments are on the PR. --------------------------------------------------------------------------- by nikic at 2011/12/02 15:13:30 -0800 So what about this now? Btw, `REGEX_STRING` and `REGEX_DQ_STRING_PART` still are hardcoded and do not depend on the specified `interpolation` tags. But this looks hard to change, at least the regexes are quite complicated. --------------------------------------------------------------------------- by arnaud-lb at 2011/12/02 15:22:45 -0800 I think I've addressed all issues (except hardcoding of `#{` and `}` in `REGEX_STRING` and `REGEX_DQ_STRING_PART`.) Can this be merged after v1.4 is release if the syntax is accepted ? --------------------------------------------------------------------------- by nikic at 2011/12/03 01:53:41 -0800 This would be my go for `STRING` and `DQ_STRING_PART` (isn't tested): ```php 1) { $interpolRest = preg_quote(substr($options['interpolation'][0], 1), '/'); $interpolRegex = $interpolPart . '(?:(?:\\\\.|' . $interpolChar . '(?!' . $interpolRest . '))' . $interpolPart . ')*'; } else { $interpolRegex = $interpolPart . '(?:\\\\.' . $interpolPart . ')*'; } $this->options['dq_string_part_regex'] = '/' . $interpolRegex . '/As'; $this->options['string_regex'] = '/"' . $interpolRegex . '"|\'([^\'\\\\]*(?:\\\\.[^\'\\\\]*)*)\'/As'; ``` It should allow both a single char interpolation syntax (like `{`) and multichar ones (like `#{`). By the way, could we maybe move the compiled regexes out of `$this->options` into `$this->regexes`? --- 1036f90e79aaff0e948261a4c297f16008c88b32 diff --cc test/Twig/Tests/LexerTest.php index 220307e,52a3371..729869f --- a/test/Twig/Tests/LexerTest.php +++ b/test/Twig/Tests/LexerTest.php @@@ -139,14 -139,89 +139,100 @@@ class Twig_Tests_LexerTest extends PHPU // should not throw an exception } + public function testBigNumbers() + { + $template = '{{ 922337203685477580700 }}'; + + $lexer = new Twig_Lexer(new Twig_Environment()); + $stream = $lexer->tokenize($template); + $node = $stream->next(); + $node = $stream->next(); + $this->assertEquals(922337203685477580700, $node->getValue()); + } ++ + public function testString() + { + $template = 'foo {{ "bar #{ baz + 1 }" }}'; + + $lexer = new Twig_Lexer(new Twig_Environment()); + $stream = $lexer->tokenize($template); + $stream->expect(Twig_Token::TEXT_TYPE, 'foo '); + $stream->expect(Twig_Token::VAR_START_TYPE); + $stream->expect(Twig_Token::STRING_TYPE, 'bar '); + $stream->expect(Twig_Token::INTERPOLATION_START_TYPE); + $stream->expect(Twig_Token::NAME_TYPE, 'baz'); + $stream->expect(Twig_Token::OPERATOR_TYPE, '+'); + $stream->expect(Twig_Token::NUMBER_TYPE, '1'); + $stream->expect(Twig_Token::INTERPOLATION_END_TYPE); + $stream->expect(Twig_Token::VAR_END_TYPE); + } + + public function testStringWithEscapedInterpolation() + { + $template = '{{ "bar \#{baz+1}" }}'; + + $lexer = new Twig_Lexer(new Twig_Environment()); + $stream = $lexer->tokenize($template); + $stream->expect(Twig_Token::VAR_START_TYPE); + $stream->expect(Twig_Token::STRING_TYPE, 'bar #{baz+1}'); + $stream->expect(Twig_Token::VAR_END_TYPE); + } + + public function testStringWithHash() + { + $template = '{{ "bar # baz" }}'; + + $lexer = new Twig_Lexer(new Twig_Environment()); + $stream = $lexer->tokenize($template); + $stream->expect(Twig_Token::VAR_START_TYPE); + $stream->expect(Twig_Token::STRING_TYPE, 'bar # baz'); + $stream->expect(Twig_Token::VAR_END_TYPE); + } + + /** + * @expectedException Twig_Error_Syntax + * @expectedExceptionMessage Unclosed """ + */ + public function testStringWithUnterminatedInterpolation() + { + $template = '{{ "bar #{x" }}'; + + $lexer = new Twig_Lexer(new Twig_Environment()); + $stream = $lexer->tokenize($template); + } + + public function testStringWithNestedInterpolations() + { + $template = '{{ "bar #{ "foo#{bar}" }" }}'; + + $lexer = new Twig_Lexer(new Twig_Environment()); + $stream = $lexer->tokenize($template); + $stream->expect(Twig_Token::VAR_START_TYPE); + $stream->expect(Twig_Token::STRING_TYPE, 'bar '); + $stream->expect(Twig_Token::INTERPOLATION_START_TYPE); + $stream->expect(Twig_Token::STRING_TYPE, 'foo'); + $stream->expect(Twig_Token::INTERPOLATION_START_TYPE); + $stream->expect(Twig_Token::NAME_TYPE, 'bar'); + $stream->expect(Twig_Token::INTERPOLATION_END_TYPE); + $stream->expect(Twig_Token::INTERPOLATION_END_TYPE); + $stream->expect(Twig_Token::VAR_END_TYPE); + } + + public function testStringWithNestedInterpolationsInBlock() + { + $template = '{% foo "bar #{ "foo#{bar}" }" %}'; + + $lexer = new Twig_Lexer(new Twig_Environment()); + $stream = $lexer->tokenize($template); + $stream->expect(Twig_Token::BLOCK_START_TYPE); + $stream->expect(Twig_Token::NAME_TYPE, 'foo'); + $stream->expect(Twig_Token::STRING_TYPE, 'bar '); + $stream->expect(Twig_Token::INTERPOLATION_START_TYPE); + $stream->expect(Twig_Token::STRING_TYPE, 'foo'); + $stream->expect(Twig_Token::INTERPOLATION_START_TYPE); + $stream->expect(Twig_Token::NAME_TYPE, 'bar'); + $stream->expect(Twig_Token::INTERPOLATION_END_TYPE); + $stream->expect(Twig_Token::INTERPOLATION_END_TYPE); + $stream->expect(Twig_Token::BLOCK_END_TYPE); + } }