Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
mondo-commit since MOE is refusing to migrate individual changes.
There's quite a bit of flux in this CL due not just to new features.
Google is now using https://github.com/google/google-java-format
internally to format most of its java sourcebase so that it's easier
for a developer to quickly familiarize and produce fixes for code
outside their main project. This should be a one-time flux.
Change: 131778022
Description:
Fix proto3 oneof semantics in Soy to match jspb semantics which are changing (in cl/129271686)
Due to a bug in jspb, primitive oneof fields that weren't set would return undefined/null even in proto3. Since this is being fixed in jspb we need to ensure that soysauce does the right thing.
Change: 131859311
Description:
Rename readme so it will appear properly in CS window.
Change: 132014466
Description:
Migrate and update some misc Soy docs.
Change: 132069248
Description:
Make the error for passing a protobuf to a template with untyped params a little less confusing. 'strict' typically refers to autoescaping, but here it was referring to 'strictly typed'
Change: 132074373
Description:
Sort SoyErrors before reporting them.
This should ensure that errors are grouped by file and are reported in lexicographic order within each file.
Change: 132133045
Description:
Add a public target for per-locale compilation users to use.
soy_js doesn't support generating per-locale outputs and it doesn't make much sense to add support since afaict there are only 4 such users and two of them aren't real applications (tests and/or examples). So this new public SoyToJsSrcCompiler target is available for them, but controlled by a whitelist.
Change: 132333434
Description:
Delete the ExprNode.ConstantNode interface.
There are no subtypes of ConstantNode which aren't also subtypes of PrimtiveNode so there isn't much point in the interface existing.
Change: 132358507
Description:
Move subcategory messages into their own files.
This change has two major components.
1. It moves subcategory messages into their own files under the subdirectory templates/
2. It adds a new Python binary (findings_server.py) which serves a web page which can be used to view rendered finding messages in both HTML and plain text. e.g. https://screenshot.googleplex.com/M98w3smAz50
The change also makes minor changes to message template content to improve readability.
Change: 132366151
Description:
Fixed typos and removed a dead link.
Change: 132383088
Description:
Soy parser: Add SourceLocation to null token
Change: 132384045
Description:
Trim the SoyToJsSrcCompiler whitelist
Change: 132387350
Description:
Fix a few bugs in the type parser
* report errors to the error reporter (and return ErrorType) instead of throwing LegacySoySyntaxException
* change the token definitions for list/maps declaration to include the '<' so that they don't overlap with the IDENT token
* migrate the errors test to use the compilation test framework
* minor improvements to error messages
Change: 132442292
Description:
Add better handling for the ErrorType throughout the compiler.
When the compiler comes across an expression that doesn't make sense (references an unknown type, references an unknown field on a proto,...) we assign the type of that expression the special 'error' type. However, we generally don't account for the error type when operating on types so this can lead to spurious and misleading error reports.
For example, this template
{template .foo}
{@param list : list<some.Proto>}
{$list['a'].someField}
{/template}
priot to this cl would report 2 errors.
1. bad index type string for list<some.Proto>
2. 'ErrorType' does not support dot access
The first one is legitimate, but the second one is spurious due to the fact that we assign the expression '$list['a']' the type 'ERROR' and then the field access analysis doesn't take that into account.
To account for this we need to modify the way we handle and assign types to expressions in error situations.
For example,
* when dereferencing a list/map, we can always assign the type of list/map to
the access expression, even if the access is erroneous
* when dereferencing an 'error' type, we should assign the type 'error' to the expression
* when calculating arithmetic operations on an error type, the resulting type should be the error type
* when taking the union of anything with the error type, the result should be an error type.
This CL seeks to fix a number of these issues by adding specific checks for the
error type and better test coverage.
Change: 132446441
Description:
Improve the performance of constructing a SoyProtoTypeProvider
due to the way we compile file descriptors it is possible for the same filedescriptor to end up in multiple filedescriptorsets, so I added some memoization to avoid walking the same common protos over and over again.
Also i simplified the DescriptorTreeWalker api to eliminate some unused callback functions and list copies.
Change: 132446506
Description:
add a few more entries to the _for_per_locale_compilation whitelist
Change: 132461639
Description:
Fix a bug in the type resolver which would throw an internal error when
calculating type constraints for expressions with the null type.
The particular case i ran into was a team using globals for conditional
compilation.
{if DEBUG}...{/if}
depending on configuration this global would take on one of several values, but
when disabled would be assigned the value 'null'. When calculating type
constraints we would throw an exception when setting the constraint that DEBUG
would be non-null within the scope of the if statement (since DEBUG can't be
null and non-null). The fix is just to not set any constraint on DEBUG in this
situation.
Change: 132466821
Description:
Remove whitelist for Soy Python backend. It's pretty vestigial now that the backend has been stable for a while.
Change: 132501709
Description:
Improve error message formatting
* drop the 'In file' prefix
* add 'error:' immediately prior to the error message
* add a count of errors!
Also, eliminate a redundant ErrorReporter implementation
Change: 132585709
Description:
Soy: eliminate grammar lookahead rules for globals.
Currently, the Soy expression grammar allows dot-separated sequences of identifiers in two contexts: field accesses ($foo.bar.baz) and globals (foo.bar.baz). Even though the $ is sufficient to disambiguate the two forms, the expression grammar uses an awkward lookahead rule to disambiguate in a different way. This lookahead makes it hard to introduce new contexts where dot-separated identifiers are valid.
This CL eliminates the lookahead rule, relying on the $ to disambiguate field accesses and globals. However, it doesn't require changes to the rest of the compiler. The expression parser now propagates global nodes up a chain of field access nodes, so that by the time parsing is complete, global nodes contain dot-delimited string values as they currently do.
Change: 132642339
Description:
Generate @typedef for opt_data.
This can be used in the JavaScript code.
Change: 132689857
Description:
Change the way we track globals in the compiler to enable better type information
Now for resolved globals we can assign proper types instead of just '?'. Also for globals that resolve to proto enums we can track the enum type.
Specifically
* move globals resolution into the ResolveExpressionTypesVisitor (we could also put it in a dedicated earlier pass)
* change the current substitution pass to just validate that everything was substituted based on pass configuration.
* allow enums to be assigned to int values. Now that we actually have enum types at runtime this is important
It is possible that we will need/want to add custom int->enum constructors in the future, but i don't think it is necessary right now.
Change: 132903356
Description:
Soy: Alias are resolved in RewriteGlobalsPass
Can't resolve aliases in the expression parser itself, because it will shortly
be ambiguous whether <IDENT> (<DOT_IDENT>)* will be a global or a function
Change: 132907161
Description:
Soy: Dots are allowed in function names
Needed for proto instantiation within Soy, which will use the fully qualified
name of the proto as the instantiation function.
Fix indentation in PrecExpr8() and PrecExpr9().
Change: 133141295
Description:
Soy: List and map literals now have source locations
Also extend function source location to last arg
Change: 133142696
Description:
Soy: Move FieldAccess(), ItemAccess(), FunctionCall() to own methods in ExpressionParser
Simplify PrecExpr9().
Change: 133148020
Description:
When reporting expression parsing errors take the errorToken into account for source location.
We are still off a bit, but now it is entirely due to the parentSourceLocation
being wrong.
Change: 133268984
Description:
Add referenced enums to the ParseInfo descriptors so that they are available to
the compiler for development mode compilations.
Change: 133327858
Description:
Soy: Integration tests for duplicate keys in map literal
Duplicate string literals will be caught, but not duplicate vars or exprs.
Change: 133398774
Description:
Delete the SoyToJsSrcCompilerForGoogle
It is finally dead
Change: 133401939
Description:
Soy: Test that list/map literals from proto optional fields are correctly typed
Change: 133422191
Description:
Fix a bad link in the soy documentation
Change: 133426183
Description:
Soy: New ExprNode to represent proto instantiation
A mostly-copy of FunctionNode at this point. New fields to be added as I figure
out what information is necessary.
Change: 133447401
Description:
Soy: Implement named expression list
To be used inside functions, for proto instantiation.
Change: 133616449
Description:
Enhance the globals generator to show the source of each global in a comment.
This should help identify projects using this when they could be adding proto dependencies.
Change: 133659809
Description:
Soy: Add "new" keyword for proto initialization
"new" is now a reserved keyword. It is no longer a valid global, and attempting
to use it as such will result in a parser error. (A better error message would
probably require a lookahead.)
This allows us to disambiguate between functions and proto initializations and
makes proto initializations unique, which will be especially helpful once proto
name aliasing gets implemented.
We no longer use the presence or absence of dots to distinguish between
functions and proto initializations:
* this.is.a.function()
* new thisIsAProtoInit()
Named params are still not allowed in functions, and non-named params (expr
lists) are still not allowed in proto initializations.
also moved some tests over to the integration suite.
Change: 133727794
Description:
Whitelist jbcsrc generated code in fences.xml.
This gap was exposed by the closure-maven-plugin demo and obviates
https://github.com/mikesamuel/closure-maven-plugin/blob/8ae44311d33c346fb068c834c9f026629911d6ef/plugin/src/it/demo/pom.xml#L144-L154
Change: 133746726
Description:
Adding a converter from SanitizedHtml.
Change: 133750646
Description:
Don't run the optimizer passes for parseinfo generation.
Since we don't generate code, just metadata for this compilation setting it is
wasteful (in time) to run the tree simplifier, but it turns out that it also
revmoves data from the parseinfos.
For example, consider this soy snippet.
{template.foo}
{foo.bar.Baz.ONE}
{/template}
where foo.bar.Baz is a proto enum. The optimizer will rewrite this to be
{template .foo}
1
{/template}
so when the parseinfo generator runs, we won't notice that a proto enum was
used for this file and so won't add the descriptor to the SoyFileInfo object.
By skipping optimization we can ensure that this data is preserved.
references to parseinfos if the javac build action isn't configured with the
proto gencode for those protos.
Change: 133757192
Description:
Soy: Run build_cleaner on everything
Also fix a couple of tests file that were importing using *.
Change: 133760334
Description:
Soy: Remove .* imports from SoyFileParser
Change: 133770891
Description:
Remove the GeneratedDescriptorPool fallback from the SoyProtoTypeProvider
This should improve performance and bring the google only behavior to be closer to what open source is using.
Change: 133773158
Description:
Soy: Add access to proto descriptor in SoyProtoType
We need the descriptor object to do proto initialization.
SoyProtoType now inherits from SoyType.
Change: 133845471
Description:
Soy: ProtoInitNode returns SoyProtoType from getType()
Also fixed primitive nodes to return their own types.
Change: 133850270
Description:
Soy: Add type information to ProtoInitNode
Change: 133857337
Description:
Soy: Remove unused SoyFileSetParserBuilder option
Change: 133860937
Description:
Delete package-info.java from the data/ordainers package
Change: 133877812
Description:
Short-term fix for an out-of-date tutorial on soy.
Eventually, it looks like this should be rewritten to use the soy_js build macro.
Change: 133884937
Description:
Soy: Refactor SoyProtoTypeImpl.Field and implementations into separate files
Change: 133899813
Description:
Soy: Allow trailing comma in expression parser's named param list
Added comments to expression parser
Change: 133901691
Description:
Soy: Integration tests for "Did you mean..." errors
Change: 133995574
Description:
Soy: Check validity of proto init calls
New default pass, CheckProtoInitCallsPass, checks that:
* all required proto fields are present
* all given params are of the correct type
Change: 134007431
Description:
Corrected Soy compiler command line options.
Change: 134007883
Description:
Small fix to soy jbcsrc g3doc, plus mdformat
Change: 134016355
Description:
More fixes to soy jbcsrc README.md
Change: 134019912
Description:
Soy: Proto init call accepts null as a param value
In jspb,
proto.setField(null);
clears the field, so we will implement that as the default behavior across all
backends.
Change: 134082913
Description:
Stop using assisted inject in SoyFileSet
This should make things a _little_ easier to understand
Change: 134143945
Description:
Generate @typedef for opt_data, take 2.
This can be used in the JavaScript code.
Compared to the first attempt, this keeps the type of opt_data inlined and only adds the @typedef for use in .js, it's not used in .soy.js. This has two reasons:
1. There's probably a bug in JS Compiler somewhere which causes renaming keys of two different objects to the same name causing conflicts (see http://yaqs/eng/q/6019533105528832 for more info).
2. The error message if the types don't check before the first attempt was:
found: {
a: string|undefined
}
required: {
a: string
}
After the first attempt, it was:
found: {
a: string|undefined
}
required: ns.template.Params
This is way less useful as you need to go to the generated code to see what the type really is. This second attempt maintains the original error message.
Change: 134297320
Description:
Improve the BytecodeProducer error message for interleaving code gen and statement construction.
Change: 134297576
Description:
Replace calls to 'new SoyFileSet.Builder()' with SoyFileSet.builder()
The former has been deprecated for over 2 years and is going away.
NOTE: This is not necessarily a safe change, the former uses static injection to try to identify custom soy plugins and type registries, whereas the regular builder just configures the default set of soy plugins. If you actually want to register soy plugins, just install SoyModule and inject SoyFileSet.Builder instead of constructing it directly.
Change: 134327090
Description:
Soy jbcsrc: ProtoUtil.getBuilderMethod()
Utility method for creating a SoyExpression that calls
ProtoMessage.newBuilder().
Sample output SoyExpression:
SoyExpression(Lcom/google/template/soy/jbcsrc/Extendable$Builder;){non-null}<Lcom/google/template/soy/jbcsrc/Extendable$Builder;>:
INVOKESTATIC com/google/template/soy/jbcsrc/Extendable.newBuilder ()Lcom/google/template/soy/jbcsrc/Extendable$Builder;
Change: 134339994
Description:
Improve the UnexpectionCompilerFailureException to include the sequence of nodes in the message so they are displayed better in unit tests
In production we coerce it to an error report in BytecodeCompiler, but most
tests don't use that api
Change: 134443862
Description:
Soy: Consolidate resolveSoyValueProvider() methods
Plus misc. cleanups
Change: 134473961
Description:
Soy jbcsrc: Clean up comments on TemplateVariableManager and ExpressionCompiler
Change: 134568663
Description:
Soy: SoyProtoTypeImpl#getFieldAccessExpr should not return null.
The interface definition (SoyObjectType#getFieldAccessExpr) is not @Nullable. The null return has existed since the first revision of SoyProtoTypeImpl, but there appears to be no reason for it. Accesses on non-existing fields (proto.someMadeUpField) are already caught by ResolveExpressionTypesVisitor.
Change: 134578216
Description:
Improve docs on the Expression contract and add TODOs to two known violations
Change: 134583124
Description:
Soy: move JS protobuf codegen test from SoyFileSetTest into jssrc.
I'm going to add more tests, but SoyFileSetTest doesn't seem like the right place.
Change: 134595026
Description:
Soy jbcsrc: Cleanup and preparation for ProtoUtil.createProto()
Change: 134686946
Description:
Soy tests: c.g.c.truth.Subject#getSubject() => actual().
Truth 0.3 deprecated getSubject() and getDisplaySubject() in favor of the clearer-named (but identically-behaving) actual() and actualAsString() respectively. (See https://github.com/google/truth/releases/tag/release_0_30.)
Change: 134690448
Description:
Soy jbcsrc: Rename expression.cast() to Expression.checkedCast()
Change: 134691583
Description:
Soy JS tests: introduce custom Truth subject and use to improve proto codegen coverage.
Change: 134692136
Description:
Initial step of adding comments for proto file descriptors.
Change: 134701442
Description:
Add support for proto initialization expression to GenerateParseInfoVisitor
This ensures that protos referenced by proto init expressions are added to the
SoyFileInfo objects and then registered into the type registry for server side
renders.
Change: 134702190
Description:
Soy jbcsrc: Implement proto initialization call, first pass
Repeated field handling is not yet implemented.
Extension handling is not yet implemented.
Change: 134731059
Description:
Tweak DelTemplateSelector to make eclipse happy
I don't know why exactly but the eclipse compiler doesn't like this, but it
is easy enough to fix.
Change: 134792946
Description:
Make soy strict autoescape presubmit check support multiline.
Change: 134793604
Description:
Fix executable file name.
Change: 134795721
Description:
Remove TemplateParseResult and the corresponding SoyFileParser entry point
Change: 134805162
Description:
Add SoyProtoValue.Builder a new api to be used by the tofu backend.
Some features are missing (notably safehtml support) but this should be enough
to get started in tofu.
Change: 134837194
Description:
Add proto initialization support to Tofu and add the barest amount of support to the integration tests.
I switched the jbcsrc/tofu compilation to use af_soy_library since it already has good support for protos and added a simple test to construct a proto. The tofu implementation mostly just delegates to SoyProtoValue.Builder
Change: 134993085
Description:
Make the SoyFileSet.Builder constructor package private.
Change: 135002874
Description:
Soy jbcsrc: Add expected error functionality to TemplateTester
Change: 135014398
Description:
Soy jbcsrc: Rename SoyExpression.soyType to SoyExpression.soyRuntimeType.
It's a SoyRuntimeType, which is neither a SoyType nor a RuntimeType.
Also move internal usage of soyRuntimeType() to use the field directly.
Change: 135032750
Description:
Soy: GenJsCodeVisitor: simplify misleading code path.
coerceTypeForSwitchComparison is called only from visitSwitchNode (line 1123):
```
String switchExpr = coerceTypeForSwitchComparison(node.getExpr(), node.getExprText());
```
SwitchNode#getExpr can never return null, so the if (v2Expr != null) check in the method body is pointless.
Change: 135041886
Description:
Add an integration test to demonstrate how line joining works.
Change: 135045895
Description:
Revert the changes that append comments.
Change: 135122890
Description:
add escaping directives for csp_nonce ij variables.
Change: 135131545
Description:
Soy jbcsrc: BytecodeUtil.unboxUnchecked(), refactor ProtoInitGenerator
BytecodeUtil.unboxUnchecked() calls the appropriate unboxing operation on
whatever is on the top of the stack at the moment. This allows us to insert
null checks before unboxing.
Plus misc. cleanups, comment fixes, etc.
Change: 135135594
Description:
Soy jbcsrc: Integration test for initializing protos with zeroed-out fields
Change: 135135873
Description:
Simplify TemplateHeader parsing.
* turn BLOCK_DOC_COMMENTS into SPECIAL_TOKENs and only interpret them if they
preceed a param declaration. Previously we supported both before and after,
but that adds some (IMHO) unnecessary complexity.
* turn BLOCK_NON_DOC comments into SKIP tokens (And stop expecting them)
* ignore whitespace in {@param ...} tags
* resolve choice conflict warnings caused by how the TemplateHeader rule expects whitespace
This paves the way to inline the TypeParser and also for extracting some of the
line joining logic..
The main change in behavior is how we associate doc comments (i.e. /**...*/ comments) with {@param } declarations. We now only do the association if the comment precedes the declaration instead of allowing it to both precede and succeed.
Change: 135181863
Description:
Extract the line joining algorithm out of the parser.
Change: 135243808
Description:
Fix a bug where if a user wrote a {msg} with no content and a genders='...'
attribute, the compiler would fail with an IndexOutOfBoundsException.
Change: 135255911
Description:
Soy jbcsrc: Refactor ProtoInitGenerator.handleNormalField()
Collapse the maybe-nullable-arg case into a single Statement. No more
Expression that jumps out of itself!
Change: 135279125
Description:
Soy jbcsrc: Clean up jstype=STRING handling in ProtoUtils
FieldDescriptor with JavaType=INT fall into
int32/fixed32/uint32/sint32/sfixed32, and will never trigger
Protos.hasJsType().
Change: 135284552
Description:
Soy jbcsrc: Handle proto fields with jstype = STRING
Proto fields that are typed as int64 with jstype = STRING annotation must be
handled as strings in Soy code, then converted to int64 for proto init calls.
Change: 135302624
Description:
Move the TypeParser into SoyFileParser.
This has the following benefits:
* source locations for parser errors in types are now accurate (see the type_parser.errs file changes)
* we no longer have to use a single token to detect the beginning/end of a type expression.
* it should improve performance.
* unlocks some potential simplifications of TemplateNodeBuilder (e.g. we should be able to delete DeclInfo altogether)
It also establishes some patterns for moving the ExpressionParser into the SoyFileParser file.
Change: 135361481
Description:
Accept goog.html types as sanitized types.
Also support goog.Uri as {uri}.
Change: 135374434
Description:
Remove TemplateNodeBuilder.DeclInfo and just have the parser construct HeaderParams directly
afaict, the main point of this class was to track the data needed for the
HeaderParam constructor and to invoke the TypeParser. Now that the TypeParser
is merged into the file parser, this is no longer necessary.
Change: 135403037
Description:
Soy jbcsrc: Proto initialization can handle repeated fields
Change: 135432141
Description:
Add type AST node classes and use them in the parser & SoyTypeRegistry
This eliminates one use of SourceItemInfo<T> in SoyFileParser and allows us to move a bunch of error logic out of SoyFileParser. Having a tree structure with names and source locations will also enable better tooling support in the future (e.g. grok). This cl takes some advantage for better error messages.
patched and heavily modified from cl/113663259 by slaks@
Change: 135498830
Description:
Improve ContentSecurityPass escaping to ensure that nonce values are valid base64.
The regex matches (afaict) the w3.org spec from
https://www.w3.org/TR/CSP2/#nonce_value though i would appreciate a second look
since im not entirely sure i followed the token spec syntax on that page.
Also remove nonce comment injection from inline event handlers. There is no standard or non-standard support for this syntax and rather than ensure that malicious nonces can't escape the comments, we will just not generate the comments.
Change: 135503925
Description:
Soy jbcsrc: Refactor proto init handleRepeated()
Part out everything-but-the-null-check to its own method.
Change: 135523888
Description:
Delete VarNode
This is largely redundant with VarRefNode and we can easily reuse VarRefNode
for it.
minor cleanup in preparation for larger expr parser changes
Change: 135698711
Description:
Soy proto: Integration tests for soy proto init
Change: 135706300
Description:
Soy jbcsrc: Remove MethodRef.forMethod(), migrate usage
Method is identical to MethodRef.create()
Change: 135706308
Description:
Add a way to run the compilation integration test that automatically updates
all the files.
Also fix a bug where we were outputting error coverage data for test failures.
It turns out that using TestWatcher as a @ClassRule doesn't work that well.
(succeeded really just means 'finished' rather than 'not failed' when run as a
classrule).
Change: 135707983
Description:
Change our token definitions in the ExpressionParser to be a little simpler.
* instead of having a token for each operator precedence level, just create one token for each operator. This has a side effect of improving error messages
* refactor our primitive literals a little bit
In cases where the operator token is the same as a SoyFileParser token, i used the name from SoyFileParser (e.g. <LANGLE> for '<' instead of <LT>)
Change: 135708824
Description:
Soy: Fix typos
Change: 135718603
Description:
Soy: SanitizedContent.to*Proto() calls
Needed for converting from SanitizedContent to corresponding proto fields, for
in-soy proto instantiation.
Change: 135721035
Description:
Soy jbcsrc: Proto initialization embedded message handling
Includes special handling for SafeHtml and friends.
Change: 135733470
Description:
Introduce HtmlOpenTagNode and HtmlCloseTagNode.
These will be the basis of our validation and parsing passes to come. They
unfortunately conflict (in name) with the existing nodes in the .html package
used by incremental dom. So i have renamed those nodes to have Incremental in
the name to hopefully reduce confusion.
The reason we can't just use the nodes in that package is for a few reasons.
* They require that all nodes and html attributes have static names.
* The tree structure of those nodes is a little difficult for us to work with,
e.g. HtmlStartOpenTagNode, HtmlEndOpenTagNode is designed so that attributes
are siblings to the open/end parts of the tags instead of children.
Also, currently the way in which those nodes are constructed is as an optional
pass immediately priot to invoking the incremental dom backend. In the long
run we should be able to merge node types by both making the new nodes in the
soytree package more flexible and by changing the incrementaldom code
generator.
Change: 135739465
Description:
Refactor collection literal grammar to use lookahead more efficiently
* 'left factor' List and Map literals into the same production which eliminates one of the lookaheads
* use a custom semantic lookahead to match the trailing commas (https://javacc.java.net/doc/lookahead.html) This should be more efficient because fewer tokens will be read prior to maing a decision
* extract a few more tokens into named constants.
* fix a bug in the map key lookahead for reporting errors for using 'bare' identifiers as map literal keys.
Change: 135821537
Description:
Soy: Move enum set of "string" soy type kinds to SoyType.Kind
Clean up TODO
Change: 135836442
Description:
Make Function/Global nodes Primary() productions again.
* add a tiny helper class Identifier and extracted a share production for dotted identifiers
* left-factor globals and function nodes to avoid lookahead
A bunch of the diffs in the .err files is due to reordering tokens in the
grammar (in particular LPAREN vs. field-access). We could make this more
stable by sorting the tokens... worth it?
Change: 135944712
Description:
Add a pass to 'desugar' the Html nodes into raw text (et. al.) nodes
This code is essentially dead since the parser isn't producing these yet, but
we have a test!
Change: 135979362
Description:
Soy proto init: Allow unknown types in field args
Change: 136054213
Description:
Eliminate the DOT_IDENT token in the ExpressionParser
This is the last (i think) refactoring (in the expressionparser) that needs to be done prior to inlining this into the SoyFileParser.
Also, add better sourceloctions for field/item access.
The one tricky part of this was that by making <DOT> an operator it made $ij a token which required some refactoring in our VarRef production.
Change: 136216058
Description:
Change the implementation of the allowUnknownFunctions PassManager option to run the pass, but just not issue 'unknown function' errors.
This will ensure that the pass does run on all known functions. This is just a
minor refactor.
Change: 136386930
Description:
Add stricthtml syntax to template and namespace. This is currently a noop.
Change: 136502109
Description:
Add StrictHtmlValidationPass that does basic validations.
This pass reports an error if one of the following is true
- enableExperimentalFeatures flag does not contain stricthtml
- stricthtml is used with non-strict autoescape mode
- stricthtml is used with non-html template
Change: 136539855
Description:
Fix a few issues in the new nodes and the desugaring pass
* It turns out we can't currently enforce restrictions on the structure of the node children without additional parser work. So downgrade some invariants to TODOs
* Fix a bug in the desugarhtmlnodespass where I had accidentally overridden 'visit(SoyNode)' instead of 'visitSoyNode(SoyNode)'. As a followup I'm going to rename visit->accept which is more accurate. (ideally it would just be final, but that is difficult currently).
* Stop inserting spaces in the attributes when desugaring. Once the parser is modified to parse attributes more specifically, this may get re-added.
* make the tag nodes implement the BlockNode marker interface. BlockNodes are intended to be nodes that have 'Standalone' children (commands or raw text). So these qualify.
Change: 136623810
Description:
Soy proto init: Simple Caliper benchmark for ExpressionParser
Change: 136630260
Description:
Fix a bug in our 'reportErrorAndSkipTo' the current version could result in an infinite loop while scanning for end tokens.
Change: 136632770
Description:
Soy jbcsrc: BytecodeUtils.doIsAssignableFrom & other fixes
doIsAssignableFrom() is supposed to return failOpen if a Class object could not
be found for either of the types.
Fixed downstream bug in ProtoUtils handleRepeated, where an expression
that returned SoyValue was incorrectly being represented by a SoyExpression
with type of SoyList.
Fixed bug in Expression.checkAssignableTo():
a.checkAssignableTo(b)
should call
isPossiblyAssignableFrom(b, a)
Fixed SharedRuntime.equal() to take in (Object, Object) instead of
(SoyValue, SoyValue), to handle SoyExpression.NULL which is typed
Type.getType(Object).
AppendableExpression.forStringBuilder() now returns an AppendableExpression typed
ADVISING_BUILDER_TYPE, which is more specific than the previous
ADVISING_APPENDABLE_TYPE and is necessary in order to call
ADVISING_STRING_BUILDER_GET_AND_CLEAR on the expression.
Added a needed cast in ProtoInitGenerator.handleRepeatedNotNull, in order to
convert the result of
List.get(i)
to SoyValueProvider.
Change: 136635550
Description:
Soy: add DSL for better control of JS gencode.
Currently, Soy assumes that every Soy expression generates a single JavaScript expression. This keeps the codegen API simple (it's just string concatenation) but, fundamentally, it is inappropriate. For example, if the codegen wants to declare a temporary variable for evaluating something conditionally, it is forced to do it inline with an expression.
Meanwhile, the JS protocol buffer API makes extensive use of non-expression statements. Setters are void, so the JS equivalent of
```java
Foo.newBuilder().setBar(1).build();
```
is
```js
const foo = new Foo();
foo.setBar(1);
```
In order to support instantiating protos from Soy-generated JS, we need to change the JS codegen API to allow Soy expressions to translate into sequences of JS statements, not just a single expression.
This CL introduces a CodeChunk DSL to represent the concept of "a sequence of statements that can be named". The DSL uses builders and fluent setters to avoid string-level concatenation.
Sample usage:
```java
CodeChunk.Generator codegen = CodeChunk.Generator.forNewTemplate();
codegen
.builder()
.assign("3 / 4")
.condition(
codegen.expr(CodeChunk.THIS, " == null"),
"someFunction();")
.build()
.getCode()
```
generates
```js
var tmp0 = 3 / 4;
if (tmp0 == null) {
someFunction();
}
tmp0 = tmp0 * 5;
}
```
Change: 136652557
Description:
Soy jbcsrc: Integration test for messages with fields of message type
Small fix to SoyExpression.unboxAs() Message handling logic
Change: 136741166
Description:
Soy proto: Proto -> SanitizedContent converters for Tofu
Also move SanitizedContent -> Proto converter code from SoyProtoValueConverter
to SafeStringTypes, and clean up unwieldy generics
Change: 136753325
Description:
Soy JS codegen: improve newline and semicolon handling.
Currently, the CodeChunk DSL handles newlines in an ad-hoc manner, requiring the final output to be trim()med of trailing newlines. This CL plumbs a boolean parameter through formatInitialStatements and formatOutputExpr that communicates whether the current code chunk is the last in its sequence (and therefore doesn't need to emit a trailing newline).
The parameter can also be used to format semicolons correctly. Currently, CodeChunk#getCode can output a single expression without a trailing semicolon as a "statement", this CL tacks on a trailing semicolon, ensuring that all calls to getCode return valid sequences of JavaScript statements.
As a side effect, this CL unblocks an optimization in CodeChunk.Builder#build(): builders consisting of a single assignment can return the RHS of that assignment.
Change: 136757120
Description:
Soy proto: Integration tests for SafeHtml and friends
Uses assertTextEquals instead of assertEquals because it is unclear what the equality semantics for SanitizedContent should be.
Currently, SanitizedContent.equals()
checks for direction, and SanitizedContent created from let blocks
and SanitizedContent pulled out of protos
set Direction differently.
Change: 136772800
Description:
Soy proto init: Remove the "new" keyword for proto initialization
"new" was causing issues in the expression parser, for soy code that uses "new"
as an identifier outside the context of proto init calls. Proto init calls now
have almost the same syntax as function calls, with the difference that
function calls have expression lists as params, and proto init calls have
params that are lists of named ident ":" expression pairs.
Added a new parser pass, RewriteFunctionsPass, to handle ambiguities between
function calls and proto init calls in the case of calls without any params.
Removed the ability for proto init calls to have trailing commas in the params list.
Fixed tests accordingly.
Change: 136773119
Description:
Move all the type expression tokens into a new lexical state.
This is prefactoring for making more of the tags use IN_CMD_TAG_STRUCTURED for
parsing. By moving these tokens into a new lexical state we can move things
like 'alias' and 'delpackage' into IN_CMD_TAG_STRUCTURED rather than default.
This will in turn make it easier to share things like the Attribute()
production.
Change: 136778322
Description:
Soy tofu: UndefinedData ctor is now private
No users other than UndefinedData.INSTANCE. Remove deprecation flag, too.
Change: 136782452
Description:
Add descriptions of our lexical states to the readme.
Change: 136873824
Description:
Move Identifier to soy/base/internal so we can share it across the different parsers and AST nodes.
Change: 136874353
Description:
Soy: Remove unnecessary @SuppressWarnings(deprecation) from IntegerData
Also make the ctor private, since everything uses static forValue() anyway.
Change: 136887782
Description:
Switch template, deltemplate, namespace, alias and delpackage to use structured parsing.
Structured parsing means that we parse these tags using the tokens defined in the IN_CMD_TAG_STRUCTURED lexical state and with bnf expressions in the grammar. For Template nodes this allows us to drop the regular expressions for extracting template names and template attributes and replace them with grammar expressions. This required us to change the builder apis (since we were no longer just passing a string).
This is a prefactoring to enable finer grained lexical states for html parsing.
Change: 136940322
Description:
Rewrite the raw text and whitespace handling in the parser
* merge the 4 BNF productions for raw text into 1. The new code is a little more complex but it should be easier than bouncing around 4 different productions.
* replace the MaybeWhitespace lookaheads with normal expectations and a few custom error messages. This was mostly trivial with the exception of the logic for parsing {msg} bodies.
* add better LOOKAHEAD assertions for raw text (along with better comments). The prior explanations were inaccurate.
This also fixes the source locations for raw text nodes which have had their leading/trailing whitespace dropped, which lead to some test churn.
Change: 136979101
Description:
Change stricthtml mode to use a separate enum (instead of boolean). In a stricthtml namespace, automatically set stricthtml mode for html templates that have not explicitly set it.
Change: 137063011
Description:
Consolidate our comment syntaxes into fewer lexical states and enable comments in the IN_CMD_TAG_STRUCTURED and EXPR states.
This will also unblock efforts to make alias, namespace and delpackage use structured tag parsing.
Change: 137069213
Description:
Soy proto init: Support enums in jbcsrc
Change: 137076643
Description:
Remove the easymock dependency from soy.
It was only used by one test and it was pretty easy to avoid. If we need a mocking solution in the future it would be better to switch to mockito, but we can cross that bridge if/when we get there.
Change: 137078444
Description:
Fix executable names and code block types.
Change: 137081160
Description:
Remove the TemplateHeaderLookaheadHelper
To do this, I used a strategy that was similar to how {msg} tags are handled when they hold a single {plural} or {select}. We collect all whitespace before the each {param} and drop it if it followed by a {param. This does mean that we might accidentally collect/drop initial whitespace, so we now have to return that so it can be prepended to the Template if necessary.
This is the last non-trivial use of lookahead in the SoyFileParser.
Change: 137087455
Description:
Remove all remaining LOOKAHEADs from SoyFileParser by 'left factoring' our
productions.
There is still a lot of unfortunate behavior in here due to
CmdText/CmdTextParts, but cleaning that up is blocked on inlining the
ExpressionParser into the SoyFileParser which is blocked on eliminating v1
expresions.
Change: 137099340
Description:
Get rid of the _AT_SOL and _NOT_SOL lexical states.
We don't need this complexity and I'm kind of embarassed that i didn't realize
it was unnecessary earlier. #boxofshame
Change: 137175957
Description:
Soy proto init: implement non-repeated extensions for jbcsrc
Change: 137205621
Description:
Soy proto init: jbcsrc handling of repeated extension fields
Change: 137221809
Description:
Soy tofu: Data access on nonexistent key/field returns NullData, not Undefined
Example:
{let $foo: ['a': 1, 'b': 2] /}
{$foo.c}
{$foo['c']}
both calls used to return UndefinedData. They now return NullData.
Also refactor EvalVisitor.visitNullSafeDataAccessNode. Logic should still be
the same.
Change: 137222886
Description:
Soy proto init: Integration tests for empty protos with no default values
Note that soy's behavior matches jspb's, not other proto implementations. Empty
protos with no default values should return type defaults (0, "", etc), but
jspb returns null instead, and so do we.
Change: 137267724
Description:
Make the visibility whitelist for gwtsoy more fine grained and add deprecation
notices to the two public build targets.
Change: 137304078
Description:
Soy cleanup: Remove SoyAbstractMap, move all code to AbstractDict
SoyAbstractMap only had one child, AbstractDict. Moved all code from
SoyAbstractMap to AbstractDict, deleted SoyAbstractMap.
Note that SoyAbstractMap.getItem() was not moved, as it was already overridden
by AbstractDict.getItem().
Change: 137305935
Description:
Soy cleanup: Remove SoyData.toBoolean() and .toString()
Explicitly implement coerceToBoolean() and coerceToString() for all children.
Change: 137313877
Description:
Soy cleanup: Remove SoyAbstractList, move all code to ListBackedList
SoyAbstractList only had one child, ListBackedList. Moved all code from
SoyAbstractList to ListBackedList, deleted SoyAbstractList.
Change: 137336799
Description:
Soy cleanup: Small fix in jbcsrc Runtime.java
Change: 137337665
Description:
Soy cleanup: Remove unused methods and vars from SoyValueHelper, DictImpl
Change: 137340847
Description:
Soy cleanup: Have data/internal objects reference SoyValueConverter
SoyValueConverter should just be a concrete class, containing only convert().
SoyValueHelper should handle the easyList/Dict() stuff.
Change: 137342267
Description:
Remove the PoisonTokenError.
This simplifies some of the error recovery code and most of the nice error messages can be handled at a different level.
The one regression here is the error message for when you mismatch angle
brackets inside of a msg tag (we now issue a very generic error parse error). I
don't think this is a big deal.
Change: 137343756
Description:
Add a visitor that checks basic tag mismatches.
- Introduce StrictHtmlValidationUtil that simplifies writing unit tests.
- Currently we enforces that tags must be balanced within every BlockNode. More complicated cases (conditional and loop nodes) will be considered in follow-up CLs.
Change: 137344483
Description:
Add missed license for StrictHtmlValidationUtilTest.
Change: 137402959
Description:
Add a check that the lexer doesn't get more than one token ahead of the parser
at any of the points where we will be inserting lexical state overrides.
This acts as a guard to make sure we don't introduce any greedy lookaheads
without noticing in the future.
Change: 137427286
Description:
Fix https://github.com/google/closure-templates/issues/111
This bug was introduced by cl/114109032 which accidentally made a 'private'
token public. So even though within a multiline comment we would consume <~[]>
(any single character) <WS> was matchable by a two character sequence '\r\n'
since it was defined in state <*> this means if a \r\n appeared in a multiline
comment it would result in a parser error.
The fix is to make the pseudo token private again. It is hard to find
documentation of this feature... but it appears that if a token is prefixed
with # then it acts more like a macro and doesn't match as a token at all.
Change: 137429139
Description:
Switch DomSelector to use Jsoup (instead of xmlunit).
Before this CL, DomSelector parses input as XML, and void tags are incorrectly thrown as errors. Jsoup (https://jsoup.org/) is a newer HTML parser that generates valid HTML.
Change: 137429597
Description:
Soy cleanup: Remove SoyAbstractRecord and SoyAbstractCachingRecord
SoyAbstractRecord code moved to children (ParamStore and children,
SoyProtoValue)
SoyAbstractCachingRecord code moved to only child (SoyProtoValue)
longer exist. Please use SoyRecord instead.
Change: 137430039
Description:
Soy cleanup: Define SoyValueConverter.UNCUSTOMIZED_INSTANCE
In preparation for cl/137345592 rosie change
Change: 137442995
Description:
Soy cleanup: RecordType no longer inherits from SoyObjectType
Change: 137445817
Description:
Add a simple check for void tags.
Change: 137453243
Description:
Soy cleanup: Stop using EasyList/EasyDict for literals
Use ListImpl and DictImpl instead, which are always immutable.
Change: 137455223
Description:
Soy cleanup: Remove most of SoyEasyDict functionality, deprecate rest
Change: 137457113
Description:
Soy cleanup: Remove most of SoyEasyList functionality, deprecate rest
Change: 137539345
Description:
Soy JS codegen: improve CodeChunk DSL to handle if/else if/else statements.
Currently the DSL only handles if/then. It's straightforward to support an arbitrary
number of else if clauses and an optional trailing else clause.
Change: 137554601
Description:
Added Soy depot readme file.
Change: 137718933
Description:
Add support for unknown proto3 enum values.
In proto2 unknown enum values get mapped to the 0 value, in proto3 the value is preseved but the enum returned is a hardcoded constant called UNRECOGNIZED. If you call getNumber() on UNRECOGNIZED it throws an exception.
So to support this case in proto3 we instead call get<Field>Value on proto3 enum fields, this will return the int directly and so we don't need to call the getNumber method and no exception will be thrown.
Also: change the java package namespace on the testing protos in test_data.
Change: 137734312
Description:
Soy proto init: Register error in pysrc
We currently have no plans to support proto init calls in pysrc.
Change: 137735471
Description:
Soy: CodeChunk DSL: enforce idempotency for initial statements.
Currently, the CodeChunk DSL has logic for formatting the initial statements of a chunk of code that cannot be represented as a single expression. But this logic runs unconditionally every time a chunk is encountered during a single getCode() call. If a chunk appears multiple times in the tree traversed by a getCode() call, its initial statements will incorrectly be formatted multiple times.
This CL introduces a FormattingContext helper class that remembers the chunks that have been formatted during a getCode() call, ensuring the initial statements are formatted at most once.
Change: 137751252
Description:
Soy tofu: UndefinedData == NullData
In javascript,
null == undefined
undefined == null
both evaluate to true. We should do the same.
Needed to roll cl/137307416 forward, since oz mobile basic has a test that
compares UndefinedData and NullData.
Change: 137771967
Description:
Introduce an explicit stack data structure to manage lexical states and add utilities for calling it from the parser.
To enable parsing html tags we will need to introduce a number if new lexical states in order to start recognizing new tokens such as '<', '"', and '>' in the lexer. To do this properly we will need to be able to change states from the parser and to re-enter old states. Currently we accomplish similar goals via a few state fields in the token manager (isInTemplate, isInMsg, priorState), but these aren't general enough for the operations we will be supporting (they aren't even general enough for current usecases, see the changes in msg_html_nesting.err).
Change: 137840784
Description:
Soy cleanup: Move contents of SoyValueHelper to SoyValueConverter
Change: 137843191
Description:
Soy cleanup: SoyProtoTypeImpl uses Kind.PROTO instead of Kind.OBJECT
Change: 137846902
Description:
Soy cleanup: SoyProtoTypeImpl no longer inherits from SoyObjectType
SoyObjectType is now test-only code. Soon to be deleted.
Change: 137851267
Description:
Soy cleanup: Remove SoyObjectType and associated code
Removing SoyObjectType and SoyType.Kind.OBJECT. We now have RECORD and PROTO
only.
Change: 137883363
Description:
Soy cleanup: Move SoyValueHelperTest to SoyValueConverterTest
Change: 137973064
Description:
Point Frameworks docsets extension files at the canonical copies in //frameworks/g3doc.
CL automatically created by:
replace_string \
\<\!--#include\ file=\"/social/boq/g3doc/head_extension.shtml\"--\> \
\<\!--#include\ file=\"/frameworks/g3doc/head_extension.shtml\"--\>
* Removed any files not owned by Frameworks team & followed up separately.
* Made a couple manual fixes where files were being double-included or not included.
Change: 137973192
Description:
Soy cleanup: Switch SoyValueHelper over to SoyValueConverter
Everything in java/.../soy except SoyModule
Change: 137992928
Description:
Store node (instead of string) in our stack.
- For HtmlOpen/CloseTagNode, Add some methods that return the child (and add assertions that we need to have at least one child).
- For RawTextNode, we compare the text contents.
- For PrintNode, we compare the expression (and all print directives) using ExprEquivalence.
- Other type of nodes are not supported for dynamic tag names.
- Add an ExprUnionExquivalence to handle both v1 and v2 expressions.
- Correctly point to the OpenTagNode for OPEN_TAG_NOT_CLOSED.
Change: 138002574
Description:
Soy cleanup: Move SoyValueHelperTest to SoyValueConverterTest
Change: 138012409
Description:
Remove double-inclusion of multicol.
Change: 138012778
Description:
Soy JS codegen: begin incremental migration to CodeChunk DSL.
Currently, the Soy JS backend assumes that a single Soy expression can be translated to an equivalent JS expression. This is no longer accurate with the ability to instantiate protocol buffers from within a Soy template. (The JS API uses non-fluent setters: `Proto(field: 32)` in Soy may translate into `var $$tmp = new Proto(); $$tmp.setField(32);` in JS.)
The existing CodeChunk DSL is designed to support chunks of JS code that may not be expressions. Because of the recursive nature of the JS codegen system, migrating to the CodeChunk DSL is not straightforward. (For example, the return types of all the TranslateToJsExprVisitor methods must be changed together.)
Here is the migration strategy. It is unusual but incremental and safe.
0. [this CL] Add temporary methods to the CodeChunk DSL to interoperate with the existing system (JsExprs). CodeChunk#assertExpr throws if the chunk is not an expression.
1. [this CL] Add a dummy JS codegen logic for proto initializers. A check in ResolveExpresionTypesVisitor ensures that only tests can use this syntax.
2. [this CL] Change the return types of all the TranslateToJsExprVisitor methods from JsExpr to CodeChunk, but do not change the actual business logic; use the methods from step 0 to wrap and unwrap as necessary. This is safe as long as production code never creates a non-expression CodeChunk, which is guaranteed by step 1.
3. [this CL] support CodeChunks natively in a single piece of Soy syntax. (In this CL, we support CodeChunks within list literal expressions.) Write unit tests against the new syntax using the dummy proto init syntax from step 1.
4. [subsequent CLs] incrementally migrate one piece of syntax at a time to support CodeChunks natively.
Change: 138018195
Description:
Fix usages of generated AutoValue implementation classes
that occur outside their AutoValue class definition.
Change: 138021495
Description:
Soy cleanup: Stop using instanceof to check soy types
Consolidate to use type.getKind() == SoyType.Kind.KIND everywhere
As a side effect, move ResolveExpressionTypesVisitorTest.testProtoInitTyping to
use a real SoyProtoTypeImpl instead of a fake implementation.
compare type.getKind() to SoyType.Kind enums.
Change: 138023356
Description:
Soy cleanup: Rename SoyType.Kind.ENUM to SoyType.Kind.PROTO_ENUM
Delete SoyEnumType
is the only child. SoyType.Kind.ENUM renamed to PROTO_ENUM.
Change: 138113990
Description:
Soy JS codegen: plumb CodeChunk.Generator throughout backend.
Instantiate once per template in GenJsCodeVisitor#visitTemplateNode and propagate it through collaborators. Because collaborators have repetitive argument lists, encapsulate CodeChunk.Generator and a couple other params inside an @AutoValue TranslationContext struct.
There is no behavior change in this CL.
Change: 138126770
Description:
Soy cleanup: Introduce soy.data.SoyProtoValue interface
TODO: Migrate all users of SoyProtoTypeImpl.Value to soy.data.SoyProtoValue
Change: 138215421
Description:
Soy cleanup: Bind SoyValueConverter instead of SoyValueHelper
Move the binding to shared/internal/SharedModule
use SoyValueConverter instead.
Change: 138245285
Description:
Soy proto init: ProtoSupportTest uses experimental flag
ProtoSupportTest uses experimental flag to disable error in
ResolveExpressionTypesVisitor, instead of using hacky error-catching
Spruce up TemplateTester a bit
Change: 138257098
Description:
Add typo suggestions to a few soy error messages.
To do this I implemented the Levenshtein distance algorithm but did it such
that it ignores case when comparing characters. I also used a heuristic from
llvm to pick a max distance so that if all the matches are farther than that we
don't suggest anything.
See https://github.com/google/closure-templates/issues/117
Change: 138407494
Description:
Soy: no need to strip @ForOverride anymore.
They appear to be from a time when the annotation wasn't available publicly.
Change: 138424865
Description:
Remove unused imports of types from indirect dependencies
and fully-qualify any references to those types in javadoc.
This fixes a hole in strict_java_deps enforcement: Normally, types a class refers to must be part of the target's direct dependencies. (This allows the type's owners to change their own dependencies without breaking compilation downstream.) However, enforcement currently excludes import statements, so dependency changes can still occasionally break users who import a type but don't use it in code.
Change: 138429516
Description:
Support JS enum expressions in the open source version of soy.
For a proto enum, the qualified name of the JS type is computed the same
as it is for messages: the proto descriptor's full name prepended
with "proto."
Change: 138450540
Description:
Soy cleanup: Make it explicit that proto accessors are not supported
Right now the pathway is
TranslateToPyExprVisitor ->
SoyProtoTypeImpl.getFieldAccessExpr ->
Field.getAccessExpr ->
NormalField.getAccessExpr ->
UnsupportedOperationException.
Change: 138452623
Description:
Soy cleanup: Remove SoyProtoTypeImpl.Value, switch to soy.data.SoyProtoValue
TODO: Move types.proto.SoyProtoValue to data.internal.SoyProtoValueImpl
Change: 138455744
Description:
Soy cleanup: Remove types.proto.SoyProtoType
Change: 138534014
Description:
Soy: CodeChunk DSL: add CodeChunk.WithValue marker class.
Currently, the DSL has a method for querying whether a chunk of code can be represented as a single expression. But there is another, even more important, way to classify chunks of code: by whether they represent a value. (It is independent from whether they can be represented as a single expression, so there are 4 possibilities.)
This CL introduces CodeChunk.WithValue to encode this fact in the type system, and strengthens the parameter types of various APIs to accept only CodeChunk.WithValue instances. (For example, it makes no sense to use a chunk of code that does not represent a value as the RHS of an assignment, or the predicate of a conditional.)
With this distinction in place, we can improve the representation of various constructs. Three-valued conditional expressions can be formatted as JavaScript ternary expressions. Conditional statements whose every branch has a value can be upgraded to conditional expressions.
Change: 138538041
Description:
Soy JS codegen: support non-expressions in data refs.
Currently, TranslateToJsExprVisitor recursively builds up a chain of null-safe data references. The generated code is suboptimal; parts are evaluated more than once. For example, the Soy expression `$boo?[0]?[1]` currently generates the JavaScript expression
```
(opt_data.boo == null)
? null
: (opt_data.boo[0] == null)
? null
: opt_data.boo[0][1]
```
The correct approach is to generate conditional statements:
```
var tmp = opt_data.boo;
if (tmp != null) {
tmp = tmp[0];
if (tmp != null) {
tmp = tmp[1];
}
}
```
This is possible with the new CodeChunk DSL. However, non-expressions cannot be emitted in non-test code yet. So this CL uses a different CodeChunk DSL (CodeChunk.Generator#ternary) to replicate the existing suboptimal gencode. Once the rest of the JS codegen system works with non-expressions, it will be possible to improve the generated code.
Change: 138540366
Description:
delete the MultipleErrorsTest. This usecase is well covered by our integration tests.
Change: 138557786
Description:
Tweak the comment on the RunParser tool and delete the shell script which isn't very useful.
Change: 138559780
Description:
Add an explicit TagName field to Html(Open|Close)TagNode
It is optional to account for dynamic tag names, but for normal tag names this will be the exact text (e.g. 'div' or 'a')
Change: 138668666
Description:
Soy: off-by-one error in translating javacc TokenMgrError messages to SourceLocations.
Change: 138674434
Description:
Soy opensource: bump Guava 19.0 => 20.0.
Change: 138697733
Description:
Fix a bug that could cause our lexical state stack to underflow if certain syntactic errors occurred.
Now if we detect that an error ocurred between a pushState()/popState() pair we are more careful about popping the stack.
Also when doing ParseException error recovery within template bodies detect the
end of the template and abort template parsing completely. This is a bit
cleaner than reporting multiple parse exceptions for each subsequent closing
tag.
Change: 138711035
Description:
Soy: roll forward cl/138538041 (support non-expressions in data refs) with fix.
Soy expression trees are "left-leaning". For example, `$a?.b.c` is represented by
.
/ \
?. c
/ \
a b
The OCL did not propagate the null-safeness of subexpressions up the expression tree correctly. It assumed that the null-safeness of an expression was the null-safeness of its topmost operator. It therefore considered expressions like `$a?.b.c` to be not null-safe, generating incorrect code like `opt_data.a.b.c`.
This roll-forward correctly propagates the null-safeness of subexpressions.
Change: 138783669
Description:
Soy cleanup: Rename SoyProtoTypeImpl to SoyProtoType
Change: 138821447
Description:
Soy cleanup: Rename SoyProtoEnumTypeImpl to SoyProtoEnumType
Change: 138916383
Description:
Add java_binary for SoyToIncrementalDomSrcCompiler
Change: 139101963
Description:
Soy: CodeChunk DSL: move isRepresentableAsSingleExpr from CodeChunk to CodeChunk.WithValue.
The definition of an expression is "something that has a value", so it naturally belongs on CodeChunk.WithValue. This allows us to remove several implementations that always returned false.
Change: 139104336
Description:
Soy JS codegen: support non-expressions in function calls.
Change: 139117639
Description:
Soy: CodeChunk DSL: fix Statement#doFormatInitialStatements.
Statement is a recursive type: formatting its initial statements involves formatting the initial statements and output expressions of its parts. Currently, the parts are formatted serially in one pass: the initial statements of the first part, the output expression of the first part, the initial statements of the second part, etc. This is wrong. It makes no sense to serialize initial statements when we are in the middle of another expression.
Change: 139119515
Description:
Soy JS codegen: support non-expressions in checkNotNull.
Change: 139139524
Description:
Change the way template, let and param commands are parsed so that the kind attribute (should it exist) can be available to the parser before consuming the '}' token.
This is neccessary to ensure that we can use lexical states to parse html tags.
Change: 139220467
Description:
Soy: add third_party data to METADATA.
Also rename opensource/README.google => README_INTERNAL.md.
README.google files have been deprecated in favor of third_party sections in METADATA files.
Change: 139225125
Description:
Soy JS codegen: Misc. cleanup for CodeChunk
Change: 139265148
Description:
Soy: CodeChunkTest: document incorrect operator precedence behavior.
Currently, the CodeChunk DSL does not really understand or use operator precedence. It will preserve the precedence of code chunks converted directly from a single JsExpr, but any expression built from multiple JsExprs getd a default precedence of Integer.MAX_VALUE, and the tree structure of the underlying code chunks is lost. This leads to improper parenthesization.
The solution is to make the DSL understand operator precedence natively by making several subclasses of CodeChunk.Expr, one for each operator, that knows its own precedence. Then the DSL can insert parens as needed.
Change: 139347876
Description:
Soy JS codegen: support non-expressions in null coalescing expressions (?:).
Currently, the JS codegen declares an inline temporary variable to avoid re-evaluating the left-hand side of ?:. For example, `a ?: b` translates to
```
($$temp = opt_data.a) == null ? opt_data.b : $$temp
```
Once the JS backend supports non-expressions, the idiomatic way to translate this will be
```
var $$temp = opt_data.a;
if ($$temp == null) {
$$temp = opt_data.b
}
```
This is currently supported by the CodeChunk DSL, but we cannot use it yet because the chunk is not representable as a single expression.
This CL replicates the existing inline temporary variable hack until we can use the non-expression approach.
Change: 139365650
Description:
Soy JS codegen: Float use of CodeChunk up to JsExprTranslator
next: converting GenJsCodeVisitor et al to be CodeChunk-aware.
Change: 139387358
Description:
Fix a lexical state stack underflow.
If we see a {/template} or a {/deltemplate} we clear the stack and switch to the default state, the problem is that if a {/template} exists where a {/param} should be then we might immediately try to pop the stack. To handle this we only pop the stack if the next token is what we expect it to be.
Change: 139516724
Description:
Soy JS codegen: Migrate GenJsExprsVisitor.visitCssNode()
Change: 139554753
Description:
Changes the html pass of the soy AST to emit HtmlOpenTagNodes with attributes
as children instead of HtmlOpenTagStartNode/HtmlOpenTagEndNode pairs with attributes as
siblings.
Change: 139589936
Description:
Soy: CodeChunk DSL: move formatOutputExpr from CodeChunk to CodeChunk.WithValue.
This CL continues refining the distinction between code chunks that represent values and code chunks that do not. Currently, all CodeChunks must implement formatOutputExpr. But, as the name suggests, this makes sense only for code chunks that have some concept of a value. Moving formatOutputExpr to CodeChunk.WithValue lets us delete a few overrides that were no-ops or throwing IllegalStateExceptions.
Change: 139597383
Description:
Soy: improve javadoc of CodeChunk.WithValue#singleExprOrName, fixing 1 impl.
Currently, when singleExprOrName() is called on an expression that cannot be represented as a single expression, the whole chunk is serialized, including its dependent chunks. This is wrong. singleExprOrName() must always return a valid expression (so that it can be passed into contexts that expect only expressions, such as plugins).
Change: 139819997
Description:
Small fix in soy message translation doc
Change: 139821885
Description:
Soy: TranslateToJsExprVisitor: change generic type from CodeChunk to CodeChunk.WithValue.
TranslateToJsExprVisitor's generic type was changed from JsExpr to CodeChunk in cl/138012778. At that time, there was no sharp distinction between code chunks that represented values and code chunks that did not. Recently, this distinction has been encoded in the type system, with Co…- Loading branch information
1 parent
8c383d1
commit 7dae5a1