Skip to content

[Strings] Add a string-builtins feature, and lift/lower automatically when enabled #7601

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 26 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 12 commits
Commits
File filter

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -201,7 +201,9 @@ There are a few differences between Binaryen IR and the WebAssembly language:
stringref operations, before any default optimization passes. Those
stringref operations can then be optimized (e.g., a concat of constants
turns into a concatenated constant). When we are about to finish running
default optimizations, we lower stringref back into string builtins.
default optimizations, we lower stringref back into string builtins. (Note:
reference types and GC must also be enabled, as imported string operations
depend on GC arrays.)

As a result, you might notice that round-trip conversions (wasm => Binaryen IR
=> wasm) change code a little in some corner cases.
Expand Down
18 changes: 13 additions & 5 deletions src/passes/pass.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -727,8 +727,11 @@ void PassRunner::addDefaultGlobalOptimizationPrePasses(Ordering ordering) {
// If we are optimizing string builtins then we lift at the very start of the
// optimization pipeline, not just at the beginning here, but only when we are
// ordered before other bundles of passes.
if (wasm->features.hasStringBuiltins() && options.optimizeLevel >= 2 &&
ordering.first) {
//
// We check for GC for symmetry with the lowering pass, see comment in
// addDefaultGlobalOptimizationPostPasses() below.
if (wasm->features.hasStringBuiltins() && wasm->features.hasGC() &&
options.optimizeLevel >= 2 && ordering.first) {
addIfNoDWARFIssues("string-lifting");
}
// Removing duplicate functions is fast and saves work later.
Expand Down Expand Up @@ -805,9 +808,14 @@ void PassRunner::addDefaultGlobalOptimizationPostPasses(Ordering ordering) {
// Lower away strings at the very very end. We do this before
// remove-unused-module-elements so we don't add unused imports, and also
// before reorder-globals, which will sort the new globals.
if (wasm->features.hasStringBuiltins() && options.optimizeLevel >= 2 &&
ordering.last) {
addIfNoDWARFIssues("string-lowering");
//
// Note we also test for GC here, as the pass adds imports that use GC arrays
// (and externref). Those imports may be unused, but they exist until
// remove-unused-module-elements cleans them up, which would cause an error in
// between.
if (wasm->features.hasStringBuiltins() && wasm->features.hasGC() &&
options.optimizeLevel >= 2 && ordering.last) {
addIfNoDWARFIssues("string-lowering-magic-imports");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we use string-lowering-magic-imports-assert to make sure we aren't accidentally doing any optimizations that would result in us emitting a non-standard custom section for non-UTF-8 string constants?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wait, isn't the section standardized?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No. The standard solution is the magic imports, which can only handle valid UTF-8 strings. The custom section is a random thing we experimented with on the way to developing magic imports.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the spec solution for non-UTF8 strings, then?

Separately, if we assert here, then any module with a non-utf8 stringref will assert if you just do -all -O2... that seems wrong to me.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is no spec solution for non-UTF-8 strings. We could synthesize them in a start function if we had to, but I don't think that should be necessary. No input module should have non-UTF-8 strings because they are not expressible with string builtins, and our optimizations should not produce new invalid non-UTF-8 strings, so no output module should have non-UTF-8 strings. Input modules that use stringref to represent non-UTF-8 strings probably don't want this lowering to occur in the first place.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can do the lifting/lowering only when stringref is not enabled

Hmm, seems weird for -all to do less than -all --disable-stringref. More features should mean more things to optimize, normally.

Ok, we can do the lifting when either stringref or string-builtins are enabled, then lower only if string-builtins and not stringref are enabled.

and also make non-UTF-8 string.const a validation error when stringref is not enabled.

When stringref is disabled, any string.const (UTF8 or otherwise) is invalid anyhow?

string.const should still be valid when string-builtins are enabled. We might want to error in binary writing if they haven't been lowered and stringref is not enabled, though.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, these are valid options, but they all seem significantly more complex to explain and to use. The readme text would need to be substantially longer.

I don't have a better suggestion for this PR yet, but let's keep thinking here.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On the contrary, I don't think there's much to explain. As always, we optimize to the extent we can given the enabled features, and we validate that the IR will be written as valid modules given the enabled features

We should document that string.const is valid IR if either of the string features are enabled, but only on the condition that it will be lowered before writing if stringref is disabled. We should also document that string.const containing unpaired surrogates is only valid if stringref is enabled. I don't think we need to document how automatic lifting and lowering works, just as we don't need to document the specifics of what other optimizations we run.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd say all of these quotes are non-trivial and potentially confusing for our users:

  • "do the lifting when either stringref or string-builtins are enabled, then lower only if string-builtins and not stringref are enabled"
  • "string.const is valid IR if either of the string features are enabled, but only on the condition that it will be lowered before writing if stringref is disabled"
  • "string.const containing unpaired surrogates is only valid if stringref is enabled."

Just adding that text by itself would double the current PR's readme entry.

I really feel we should find something simpler here.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd say all of these quotes are non-trivial and potentially confusing for our users:

  • "do the lifting when either stringref or string-builtins are enabled, then lower only if string-builtins and not stringref are enabled"

This is just us doing optimizations based on the target features. It doesn't need to be documented or understood by users.

  • "string.const is valid IR if either of the string features are enabled, but only on the condition that it will be lowered before writing if stringref is disabled"

Yeah, this one is weird. If lowering were automatic in the binary writer, this wouldn't need to be documented or understood, either. This complexity is the price we pay to make lowering a separately sequenced pass.

  • "string.const containing unpaired surrogates is only valid if stringref is enabled."

This one just matches the expressive capabilities of the underlying features. I don't think it needs to be documented beyond error messages.

Just adding that text by itself would double the current PR's readme entry.

I really feel we should find something simpler here.

This is the validation behavior we would want independent of how we lift and lower strings. The only reason this complexity comes up in this PR and not before is because we didn't have separate string features before this PR. But it's strictly more useful to users to have separate string features with detailed validation to ensure they actually produce valid binaries for the intended engine feature set.

}

addIfNoDWARFIssues("remove-unused-module-elements");
Expand Down
2 changes: 2 additions & 0 deletions src/wasm/wasm-binary.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -5195,6 +5195,8 @@ void WasmBinaryReader::readFeatures(size_t payloadLen) {
feature = FeatureSet::FP16;
} else if (name == BinaryConsts::CustomSections::CustomDescriptorsFeature) {
feature = FeatureSet::CustomDescriptors;
} else if (name == BinaryConsts::CustomSections::StringBuiltinsFeature) {
feature = FeatureSet::StringBuiltins;
} else {
// Silently ignore unknown features (this may be and old binaryen running
// on a new wasm).
Expand Down
2 changes: 1 addition & 1 deletion test/binaryen.js/kitchen-sink.js.txt
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ Features.RelaxedSIMD: 4096
Features.ExtendedConst: 8192
Features.Strings: 16384
Features.MultiMemory: 32768
Features.All: 4194303
Features.All: 8388607
InvalidId: 0
BlockId: 1
IfId: 2
Expand Down
2 changes: 1 addition & 1 deletion test/example/c-api-kitchen-sink.txt
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ BinaryenFeatureMemory64: 2048
BinaryenFeatureRelaxedSIMD: 4096
BinaryenFeatureExtendedConst: 8192
BinaryenFeatureStrings: 16384
BinaryenFeatureAll: 4194303
BinaryenFeatureAll: 8388607
(f32.neg
(f32.const -33.61199951171875)
)
Expand Down
Loading
Loading