Skip to content

Multiple import maps#10528

Merged
domenic merged 63 commits intowhatwg:mainfrom
yoavweiss:dynamic_module_imports
Nov 14, 2024
Merged

Multiple import maps#10528
domenic merged 63 commits intowhatwg:mainfrom
yoavweiss:dynamic_module_imports

Conversation

@yoavweiss
Copy link
Copy Markdown
Contributor

@yoavweiss yoavweiss commented Jul 30, 2024 •
edited by pr-preview Bot
Loading

Introduction

Import maps in their current form provide multiple benefits to web developers. They enable them to avoid cache invalidation cascades, and to be able to work with more ergonomic bare module identifiers, mapping them to URLs in a convenient way, without worrying about versions when importing.

At the same time, the current import map setup suffers from fragility. Only a single import map can be loaded per document, and it can only be loaded before any module script is loaded. Once a single module script is loaded, import maps are disallowed.
That creates a situation where developers have to think twice (or more) before using module scripts in situations that may introduce import maps further down in the document. It also means that using import maps can carry a risk unless you’re certain you can control all the module scripts loaded on the page.

Beyond that, the fact that import maps have to be loaded before any module means that the map itself acts as a blocking resource to any module functionality. Large SPAs that want to use modules, have to download the map of all potential modules they may need during the app’s lifetime ahead of time.

So, it seems like there’s room for improvement. Enabling more dynamic import maps would allow developers to avoid these issues and fully benefit from import maps’ caching and ergonomic advantages without incurring a cost when it comes to stability or performance.

At the same time, the current static design gives us determinism and isn’t racy. A module identifier that resolves to a certain module will continue to do so throughout the lifetime of the document. It would be good to keep that characteristic.

Objectives

Goals

  • Increase robustness when using ES modules and import maps
  • Enable expanding the Window’s import map throughout its lifetime
  • Satisfy the EcmaScript HostLoadImportedModule requirement that multiple calls will always resolve to the same module
  • Minimize race-conditions which can result in different module resolutions on different loading sequences of the same page.

Non-Goals

  • Provide a programmatic way to expand or modify the Window’s import map - out-of-scope for the current effort
  • Completely avoid race conditions that can result in different module resolutions based on network conditions
    • Such races are already possible today (e.g. if an import map is dynamically injected by a classic script which may or may not run before a module is loaded)
    • Specifically for dynamic modules, requiring this would conflict with the “expanding the import map over time” goal.

Use Cases

Third party scripts

When third party scripts integrate themselves to web pages today, they cannot do that as ES modules without taking on some risk. That risk varies somewhat, depending on their form of integration.

Injected without developer supervision

That could include third party scripts injected by the CDN, by a CMS or some other automated system that isn’t content-aware.

For such scripts to be loaded as ES modules, they have to make sure that they are not loaded before any import maps in the content.

They can do that by:

  • Loading at the bottom of the page, which may or may not correspond with the point in which they typically need to load in order to function optimally.
  • Buffering the content and validating that no import maps are present, which can incur performance penalties.

Developer-injected snippets

For snippets-based 3Ps, they need to provide instructions so that the developer is aware of import maps in their page and only injects the snippet after it. That may or may not be a realistic thing to ask. It’d definitely increase the integration’s complexity, resulting in a higher percentage of failures or support calls.

Content Management Systems

Content management systems often have markup and code arriving from multiple different sources. Site owners, theme developers and application/extension/plug-in developers all take part in generating the final markup of the page delivered to the user, which often contains lots of scripts. Some of that code can be static, while other parts can vary per user.

If any of that code contains an import map, extreme caution needs to be taken when integrating all these different script entry points, if any of them is an ES module.

Browser Extensions

A similar problem exists with browser extensions, where if extension-injected code wants to use ES modules or import maps, it needs to verify ahead of time that it doesn’t collide with the content itself and where the code is added relative to the rest of the page.

Large Single-Page Apps

Serving hundreds to thousands of different modules is a reality for large SPAs. While bundling is used to speed up the loading-performance cost of modules, in later stages of the application lifetime, it doesn’t always make sense to bundle - while it can reduce the weight of modules over the network (by improving compression ratios), it can also cause over-fetching and less-granular caching which can result in frequent invalidations.

So apps end up with several thousands of modules that may load during the lifetime of the app, using dynamic import.
Using import maps can significantly help such apps avoid cache invalidation cascades, but it also presents a challenge.
An import map for such a site needs to include all the thousands of different modules it may import, and it needs to do that before any module loads. As such, the quite-large import map would be blocking any module-based functionality. That’s a significant performance tradeoff.

Usage examples

There are two cases when rules of the new import map don't get merged into the existing one.

The new import map rule has the exact same scope and specifier as a rule in the existing import map. We'll call that "conflicting rule".

The new import map rule may impact the resolution of an already resolved module. We'll call that "impacted already resolved module".

Two import maps with no conflicts

When the new import map has no conflicting rules, and there are no impacted resolved modules, the resulting map would be a combination of the new and existing maps. Rules that would have individually impacted similar modules (e.g. "/app/" and "/app/helper") but are not an exact match are not conflicting, and all make it to the merged map.

So, the following existing and new import maps:

{
   "imports": {
    "/app/": "./original-app/",
  }
}

{
  "imports": {
    "/app/helper": "./helper/index.mjs"
  },
  "scopes": {
    "/js": {
      "/app/": "./js-app/"
    }
  }
}

Would be equivalent to the following single import map:

{
  "imports": {
    "/app/": "./original-app/",
    "/app/helper": "./helper/index.mjs"
  },
  "scopes": {
    "/js": {
      "/app/": "./js-app/"
    }
  }
}

New import map defining an already-resolved specifier

When the new import map impacts an already resolved module, that rule gets dropped from the import map.

So, if the top-level resolved module set already contains the pair (null, "/app/helper"), the following new import map:

{
   "imports": {
    "/app/helper": "./helper/index.mjs",
    "lodash": "/node_modules/lodash-es/lodash.js"
  }
}

Would be equivalent to the following one:

{
  "imports": {
    "lodash": "/node_modules/lodash-es/lodash.js"
  }
}

New import map defining an already-resolved specifier in a specific scope

The same is true for rules defined in specific scopes. If the resolved module set contains the pair ("/app/main.mjs", "/app/helper"), the following new import map:

{
  "scopes": {
    "/app/": {
      "/app/helper": "./helper/index.mjs"
    },
  }
   "imports": {
    "lodash": "/node_modules/lodash-es/lodash.js"
  }
}

Would similarly be equivalent to:

{
  "imports": {
    "lodash": "/node_modules/lodash-es/lodash.js"
  }
}

The script in the pair is the script object itself, rather than its URL, so these examples are somewhat simplistic in that regard.

Already-resolved specifier and multiple rules redefining it

We could also have cases where a single already-resolved module specifier has multiple rules for its resolution, depending on the referring script. In such cases, only the relevant rules would not be added to the map.

For example, if the rop-level resolved module set contains the pair ("/app/main.mjs", "/app/helper"), the following new import map:

{
  "scopes": {
    "/app/": {
      "/app/helper": "./helper/index.mjs"
    },
    "/app/vendor/": {
      "/app/": "./vendor_helper/"
    },
    "/vendor/": {
      "/app/helper": "./helper/vendor_index.mjs"
    }
  },
   "imports": {
    "lodash": "/node_modules/lodash-es/lodash.js"
    "/app/": "./general_app_path/"
    "/app/helper": "./other_path/helper/index.mjs"
  }
}

Would be equivalent to:

{
  "scopes": {
    "/vendor/": {
      "/app/helper": "./helper/vendor_index.mjs"
    }
  },
  "imports": {
    "lodash": "/node_modules/lodash-es/lodash.js"
  }
}

This is achieved by the fact that the merge algorithm uses a copy of the resolved module set and removes already referring script specifier pairs from it if they already resulted in a rule being ignored.

Two import maps with conflicting rules

When the new import map has conflicting rules to the existing import map, with no impacted already resolved modules, the existing import map rules persist.

For example, the following existing and new import maps:

{
   "imports": {
    "/app/helper": "./helper/index.mjs",
    "lodash": "/node_modules/lodash-es/lodash.js"
  }
}

{
  "imports": {
    "/app/helper": "./main/helper/index.mjs"
  }
}

Would be equivalent to the following single import map:

{
  "imports": {
    "/app/helper": "./helper/index.mjs",
    "lodash": "/node_modules/lodash-es/lodash.js",
  }
}

High-level design

At a high-level, we want a module resolution cache that will ensure that a resolved module identifier always resolves to the same module. That is implemented using the "resolved module set", which ensures that URLs for modules that were already resolved cannot be added to future import maps.

We also want top-level imports that start loading a module tree won’t have that tree change “under their feet” due to an import map that was loaded in parallel. That is achieved by providing a copy of the import maps to the module resolution algorithm of these top-level modules and propagating it recursively down its module tree.

And finally, we want a way to create a single, coherent import map from multiple import map scripts loaded on the document. That is done with the "merge new and existing import maps" algorithm.

(See WHATWG Working Mode: Changes for more details.)

/scripting.html ( diff )
/webappapis.html ( diff )
Loading
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

addition/proposal New features or enhancements topic: script