...
"But in CommonJS you can use those elsewhere too, and that breaks static analyzers!", I hear you say. Well, yes, absolutely. But that is inherent in dynamic imports, which by the way, ESM also supports with its dynamic import() syntax. So it doesn't solve that either! Any static analyzer still needs to deal with the case of dynamic imports somehow - it's just rearranging deck chairs on the Titanic.
I think while OP's right in theory, there is still a lot of difference between the two: ESM has dedicated syntax for static loading of modules and that syntax is strongly communicated to be the standard solution to use if you want to load a module. Yes, dynamic imports exist but they are sort of an exotic feature that you would only use in special situations.
In contrast, CommonJS imports are dynamic by default and only happen to be statically analysable if you remember to write all your imports at the beginning of the module. That's a convention that's enforced through nothing and is not part of the language or even of CommonJS.
As an exercise, try to write a static analyser that simply ignores dynamic imports and just outputs a dependency graph of your static imports - and compare how well this works with CommonJS vs ESM.
"But no one would do that!" Yeah, and no one does it with require either. The times people pass in variables to require are well warranted and usually not in a browser context (listed this in my other post, but for example loading "x-mac.js" vs. "x-windows.js" in node, where there is no problem since you're not bundling).
So neither in the practical sense nor the hypthetical sense has the "static analysis problem" been solved any more than it was already solved with require. People used require basically like a static keyword before and you could basically make the same usage assumptions as import in those cases. Similarly, you can get as dynamic as require with import, so in the purely theoretical sense not much has been made better either.
I wouldn't necessarily describe dynamic imports as an exotic feature. They are basically required if you're building a semi-large app (and heavily advised by most frameworks in that case!). Otherwise, your homepage is going to load in some complicated AuthorPage / HeavyComponentWithLotsOfDependencies component the user might not even want.
The main advantage of ESM in this context would be its asynchrony, which you won't get with require. Webpack tried to tack this on with require.ensure, but it was nonstandard and eventually deprecated.
Another advantage of ESM in general would be that you wouldn't have to use preprocessors and compilers like Webpack. My main reasoning here is that, prior to the import spec, web didn't have any form of imports aside from loading in scripts and polluting the global namespace (which isn't necessarily bad). The majority of websites IME use some sort of bundler though, so this isn't really a major change so much as a nice to have, I suppose?
There are web frameworks pushing this pretty hard for basic stuff (for example: loading React components with dynamic import) to build page-content-streaming functionality around it.
In practice, it is used like a static feature, and when it isn’t, it’s for a good reason that import doesn’t solve and just expects you find a harder solution to. For example, if you want to load a platform specific file depending on your host environment. With import, the entire function now has to become needlessly async just because any use of the import expression needs to be async. Another good example is modules that put their requires inside the calling function to avoid needlessly increasing the startup time of an app for a feature it may not use. This way, only if you call that specific function will you have to suffer the require/parse/runtime hit for it. Notice all these cases are in node, so they wouldn’t result in some complicated decision as to whether to include these “dynamic requires” into the main bundle or not — it just doesn’t come up in bundling since they are use cases that are specific to node. But because of ESM, node now needs to make a bunch of synchronous functions be asynchronous to accommodate a set of restrictions designed with the browser in mind. And again, at the end of the day import does still have an expression form so you haven’t actually resolved the static analysis problem, just made dynamic imports more annoying in non-browser contexts.
That's the whole point, and a very good thing.
These subtly erroneous arguments are the essence of this push. Look, we are maintaining X, Y and Z, and they’re unable to do that R, so it’s bad. No, it’s you making them unable to do that consciously.
"require()" can take a non-static string. A variable or a string calculated at runtime.
You can only statically check if that particular feature is not used, but there is no checking the entirety of what require() can be used for/with.
require() is more like dynamic imports in Es modules that are awaited and not like the static ES modules.
2) exporting module
The other issue is on the exporting module's side: You can do strange things with the "exports" object. ES module exporting is more strict to make it guaranteed statically analyzable.
It may not be as simple, but I venture it's easier to implement than having all the software ever written needing to be migrated
I can't say how common those forms are, but my point is that there is nothing immediately discouraging a programmer from using them as all of it is "just javascript". So even if you just restrict yourself to static imports, its hard to be sure you caught all of them without running the program.
They're also decidedly inconvenient to use in most situations, because they return a Promise (unlike require()), which means anything that depends on them has to itself be deferred. This further discourages using them unless you really need to.
CommonJS was never going to be natively supported in browsers. The synchronous require semantics are simply incompatible with loading over a network, and the Node team should have known this and apparently (according to members of TC39 at the time) were told their design would not be compatible with future a JS module standard.
So the primary thing that JS modules fix is native support, and for that you need either dedicated syntax or an AMD-style dependencies / module body separation. AMD is far too loose (you could run code outside the module body), so dedicated syntax it is.
Everything else flows from there. I really hate how people blame the standards instead of the root cause which is Node not having taken the browser's requirements into consideration. Culturally, I think that's mostly fixed now, but it was a big problem early on in Node's evolution.
Loading all the dependencies over the network to me is just inefficient, you will have hundreds of requests instead of a single one, you will load the full source not a minified and optimized one, I just don't see the point.
And yes, in production you probably will want to bundle, but you probably also want to minify. Does that imply that we should require a minifier to even run any code at all, even in dev? No, of course not.
By adding a standard and native support we allow for sites that work without bundling and bundling that can adhere to the standard and not have to even be configured because the input is standard and the output must preserve those standard semantics. That gives tool independence and simplifies usage of the toolchains, and that's a great goal to shoot for.
So, isn't adding ESM to Node is strictly a new feature? What am I missing?
Web standards are goddamn late for 20 (twenty) years and it’s not their moral right to decide what should be broken or deprecated, sitting on the top of the mountain of working code which a workhorse named “node” produced in less than a decade.
1. Asynchronous dynamic import() vs. blocking require(): allows the program to continue while a module is being dynamically loaded.
2. Circular dependencies: ESM correctly resolves most of them, while CJS does not. [example below] I believe this is possible because ESM top-level imports and exports are resolved before JS execution begins, while require() is resolved when called (while JS is already executing.)
3. Reserved keywords `import` and `export` vs. ordinary identifiers require, exports and module: Allows tooling to be simpler and not have to analyze variable scope and shadowing to identify dependencies.
I haven't really encountered #3, but I can say I've benefited from #1 and #2 in real-world Node.js projects using ESM.
----
Circular dependencies example:
// a.js
const b = require('./b.js');
module.exports = () => b();
// b.js
const a = require('./a.js');
module.exports = () => console.log('Works!');
a();
One fallacy the author falls for is that they think one needs a build step "anyway" because otherwise there would be too many requests to the backend.
Loading an AirBnB listing causes 250 requests and loads 10MB of data.
With a leaner approach, using ES Modules, the same functionality can be done with a fraction of those requests. And then - because not bundled - all the modules that will be used on another page will be cached already.
I use ES Modules for all my front end development and I get nothing but praise for how snappy my web applications are compared to the competition.
If you already have a build system, the most sensible thing to me is letting the build system do their stuff and not worry about it. When I write a web application in React with TypeScript (the setup that I usually use) I don't worry about dependencies, and I use the ES modules import syntax (that is better than the CommonJS one) that gets transpiled to CommonJS without I even notice. So why bother changing that? It works, it produces a minified and optimized single .js file that is easy to serve from a webserver, I don't see points against it.
How many of these requests are dependent? Lazily loading hundreds of images doesn't impact page responsiveness, but loading an import of an import of an import before your page does anything is unacceptable.
> I use ES Modules for all my front end development and I get nothing but praise for how snappy my web applications are compared to the competition.
So you actually ship unbundled ES modules? How much code is that? I dare you to bundle it up (rollup/esbuild) and tell me that doesn't improve load times. Comparing to the average website overloaded with crap is a very low bar.
tell me that doesn't improve load times
Either you only bundle what is needed on the current page. Then the next page will load slower because it needs to bundle all those modules again as it uses a slightly different set of modules.
Or you bundle everything used on any page your users might go to during their session. This will give you a giant blob that has to be loaded upfront which contains a ton of modules the user never need during their browsing session.
Why isn't anyone else mentioning this feature? I'm not a browser developer but this seems like a clear win, and indeed makes bundling unnecessary. I'm assuming that it's shared between domains, too - or are people's dependencies so fragmented that there's basically no sharing between domains?
Browsers fixed this by making the browser cache no longer shared between domains.
> As of Firefox v85 and Chrome v86 the browser cache will be partitioned, this means that the same resource included on two sites will have to be downloaded from the internet twice and cached separately.
Source https://www.peakhour.io/blog/cache-partitioning-firefox-chro....
Word up. Share a link? I feel like the pro-ESM crowd shoudl stick together!
> And then there's Rollup, which apparently requires ESM to be used, at least to get things like treeshaking. Which then makes people believe that treeshaking is not possible with CommonJS modules. Well, it is - Rollup just chose not to support it.
Rollup was created specifically for ESM. It's not been thrust onto the ecosystem or into anyone's tool chain. One uses it specifically for ESM, and plugins that bolt on for added functionality if they apply. Trying to hammer a nail with a paintbrush doesn't make the paintbrush a bad thing - you just chose the wrong tool.
I wasn’t able to use the latest version of node-fetch in a node.js script since it doesn’t support commonjs. The project literally has “node” in the name and doesn’t support default node.js.
Since you're using it for an async operation anyway, dynamic import should have worked quite well.
This is one of those 'worse is better' things in language design, I believe. It guarantees simplicity, traded off against extra verbosity. In fact, when it comes to the common and probably most valuable case of reading and understanding code written by others quickly, it is not even a tradeoff really, as both are good.
Whether or not that was one of the driving reasons, it certainly is a benefit in my opinion. The two examples given in the post of an inline require don't demonstrate this well, as they're both really simple. I'd say the benefit isn't to stop things examples like that being written and replace them with two lines of code, which admittedly might sometimes be slightly cumbersome. It's that it stops the long tail of much more complex/unreadable statements being written.
For what it’s worth, whether it’s bitten you or not, every single instance of my sillycode[1] is in use in Jest (granted in obviously more useful ways). And it’s an enormous headache to debug when it goes wrong. A trivial example: require a logging library which creates a singleton at module definition time and provides no teardown API (yeah that sounds like a bad design but believe me they exist, are easy to find, and hard to replace on a busy and/or opinionated team). If you have a single suite with 100 tests, Jest will leave 100 instances of that singleton running and consuming memory even while totally idle, completely inaccessible to most any machination you might come up with to try to free them.
Which isn’t to say ESM doesn’t have this same problem if you try to bust the import cache with eg query parameters. But at least you’ll probably notice it’s a problem because you’re very probably doing it directly and not with some opaque Babel transform that hijacks the entire module system and any code referencing it.
1: https://news.ycombinator.com/item?id=29140847
Edit: forgot which sub thread I was in, added link to my sillycode
And with top-level await the restriction goes away (albeit the ESM equivalent is still a bit more verbose).
(await import('anything'))(...yup)1. --experimental-loader is more complex and less stable than --require. But it’s also a lot more robust.
2. There’s no equivalent to the require cache, which makes mocking and long running processes like watch mode challenging. This is partly a benefit, as it discourages cache busting patterns like those used in eg Jest which create awful memory leaks.
module.exports = {
get foo() {
const otherModule = require('equally-dynamic-cjs')
if (otherModule.enabled) {
return any.dynamic.thing.at.all
}
},
get bar() {
this.quux = 'welp new export!'
return 666
},
now: 'you see it',
}
setTimeout(() => {
console.log(`now you don’t!`)
delete module.exports.now
}, Math.random() * 10000)
if (Date.now() % 2 === 0) {
module.exports = something.else.entirely
}
Edit: typed on my phone, apologies for any typos or formatting mistakes.
I say “need” because Typescript wont ingest types from “required” files, you have to import them as modules.
So before we converted a single file to TS we has to audit all commonjs imports and exports to convert them to ES modules.
I agree wholeheartedly that the end result was a fools errand. I would have rather spent the time adding support for importing types via a require which for some reason returns any “any” today.
There is currently no non-hacky way for using both legacy modules and ES Modules in the same project, and many libraries on NPM have moved to ESM-only. TypeScript's transpilation needs to know what to target as regards modules and JS version, which makes things even crazier than they already are.
This is like saying "binding two pieces of wood together is terrible" and using the fact that screwdrivers are poorly designed as your main argument.
I tried doing this with one of the new fangled frameworks and seeing my browser work through like 5000ish required files was quite comical.
If he says there is a problem, he didn't invent it. He knows his stuff
My solution in NodeJS programs for now is to use an `esbuild` -based require hook to transpile all the files we import or require into CommonJS on the fly. We need esbuild anyways to run Typescript code without a build step, and combined with basic mtime based caching, it’s fast enough that you really don’t notice extra build latency especially on a second run — much MUCH faster than a Babel require hook.
I plan to tune back into this issue once the average comment is more measured and thoughtful, and the ecosystem tooling for dealing with the migration has evolved more.
We are using typescript which is using "import" syntax, but as far as I know, it's still transpiling to good old "require".
ES modules are part of the Ecmascript spec. DENO uses them. Node.js modules aren't part of the ES spec.
1) JavaScript could have simply stolen an already good solution. For example namespaces (ex: Clojure/Script, Typescript, even PHP to some degree) provide a powerful mechanism to modularize names - by disentangling them from loading code. They make it straight forward to avoid collisions and verbose (noisy) names. In Clojure namespaces are first class and meant to be globally unique. This implies long-term robustness.
2) Loading modules dynamically should be the _default_. The whole point of JavaScript is that it is a dynamic language. The caveats, hoops and traps that we have to sidestep to for example run a working, real REPL during development is astounding. If you want to be dynamic, go _all_ the way and understand what that means. Yes, it's a tradeoff to be a dynamic language, but why take the worst of both worlds?
3) Like 'async/await', 'class' and many browser features such as IndexedDB it is neither designed from first principles nor fully based on past wisdom. Many things in the JS world smell of "lowest common denominator". Way too much effort is focused on the convenience side of things and way too little on the leverage side.
That's not true with skypack, right?
I really like ESM because I like where it's trying to steer the community of browser application builders. I think front-end builds are terrible on many levels, not the least of which is the obfuscation of code that undermines one of the best features of the web's software distribution, which is its openness. And another major benefit of webapps is that none of the front-end languages require a build step! This makes iteration very fast; if you can make do without the the safety net of a compiler, you can enjoy the speed of not using the bundler.
1) on the client, across the network, one roundtrip for every level of depth, or 2) in a build environment, directly on the filesystem
Option 2 means you need some kind of build tool to make it work, and by that point it doesn't really matter anymore whether the tool just traverses the dependencies and makes a list of filenames, or also concatenates their contents into a bundle.
And that is why the fundamental premise of ESM cannot work; there are no technical options besides those two. If you want to avoid network roundtrips, you must have build tooling. No way around it.
A webmaster could totally avoid the complexity of learning a JS tool chain. Right now even 1000 lines of JS has you reaching for a bundler. It would make shipping a small html+css+js site again as simple as dragging files to your webserver.
only a bundler can fix that (or a hypothetical process which would set up preloads for all required libraries, but that would just be a bundler without the actual bundle creation step)
[1] https://medium.com/@nikita.malyschkin/the-future-of-web-depl...
My inner conspiracy theorist suspects that maybe the powers that be don't want to allow plain JavaScript to extend its primacy over the web. The way things went makes no sense. Computing dependency trees on the server-side and using it to optimistically push scripts to the browser would have been be far simpler and less hacky than computing source maps, for example. Optimistically pushing server-side resources was supposed to be the whole point of HTTP2...
Plus transpilers are so slow, it's embarrassing (albeit things are improving with tools written in rust).
As someone who's been doing frontend for 20 years and node.js for 10 years, JS development has never been so crap like now.
After attending a conference talk about how the TC39 works, I understand why that's the case. TC39 is basically a bunch of engineers from big tech companies who can afford to waste productivity to follow the whims of whatever the group decide. It's completely detached from reality.
They operate on a full consensus basis, which means everyone needs to be onboard with the decisions - and if you want your changes to be approved in the future, you'd better play nice with the current change as well.
To be honest, I can't wait until browsers get replaced with some native crossplatform toolkit or frameworks in other languages become popular so that we can finally leave JS alone.
That sounds like Flutter to me.
Working on large projects and having everything first loaded and then you can load it in the browser is a waste of time that every web developer has every day.
Some real world times FOR DEVELOPMENT: Storybook first load: 90 sec, Storybook after first load changes: 3 sec, Vue App first load: 63 sec, Vue app change after that: 5 sec, Vue App with Vite first load: 1sec, Vue App with Vite after that: the time it takes me to press command+tab to switch to the browser
Do we really have people that use unminfied unbundled esm in production? If Yes, please comment why?
I would also ask the author what about cyclic dependencies? ES Modules resolve them automatically. Something which in large code bases can happen.
Why do we still put it through babel? Because most of us don't have the luxury of not supporting old browser... https://caniuse.com/?search=modules Even if the not supported browsers for our company is 1% it is still a big chunk of money in the end.
and this example: ``` app.use("/users", require("./routers/users")); ```
Really? this is "good code" having a require in the middle of a file?
Also funny: The author is annoyed that rollup did not support tree shaking in commonjs and then complains that people are wasting time on esm. Maybe the rollup team does not want to waste time on commonjs? Also then he points to a package which did not got any update in 3 years and would make the hole process he complains is to complex even more complex by introducing a new dependencies.
Sorry but the more I read that thing the more it sounds to me like a Junior Dev that does not want to learn new things and just likes to rant about things.
If you are having very slow build times with your existing toolchain, the problem isn't the bundling, which is an extremely fast operation. It's almost certainly going to be one specific computationally-intensive plugin that you either don't need, or would also need if using ESM.
These heavy plugins are usually for old browsers to also run in them. That is the only job ob babel.
CJS has some deeper problems. - Check if your fav CJS lib freezes the objects? - ESM is more http friendly (mime type)
All of this js-related stuff is just a fast fashion, the bad part is web developers are locked into it with no chance to relax and to just create their boring services and apps.
That said the problem isn’t modules are all. It’s reliance on a forest of legacy nonsense. If you need a million NPM modules to write 9 lines of left pad these concerns are extremely important. If, on the other hand, your dependencies comprise a few TypeScript types there is nothing to worry about.
So it’s a legacy death spiral in the browser. Many developers need a bunch of legacy tools to compile things and bundles and build tools and all kinds of other extraneous bullshit. Part of that need is to compensate for tooling to handle modules that predates and is not compatible with the standard, which then reenforces not using the module standard.
When you get rid of that garbage it’s great. ES modules are fully supported in Node and the browser.
I think the true anger is that something so essential and basic to JS development has this giant breaking change if you want to switch over to ESM - there's no reverse compatibility or fallback - it just breaks.
So when you dig even deeper this is really a people and training problem.
Instead I'm gonna try to go back to the topic.
I think that in practice these are my pain points in using ESM regularly without any build tool. I'm using ESM modules both in node.js and in the Web Browser via <script type module>:
- package.json/exports "hack" works only in node.js and not in the Browser as there's also no equivalent API available. This hack allows to namespace entry points for your library, so that you can use "import foo from 'bar/qux';" without having to use "../../../" fatigued paths everywhere (that also might be different in the Browser compared to the nodejs entry points).
- "export * from '...';" is kind of necessary all the time in "index" files, but has a different behaviour than expected because it will import variable names. So export * from something won't work if the same variable name was exported by different files; and the last file usually wins (or it throws a SyntaxError, depending on the runtime).
- Something like "import { * as something_else, named as foobar } from 'foo/bar';" would be the killer feature, as it would solve so many quirks of having to rename variables all the time. Default exports and named exports behave very differently in what they assign/"destruct", and this syntax would help fix those redundant imports everywhere.
- "export already_imported_variable;" - why the HECK is this not in the specification? Having to declare new variable names for exports makes the creation of "index" files so damn painful. This syntax could fix this.
You can do:
export {already_imported_variable}Your solution will work only once in a file, therefore it is useless to batch-export lots of imports for the mentioned use case of an "index" file that exports all your classes and definitions.
Not to mention the security aspects: there is no subresource integrity for imports, so it’s less secure than bundling or using a script tag with CDNs.
The point about it being a new syntax is also very valid. Everything import patterns do is almost identical to destructuring, so we should have just extended that feature instead, especially because I do wish destructuring could do those things. For example, if destructuring had an “everything” pattern to complement the “rest” pattern:
const { x, …rest, *original } = something();
The semantics of imports are also very confusing to beginners, as they implement yet another unique form of hoisting. It is so weird that despite being allowed anywhere in your code, they run first. Notice I didn’t say they fetch first, they run first. So for example, the following code is broken:
process.env.S3_LOCATION = “https://…”; // The below library expects this as an environment variable.
import download from “s3-download”;
import unused from “./set-env-variable.js”
import download from “s3-download”
Anyways, the list of problems actually continues, but there is widespread acceptance that this feature would not have been accepted in its current form if introduced today. But for some reason everyone just takes a “but it’s what we got” position and then continues piling more junk on top of it making it even worse.
That build step is ideally performed by something like rollup or esbuild, which means I use import/export anyway. If you still use Babel, I feel bad for you son, I've got 99 problems but Babel ain't one. I don't care if the old stuff is not supported, simply deleting 98% of the code in the JS ecosystem would be a step forward. Perhaps that's a minority view, but none of these arguments fly with me.
1 file per thing is midwit code organization strategy for people with no actual sense for it.
on a personal note, im making a cms with es modules and couldnt be happier.
Breaking backwards compatibility is always painful but there's not one actual criticism of ES Modules as a spec here other than its incompatibility with CommonJS
What are the advantages of ESM that led to selecting that over CJS modules for standardization in the first place?
That might sound irrelevant on the face of it, but it has
very real consequences. For example, the following pattern
is simply not possible with ESM:
const someInitializedModule = require("module-name")
(someOptions);
Or how about this one? Also no longer possible:
const app = express();
// ...
app.use("/users", require("./routers/users"));
import someModule from "module-name"
const someInitializedModule = someModule(someOptions)
const app = express();
app.use("/users", (await import("./routers/users")).default);
Actually, the first example could be rewritten as
const someInitializedModule = (await import("module-name")).default(someOptions);
import { createRequire } from 'module';
const require = createRequire(import.meta.url);
const cjsOrJson = require('./somewhat/module/pathway');What do you mean?
The rest of the text can mostly be summarized as https://xkcd.com/927/
People, especially a few years ago, were trying to get clever with the require calls, were fiddling around with require cache and while with ESM we no longer can “easily” do stuff like dynamic reloads, I genuinely feel it’s for the better.
I strongly agree with privatenumber’s point that import() syntax is the true first class citizen here.
This.
Also, the bulk of the rant is focused on how the author struggles with configuring his pick of JavaScript bundlers and transpilers, and proceeds to come up with excuses to justify not migrating away from CommonJS .
This article was a waste of a perfectly good click.
// ...
if (condition)
{
const x = require('../../../hugeFuckingLibraryThatTakesSeveralSecondsToLoadUponColdStart')
// do something with x
}
// ...
if( condition ){
import('../../../hugeFuckingLibraryThatTakesSeveralSecondsToLoadUponColdStart').then( x => {
//do something with x
})
}
I hate to break it to you darling, but programming is nerd bullshit.
edit: alternate that might too much "nerd bullshit", but uses async/await if the surrounding code is an async function:
async function doSomeStuff()
{
if( condition ){
const x = await import('../../../hugeFuckingLibraryThatTakesSeveralSecondsToLoadUponColdStart');
//do something with x
}
}https://www.sitepoint.com/file-bundling-and-http2/
https://medium.com/@asyncmax/the-right-way-to-bundle-your-as...