My Expertise Modernizing Packages to ESM · Mark’s Dev Weblog

Desk of Contents 🔗︎

For the final 8+ years, the JS ecosystem has been present process a gradual transition in direction of utilizing ES Modules (“ESM”) because the default method for publishing and utilizing JS code. Just like the Python 2->3 transition, this has been extremely troublesome and painful to take care of.

As a bundle maintainer, I need to be sure that my libraries are maximally suitable and usable within the widest array of environments I can feasibly assist. Sadly, this additionally signifies that I’ve needed to turn into conversant in the nuances and habits quirks of quite a lot of completely different construct instruments and runtime environments

Early this 12 months I began engaged on attempting to replace the bundle formatting for the Redux household of libraries to provide them “full ESM compatibility”. I suppose I’ve lastly give you a set of configurations that appear to work fairly properly, nevertheless it’s been a wrestle.

Considered one of my largest frustrations is that there isn’t a single authoritative and complete information on “Tips on how to Publish a JS Package deal Accurately”. I’ve repeatedly begged for some professional who really is aware of what they’re doing to put in writing and publish such a information. Ideally, it might cowl issues like what file codecs to incorporate, the right way to configure ESM/CJS interop and bundle.exports, coping with TS sorts variations, making certain tree shaking, checking for compatibility points, the right way to correctly assist particular construct instruments and what they search for, and so forth. I’ve discovered some guides (which I will hyperlink beneath), however nothing that fairly matches the breadth and contents I have been wishing for.

This submit is not that “authoritative information”. It is a recap of what I’ve tried, and hard-earned classes I’ve discovered alongside the best way. Based mostly on the variety of instances of us have popped up and stated “you are doing this incorrect”, I am positive there’s loads of items I am nonetheless lacking 🙂 However, I hope this data can be helpful and informative even when it isn’t totally complete and authoritative.

There’s loads of articles on the market already recapping the historical past of the ESM spec, the arguments and choices which have led to the present confusion and compatibility points, and the way we bought into this mess. Within the curiosity of maintaining this a considerably manageable measurement, I will attempt to dig up hyperlinks to some of these and record them on the finish, and focus primarily by myself expertise and steps all through this course of.

Redux Packages Background 🔗︎

Packages and Configurations 🔗︎

On the finish of 2022, I maintained and revealed these packages as a part of the Redux org:

Every of those packages had their very own improvement historical past, packaging setup, and construct configuration. Normally:

• All of the packages included ESM, CJS, and UMD construct artifacts (with various mixtures of embedded course of.env.NODE_ENV values, or pre-compiled to "improvement" or "manufacturing" variations)
• All construct artifacts used a .js extension
• All of the packages had been being transpiled to ES5 syntax for IE11 compatibility
• redux, react-redux, redux-thunk, and reselect had been being transpiled with Babel and bundled with Rollup. RTK was constructed utilizing a customized ESBuild wrapper script that did the bundling and first transpilation, but in addition used tsc to decrease ES2015 code to ES5.
• All the packages used the "major", "module", and "sorts" fields in bundle.json. Not one of the packages used the comparatively new "exports" subject for outlining which construct artifacts get loaded.
• A lot of the packages besides RTK output construct artifacts to completely different folders by kind: dist for UMD, lib for CJS, es for ESM

Some examples from these bundle.json information:

{
  "identify": "redux",
  "model": "4.2.1",
  "major": "lib/redux.js",
  "unpkg": "dist/redux.js",
  "module": "es/redux.js",
  "typings": "./index.d.ts",
  "information": ["dist", "lib", "es", "src", "index.d.ts"]
}

{
  "identify": "react-redux",
  "model": "8.0.5",
  "major": "./lib/index.js",
  "sorts": "./es/index.d.ts",
  "unpkg": "dist/react-redux.js",
  "module": "es/index.js",
  "information": ["dist", "lib", "src", "es"]
}

{
  "identify": "@reduxjs/toolkit",
  "model": "1.9.5",
  "major": "dist/index.js",
  "module": "dist/redux-toolkit.esm.js",
  "unpkg": "dist/redux-toolkit.umd.min.js",
  "sorts": "dist/index.d.ts",
  "information": [
    "dist/**/*.js",
    "dist/**/*.js.map",
    "dist/**/*.d.ts",
    "dist/**/package.json",
    "src/",
    "query"
  ]
}

RTK’s setup was extra sophisticated, as a result of it has 3 separate entry factors: @reduxjs/toolkit, @reduxjs/toolkit/question, and @reduxjs/toolkit/question/react. Notice that RTK’s bundle.json did not record the 2 RTKQ entry factors. As a substitute, there was an precise /question folder within the revealed bundle, with /question/bundle.json and /question/react/bundle.json information that in flip pointed over to the precise artifacts within the dist folder. (This setup was the results of appreciable experimentation, aka “it appears to work with Webpack and a pair different instruments I believe???”).

Whereas indirectly related for the remainder of the story, I would like to provide a shout out to 2 extremely helpful instruments that I exploit as a part of the publishing course of:

• release-it: automates the precise steps for publishing to NPM, together with Git tagging and pushing
• yalc: allows you to do a full native “publish” of a bundle with the intention to check out putting in it into instance initiatives. Avoids points with symlinks (ie npm hyperlink), and exams out the actual construct and publish steps.

Challenge Historical past 🔗︎

In mid-2021, we acquired a problem reporting that RTK couldn’t be loaded correctly in each consumer and server code concurrently. In early 2022, an analogous challenge reported that RTK couldn’t be accurately imported in a .mjs ESM file, due to make use of of "module" however no "exports" subject in bundle.json. Lastly, one other challenge famous that RTK did not work with TypeScript’s new moduleResolution: "node16" possibility.

I would beforehand requested round concerning the implications of including "exports" to a bundle, and I would been informed that “this qualifies as a breaking change”. That meant that I could not start to think about doing it till the following main launch for every of the packages. However, I had no thought once we’d get round to publishing majors. Redux 4.0 got here out all the best way again in 2018, and RTK 1.0 in late 2019. React-Redux 8.0 was more moderen, in mid-2022.

The Redux core had really been transformed to TS in 2019, however we might by no means shipped it. 4.x and its hand-written typedefs labored, and we had considerations about potential ecosystem churn from transport a 5.0 main. We additionally had loads of function work to do with RTK.

We shipped RTK 1.9 in November 2022. After taking a pair month break, I lastly sat down to begin severely engaged on attempting to modernize our packages.

Early Makes an attempt 🔗︎

I would been stockpiling a listing of tabs and articles round “publishing fashionable JS”, ESM, and ESM/CJS interop for the previous couple of years, figuring out that today would come finally. (Ultimately examine, I had roughly 175 articles in that record!).

I reviewed a number of of these articles to determine my preliminary steps. From what I learn, I concluded that:

• I wanted so as to add "kind": "module" to my bundle.json information, to ensure that Node and bundlers to detect the bundle as containing ESM information
• I additionally wanted so as to add the "exports" key to bundle.json, and add keys inside that listed the doable entry factors and what construct artifacts to make use of when imported in several environments

I would seen mentions of utilizing .mjs as a file extension to drive Node to acknowledge a given file as being an ES Module. To be trustworthy, I felt that seemed ugly and ridiculous, and I did not need to use that extension in any respect.

My first try was RTK PR #3095: Migrate the RTK bundle to be full ESM. Per my PR description, it contained these adjustments:

This PR makes an attempt to transform the RTK bundle from its current “comprises ESM and CJS modules, however not totally ESM”, to be totally ESM with {kind: "module"} and nonetheless assist CJS:

• BREAKING: Units the principle RTK bundle.json file to be {kind: "module"}
• BREAKING: Updates all entry level bundle.json information to make use of exports to level to the categories, ESM file, and CJS file
  • I nonetheless have major and module in there as a result of WHO KNOWS WHETHER THOSE STILL END UP GETTING USED BY SOME TOOLS OR NOT 🤷‍♂️
• Updates the construct script:
  • Mounted ESM compat on execution by changing use of __dirname and fixing the Terser import
  • Switched all construct targets to be "esnext", to make sure the output is untouched apart from TS transpilation
  • Moved all CJS construct artifacts to be nested a degree deeper in every entry level in a ./cjs/ folder, ie, ./dist/question/cjs/
  • Added {kind: "commonjs"} bundle information to these folders
  • Turned off the UMD construct artifacts for now

The ensuing bundle.json seemed like this:

{
  "identify": "@reduxjs/toolkit",
  "model": "2.0.0-alpha.1",
  "kind": "module",
  "module": "dist/redux-toolkit.fashionable.js",
  "major": "dist/cjs/index.js",
  "sorts": "dist/index.d.ts",
  "exports": {
    "./bundle.json": "./bundle.json",
    ".": {
      "sorts": "./dist/index.d.ts",
      "import": "./dist/redux-toolkit.fashionable.js",
      "default": "./dist/cjs/index.js"
    },
    "./question": {
      "sorts": "./dist/question/index.d.ts",
      "import": "./dist/question/rtk-query.fashionable.js",
      "default": "./dist/question/cjs/index.js"
    },
    "./question/react": {
      "sorts": "./dist/question/react/index.d.ts",
      "import": "./dist/question/react/rtk-query-react.fashionable.js",
      "default": "./dist/question/react/cjs/index.js"
    }
  }
}

I did attempt testing out native builds in Vite, CRA4/5, Subsequent, and Node, in addition to operating the publint instrument. Issues appeared to principally work regionally, so I put up the PR to see what would occur with CI

AND EVERYTHING EXPLODED!!!! 💣💣💣

I spent just a few extra hours twiddling with issues, and reported my findings:

Nicely. The excellent news is I believe the runtime code works.

Unhealthy information is Jest is being a ache. Particularly, one thing about the best way it is importing redux-thunk makes the default import an object like {default}, which isn’t a middleware perform, and thus the exams explode once we attempt to create a retailer.

I spent the final couple hours hacking round with the thunk exports and republishing it regionally. Switching the thunk bundle over to not having a default export in any respect sorta helped, however now one thing concerning the “no dev middleware in prod” check is failing.

So, shut, however cannot even construct this department but.

edit

the place I left off yesterday was that:

• redux-thunk has a default export, and Jest was now choking on that
• I revealed redux-thunk@3.0.0-alpha that attempted to transform it to ESM, however nonetheless had a default export included. That helped construct with Subsequent, however not our native Jest exams
• I then did a local-only publish of redux-thunk that dropped the default export and solely had named exports. That truly appeared to assist, however certainly one of our RTK exams that asserts NODE_ENV=manufacturing habits for getDefaultMiddleware was breaking

Some of us in Reactiflux prompt that it is perhaps value investigating a change to Vitest. I would desire not to do this if doable, on the grounds that migrating to a unique check runner shouldn’t be my precedence or one thing I need to spend time on. Alternatively, it is also one thing that may be useful basically and for this particular drawback.

Migrating to Vitest 🔗︎

I actually did not need to burn time attempting emigrate our total check setup from Jest to Vitest. However, I would heard loads of optimistic feedback about Vitest, together with that it ran considerably sooner and had higher ESM assist.

A pair days later I made a decision to provide it a shot. To my nice shock, the conversion was pretty simple.

I ended up with RTK PR #3102: Migrate RTK check suite from Jest to Vitest.

The primary check setup made sense, and the jest.fn() -> vi.fn() swaps had been simple. The most important ache level I bumped into was the place we tried to mock the redux bundle to claim that configureStore was calling via to the core library. Needed to do a bunch of twiddling with vi.mock() till one thing lastly appeared to work. Alternatively, timer habits appeared to work extra persistently.

As a part of this course of, I discovered myself additionally needing to transform different auxiliary information within the repo to ESM syntax, equivalent to Jest config information and construct scripts with a .js extension.

I used to be capable of get that PR passing, and merge it a pair days later.

Preliminary Alpha Testing 🔗︎

I revealed @reduxjs/toolkit@2.0.0-alpha.1 on January 21. This included the RTK packaging adjustments, in addition to an analogous change to redux-thunk, and modernized the construct artifacts to now not transpile any JS syntax and drop IE11 compat.

In fact, this didn’t work in addition to I would hoped 🙂

Mateusz Burzyński ( @andaristrake ) maintains a number of libraries, together with Emotion and Preconstruct, and spends a lot of his time engaged on the TypeScript compiler for enjoyable. He is an professional on lots of the intricate nuances of packaging formatting.

After I introduced alpha.2 on Twitter, Mateusz replied with a number of strategies for tweaks ( each RTK 2.0 and Redux core 5.0 alphas):

no thought what dist/es/redux.mjs is for now if it isn’t even within the exports map

by utilizing the categories situation like this TS may at all times assume that this can be a module, even when loaded/required from CJS goal~. Since you do not have a default export… that is in all probability nice

I would come with the module situation and level to the ./dist/es/index.js with it, it will permit the bundle to be solely loaded as soon as by bundlers, regardless of the consuming file’s format (cjs vs esm)

eliminate course of.env.NODE_ENV in your dist information, use improvement/manufacturing circumstances to perform these items (in all probability finest to make manufacturing the default and the event stuff an opt-in)

I saved these as a problem for reference.

Shortly thereafter, we acquired a few new challenge stories back-to-back complaining of issues with the config in alpha.1/2:

When importing something from @reduxjs/toolkit@2.0.0-alpha.2, typescript can not resolve sorts when tsconfig.json's moduleResolution is about to "node16" or "nodenext".
I discovered that including the extension .js to the declaration imports resolved the difficulty.

The present config within the Alpha doesn’t permit for consumption of the CJS bundle in fashionable model of Node/any instrument that follows Node’s module decision spec, because it makes use of .js to confer with a CJS module regardless of the bundle setting "kind": "module". If "kind": "module" is about, .cjs is critical in "exports".

Some bundlers do work round this and are extra forgiving (whether or not or not that is an excellent factor is one thing else totally), however this config is not going to work in Node and/or any environments that comply with its decision mechanism.

I used to be… not a cheerful camper:

I do actually respect you submitting the difficulty, however I’m additionally legitimately getting indignant at how tousled this complete state of affairs is 🙁

I need to do proper by my customers and assist the number of construct instruments and environments I count on they’re going to be utilizing for his or her apps.

However I can not do this if each single article and individual is giving me contradictory directions on what I am purported to do 🙁

Clearly this was going to take quite a lot of effort to determine what was occurring and catch doable errors.

Researching Higher Configuration 🔗︎

Proper on the identical time, Devon Govett tweeted about bettering Node+ESM assist in a React-Aria bundle replace.

I replied noting I used to be engaged on some related efforts, and tagged Mateusz. Shortly after that I noticed the problems get filed, linked and griped about them, and Mateusz once more prompt eradicating kind: "module".

I used to be feeling annoyed and begged him to publish a full weblog submit that may give particulars on his suggestions. As a substitute, he prompt we do a cellphone name and speak via issues instantly.

On February 27, Mateusz and I hopped onto a name together with Nathan Bierema (Redux DevTools maintainer). I saved the dialogue notes in a gist:

Mateusz threw out quite a lot of ideas round how ESM and CJS can get used, and questioned whether or not it even totally is sensible to ship ESM in any respect. The knowledge was helpful basically, nevertheless it left me nonetheless feeling fairly confused about subsequent steps.

Someplace in that Twitter dialogue, I bought in contact with Andrew Department (@atcb), a TypeScript crew member who had applied the brand new moduleResolution: "bundler" possibility for TS, and has been doing work on JS/TS ecosystem module habits as preparation for writing. We arrange a name on February 28.

Andrew gave me a rundown of how ESM works, how TS treats ESM and module import paths, and the way Node and different instruments decide if a file is definitely ESM.

The TL;DR of that final half is roughly:

• In case you add kind: "module", each file with a .js extension will get interpreted as ESM, interval. .cjs information can be interpreted as CommonJS.
• Alternately, for those who do not have kind: "module", .js information are handled as CommonJS. You should utilize .mjs to mark particular person information as ESM.

There was additionally some dialogue of whether or not or not we ought to be pre-bundling our TS typedefs or leaving them as particular person someSourceFile.d.ts information within the revealed bundle.

Setting Up CI Checks for Packaging 🔗︎

Preliminary CI Setup 🔗︎

I would suspected for a very long time that I used to be going to finish up needing to place collectively some sort of battery of instance purposes, every utilizing completely different construct tooling, to catch doable errors in packaging throughout PR CI checks.

After the alpha.2 challenge stories, I reluctantly concluded I actually wanted to spend time establishing these CI checks earlier than I did any extra work on the precise bundle configurations.

As a part of my preliminary testing, I had regionally created a small instance app that exercised all of RTK’s entry factors. It had a counter to train configureStore and createSlice from the core, a UI-agnostic RTKQ createApi endpoint, and a React-specific RTKQ createApi endpoint. I would pasted that into a number of completely different challenge setups.

We already had our CI set as much as pre-build a tarball containing the bundle contents from the present PR, and had been utilizing that to run our unit exams in opposition to the PR bundle model as an alternative of our supply code. I made a decision to attempt increasing on that to check these completely different apps in opposition to that PR construct as properly.

I copied the primary couple instance initiatives into a brand new $REPO/examples/publish-ci/ folder, and up to date the GH Motion workflow to matrix the folder names inside /publish-ci/, set up the PR construct into every instance, then construct+check it:

I additionally wrote a small Playwright check that may examine the web page contents to confirm it might change the counter, and that each API endpoints had fetched the precise mock knowledge, as a way to make sure that the apps really ran accurately.

I really focused the PR in opposition to 1.9.x on our grasp department, to see how issues labored with the prevailing bundle setup first.

I ultimately ended up with instance apps that coated:

• CRA 4 (with Webpack 4)
• CRA 5 (with Webpack 5)
• Subsequent.js (with Webpack 5)
• Vite 4
• Node in each CJS and ESM modes

There have been loads of different construct instrument mixtures that in all probability must be checked, however this can be a good begin.

Are The Sorts Incorrect? 🔗︎

Someplace on this course of, I had found that Andrew Department had created a instrument known as Are The Sorts Incorrect. It is a web site that permits you to choose a printed NPM bundle model, or add a .tgz file, and analyzes the bundle exports to report how TypeScript interprets the configuration and whether or not the JS information and TS typedefs match up accurately. It then exhibits all of your detected entry factors, and stories particulars on any mismatches and errors.

This is an instance of the report for RTK 2.0.0-alpha.2 ( https://arethetypeswrong.github.io ):

You possibly can see that it detected all of RTK’s entry factors, and that a lot of the entry level + moduleResolution mixtures look okay. However, moduleResolution: "node16" + some CJS atmosphere apparently has a number of points.

I actually wished to make use of this evaluation in RTK’s CI to assist confirm that any future PR adjustments would really work accurately. I checked out the attw repo, and famous that Andrew had cut up it into core and web site packages. However, the core logic wasn’t but revealed as a bundle.

I initially tried establishing a CI job that may clone the attw repo, and let me write a command-line script that may import the core logic and analyze the PR construct artifact. That technically labored, however thankfully I used to be capable of persuade Andrew to publish the logic as an precise @arethetypeswrong/core bundle.

From there, I put collectively a CLI script that ran the core attw logic, collected the stories, and wrote it out as a console desk to match the show on the web site. I did this utilizing the ink React CLI renderer (and doubtless spent a bit an excessive amount of time twiddling with rendering tables within the console). The outcomes ended up fairly good:

I then configured RTK’s CI to name that as one other examine alongside constructing the instance apps.

There was an current thread asking for attw so as to add a CLI, so I supplied mine up as a possible place to begin. (Another person later filed a PR so as to add a CLI. That CLI has now been revealed formally, and I have to get round to switching over to utilizing that in CI as an alternative of my homegrown script.

Packaging Updates, Spherical 2 🔗︎

I made a decision it was finest to attempt updating the smaller packages that RTK depends upon first.

I would already needed to publish a 3.0-alpha.0 for redux-thunk to change its habits. I made a decision I would change over and check out making additional updates with that.

Switching Construct Tooling 🔗︎

redux-thunk is a single tiny supply file about 20 strains lengthy (plus some further TS sorts). That made it an excellent place to begin to mess with altering packaging.

I famous that it was nonetheless utilizing Babel+Rollup for the construct step. I made a decision I would attempt utilizing ESBuild as an alternative. However, how ought to I exploit that?

We already had a customized ESBuild wrapper script over in RTK. I briefly thought-about copy-pasting that over to the redux-thunk repo, however determined it might be overkill.

I would performed some earlier trying to find different ESBuild wrappers. I did some extra wanting and determined to provide https://github.com/egoist/tsup a shot.

tsup really labored out fairly properly! In a pair hours I had a easy tsup.config.ts file that generated the 2 artifacts I wished. On this case the thunk code had no dev/prod conditional checks, so I stored it actually easy – only a single ESM and CJS file apiece.

I additionally eliminated kind: "module", and switched to utilizing .mjs and .cjs for the artifacts to drive ESM or CJS appropriately.

I up to date the thunk bundle.json file to make use of these:

{
  "identify": "redux-thunk",
  "model": "3.0.0-alpha.1",
  "major": "dist/cjs/index.cjs",
  "module": "dist/index.mjs",
  "sorts": "dist/index.d.ts",
  "exports": {
    "./bundle.json": "./bundle.json",
    ".": {
      "sorts": "./dist/index.d.ts",
      "import": "./dist/index.mjs",
      "default": "./dist/cjs/index.cjs"
    },
    "./extend-redux": {
      "sorts": "./extend-redux.d.ts"
    }
  }
}

In the meantime, I ended up doing all the identical Jest->Vitest conversion because the RTK repo.

UMD Construct Artifact Adjustments 🔗︎

I additionally spent a bunch of time debating whether or not it was value maintaining UMD information or not. redux-thunk had shipped with a UMD bundle, primarily to be used as a script tag (which I assumed was principally being performed in CodePens or related examples).

I’ve repeatedly requested about whether or not to preserve publishing UMD builds during the last couple years.

The closest I bought to precise recommendation and solutions had been:

• Fred Ok Schott: “HTML examples and code editors aren’t even causes to make use of UMD anymore, on their very own. Ex: @CodePen ships with a built-in Skypack integration”
• Marvin Hagemeister: “I believe it is nice to skip UMD. All bundlers can eat ESM, and with websites like https://esm.sh that may be simply used within the browser.”

So, I made a decision it was lastly time to drop UMD builds from the Redux packages 🙂

Later, I made a decision the perfect substitute for UMD was to incorporate one other ESM-format construct artifact that was pre-compiled to manufacturing mode and now not had course of.env.NODE_ENV references, in order that it could possibly be safely loaded in a browser. That approach individuals might use it as a <script kind="module"> and cargo it from the bundle as hosted by Unpkg or an analogous CDN.

(I requested for suggestions on use instances for maintaining UMD as a part of our alpha/beta launch notes and haven’t but acquired a single remark… however everyone knows that nearly nobody provides suggestions on pre-release variations anyway 🤷‍♂️)

Webpack 4 Compat 🔗︎

In early March, I noticed a tweet from Dominik Dorfmeister (maintainer of React Question) noting that Webpack 4 nonetheless has extra downloads than Webpack 5. That is principally attributable to current initiatives that use Webpack internally, equivalent to CRA 4, Storybook 6, Expo’s net goal, and many others, in addition to normal ecosystem utilization.

Sadly, Webpack 4 doesn’t assist the bundle "exports" subject, and it additionally can not correctly parse code with elective chaining syntax. Lastly, I additionally discovered that it does not like having a .mjs file within the "major" subject both, and wishes a .js extension as an alternative.

A part of my objective for these main variations was to cease transpiling away any of our JS syntax, solely transpile TS sorts, and solely ship totally fashionable JS. Nonetheless, I additionally care about giving our customers an excellent out-of-the-box expertise. It was clear that attempting to solely ship modernized code would trigger issues for anybody nonetheless on Webpack 4.

I grudgingly determined that I would come with an extra artifact particularly for Webpack 4 compat: ESM module format, transpiled to ES2017 syntax, and utilizing a .js extension, and level to that within the "major" subject.

Immer 10 Beta 🔗︎

Michel Weststrate, creator of the Immer immutable replace library, had stated again in January that he deliberate to work on Immer 10 within the spring. The main deliberate updates had been round efficiency and dropping legacy ES5 compat. However, Immer additionally had some related packaging points, together with use of each default and named exports. Immer had shipped the addition of "exports" in a 9.x patch launch, just for it to interrupt many customers. The change was tweaked and half-reverted instantly, leaving Immer with an odd bundle config.

I would left a remark in January noting how I would run into points with packages having each default and named exports, so Michel opted to drop the default export in 10.0.

As soon as Immer 10 got here out in beta, I up to date the RTK 2.0 department to rely on that.

TypeScript Declarations 🔗︎

Redux 4.x ships with a hand-written TS typedefs file. RTK 1.x has particular person TS typedefs information per-source-file, generated from sources.

tsup has a dts: true possibility that may name tsc to generate typedefs, and bundle them collectively right into a single file. That labored nice for Redux, however I bumped into some sort of challenge doing that for RTK. I ultimately settled for sticking with particular person per-source typedefs information in RTK.

At this level I had bundle configurations that handed the are-the-types-wrong checks, with one exception: a "FalseCJS" warning for the ESM artifacts in moduleResolution: "node16" mode.

I had some back-and-forth with Andrew Department about this. The issue is that technically you ought to have separate TS typedefs for “my artifacts in CJS mode”, and “my artifacts in ESM mode”, as a result of there may be precise variations in what’s exported and the way that will get accessed.

The method Andrew recommends to repair that is to really compile your challenge with tsc twice, with two completely different TS module settings, and ship two completely different units of typedefs with .d.mts and .d.ts extensions to match your .mjs and .cjs/js artifacts.

Sadly, no construct instrument that I knew of at the moment did this by default, and the concept of transport 99%-duplicate typedefs bothered me. So, I opted to not attempt to repair this "FalseCJS" challenge for our packages (at the least in the intervening time).

Andrew Department later filed a PR for tsup that tries to really output one typedefs file per output format. As of penning this submit, I’ve not but really tried this out myself, however I will give it a shot later and see what occurs.

Notice that Andrew at the moment has a really lengthy gist along with his WIP complete documentation on how TS interprets module codecs, in addition to articles within the are-the-types-wrong repo documenting the entire points it may possibly discover.

Spherical 2 Outcomes 🔗︎

I revealed each redux@5.0.0-alpha.4 and @reduxjs/toolkit@2.0.0-alpha.4 in early April, with a set of bundle configurations that I used to be pretty positive ought to really be “appropriate”:

I ended up with these configurations:

{
  "major": "dist/cjs/redux.cjs",
  "module": "dist/redux.mjs",
  "sorts": "dist/redux.d.ts",
  "exports": {
    "./bundle.json": "./bundle.json",
    ".": {
      "sorts": "./dist/redux.d.ts",
      "import": "./dist/redux.mjs",
      "default": "./dist/cjs/redux.cjs"
    }
  }
}

{
  "module": "dist/redux-toolkit.legacy-esm.js",
  "major": "dist/cjs/index.js",
  "sorts": "dist/index.d.ts",
  "exports": {
    "./bundle.json": "./bundle.json",
    ".": {
      "sorts": "./dist/index.d.ts",
      "import": "./dist/redux-toolkit.fashionable.mjs",
      "default": "./dist/cjs/index.js"
    },
    "./question": {
      "sorts": "./dist/question/index.d.ts",
      "import": "./dist/question/rtk-query.fashionable.mjs",
      "default": "./dist/question/cjs/index.js"
    },
    "./question/react": {
      "sorts": "./dist/question/react/index.d.ts",
      "import": "./dist/question/react/rtk-query-react.fashionable.mjs",
      "default": "./dist/question/react/cjs/index.js"
    }
  }
}

I additionally included the browser-focused ESM artifacts as properly, though they don’t seem to be explicitly listed in there. (I could attempt to deliver them again underneath the "browser" or "unpkg" keys – might want to do extra analysis there.)

I additionally stored the nested entry level bundle.json information within the RTK bundle as a part of the Webpack 4 compat.

These configurations appear to construct and run in the entire pattern initiatives I had configured, though that did not embody a React Native challenge.

Different Package deal Updates 🔗︎

I later utilized the identical bundle updates to reselect@5.0.0-alpha.0.

I wasn’t initially planning on doing a significant model launch for React-Redux. Nonetheless, after seeing another potential breaking adjustments, I concluded it was value doing a React-Redux v9.0 main to incorporate the packaging updates, deal with TS sorts adjustments from Redux 5, and cease utilizing the React 18 useSyncExternalStore shim by default.

As of this writing, I’ve a PR open to replace React-Redux’s packaging, however had run into some TS points and did not get again to attempting to repair these.

Updating Immer’s Packaging 🔗︎

I did some perf checks on Immer 10 beta, and noticed that it was considerably sooner. Nonetheless, I additionally famous that in some way Immer 10 beta appeared larger than Immer 9. I pulled down the Immer 10 beta PR department and ran some construct measurement comparisons.

After dialogue, part of the rise was attributable to Immer’s Map/Set assist now being enabled by default, and Michel agreed to revert that since Redux does not want it and RTK is likely one of the largest customers of Immer. Nonetheless, even with that change, Immer 10 beta was nonetheless larger than Immer 9.

I dug into Immer’s construct system, and discovered that it was nonetheless utilizing the mostly-dead tsdx construct tooling bundle and concentrating on ES6 syntax. This was including a bunch of polyfills and useless code. I additionally discovered some points with the sourcemaps and the listed bundles.

Since I’d simply spent a bunch of time updating the Redux and RTK construct configs, I volunteered to port these adjustments over to Immer as properly. I shortly put collectively a proof-of-concept, and noticed noticeable reductions in Immer’s bundle measurement. I put up a PR for these Immer adjustments, and Michel merged that as a part of v10.

Issues with Subsequent.js and React Server Elements 🔗︎

In early Could, Subsequent 13.4 was launched. The headline function was that the brand new “App Router”, which relies on React Server elements, was now thought-about secure and prepared for manufacturing. As a part of that, Subsequent’s CLI defaulted to creating new initiatives with the App Router and /app folder enabled out of the field, and the docs had been up to date to show use of the App Router and React Server Elements because the default.

We quickly started getting a stream of latest points filed in opposition to React-Redux and Redux Toolkit, complaining that React-Redux did not work accurately. The primary stories had been errors being thrown as a result of React.createContext() was null or useLayoutEffect not current in an RSC atmosphere.

In the meantime, my Redux co-maintainer Lenz Weber-Tronic, who additionally works on Apollo Consumer for his day job, had spent the earlier couple months doing analysis into the right way to use client-side state administration and knowledge fetching libraries with RSCs.

One challenge he’d run into was the right way to use RTK Question’s createApi, which might generate React hooks, on each server and consumer, and he’d filed a React challenge asking for strategies on the right way to deal with this.

He additionally wrote an extended RFC for the right way to combine Apollo and Subsequent 13, which later led to publishing an experimental Apollo + Subsequent interop bundle.

In mid-June, an Apollo consumer filed a problem reporting that Apollo broke with Subsequent 13.4.6-canary.2. That was mounted shortly thereafter in one other canary construct, however the challenge spawned an extended and annoyed dialogue.

In that thread, a Subsequent dev reported that Subsequent was now discovering the ESM artifact, might thus higher analyze what was being imported, and the usage of consumer code in server elements was now being thought-about an error. They talked about: “If apollo consumer goes have server elements answer then it must have a separate "react-server" export situation that solely comprise the server solely exports”.

After studying that, I chimed in and famous that RTK Question has a combination of each UI-agnostic and React-hooks-based code, and customers may need to use createApi on each server and consumer. I additionally identified that we pre-bundle RTK’s artifacts, and I do not know the right way to additional cut up these out simply to fulfill these Subsequent/RSC-imposed constraints.

Seb Markbage, the React crew’s long-time lead architect (and now engaged on Subsequent at Vercel), replied: “The consumer of your api can nonetheless use the identical api, so long as you publish an optimized model of the internal implementation that excludes the pointless code branches.”

That led to a strongly worded debate. Lenz identified that this was asking all the ecosystem to make doubtlessly breaking packaging adjustments, particularly since many packages like Apollo nonetheless do not have an "exports" declaration. That appeared to be a shock to Seb, who prompt an extremely hacky workaround of import * as React to idiot their static analyzer.

I bought extraordinarily annoyed studying this change, and replied: “I’ve spent months attempting to improve our packaging, and now you are telling me I’ve to do extra work simply to maintain our code from breaking in RSC environments. That is very demoralizing.”.

The dialog in that individual thread died down, but in addition coincided with a complete bunch of debate threads about RSCs on Twitter and Reddit.

A couple of weeks later, Lenz revealed an in depth submit titled My tackle the present React & Server Elements controversy. In it, he famous that we predict RSCs are a really helpful expertise, however:

• It is a lot tougher for us to assist our customers, and so they’re submitting quite a lot of new assist points
• There’s much more about React (and Subsequent) we now have to grasp
• It is now a lot tougher to take care of and publish a library that works with React
• It appears like there’s been very poor communication from the React crew concerning the standing of RSCs and the use hook, and little dialogue on how it will influence the ecosystem

He additionally listed quite a lot of doable APIs that may assist libraries higher take care of RSC environments and knowledge fetching.

Lenz’s submit acquired plenty of robust optimistic suggestions from of us agreeing with the record of points and strategies, and expressing sympathy.

We by no means actually bought any precise response from the React crew round any of these API strategies, or the right way to correctly publish packages that cooperate with RSCs. I did get some outreach from some of us within the React org who’re attempting to work on including official docs round RSCs, and was capable of cross on quite a lot of group suggestions round RSCs, advertising, and utilization considerations.

The Subsequent docs did get a web page overlaying the right way to wrap third-party context suppliers with "use consumer", though that appears like a little bit of a band-aid.

After spending a lot time on these adjustments, I used to be getting pretty annoyed on the sheer variety of issues I used to be having to maintain observe of.

In late April, I tweeted:

Issues I’ve to remember when publishing a library in 2023:

• Construct artifact codecs (ESM, CJS, UMD)
  • Matrixed with: dev/prod/NODE_ENV builds
• Bundled or particular person .js per supply
• exports setup
• Webpack 4 limits
• TS moduleResolution choices
• Person environments
• Habits variations between bundlers
• Node ESM/CJS modes
• TS typedef output (bundled? particular person? .d.ts, or .d.mts?)
• Edge runtimes?
• And now React’s new "use consumer" and RSC constraints
• All of this for upstream deps too

That is getting totally ridiculous 🙁

I do not know the way anybody is meant to have the ability to sustain with all these doable configuration adjustments, edge instances, runtime environments, and conflicting constraints.

And there aren’t any precise complete guides on how to do that stuff. Everybody’s cargo-culting from others.

I am attempting to do proper by our customers and publish packages that work in as many environments as fairly doable, however that is extremely irritating to take care of.

It is a miracle something about this ecosystem works in any respect.

This struck a nerve. The tweet went fairly viral (for me), with dozens of retweets, quotes, and replies. It additionally bought linked in some newsletters as properly.

Equally, after I wrote my remark about “feeling demoralized” in response to RSC adjustments, I adopted that up with one other tweet linking the dialogue and venting:

As a library maintainer within the React ecosystem, I am getting fairly annoyed by the churn round React Server Elements.

Actually beginning to query whether or not the touted advantages are definitely worth the ache being inflicted on lib maintainers

That tweet additionally unfold fairly extensively and bought quite a lot of reactions as properly.

A couple of days later, I had a follow-up thought:

I discover it ironic that each of my latest tweets a/b frustrations coping with JS ecosystem churn as a lib maintainer (bundle setup, RSCs) have gone viral.

I usually attempt to keep optimistic and never gripe a lot publicly.

I assume people really feel or sympathize with these frustrations

As one reply famous: “I believe individuals know the way affected person and optimistic you might be in direction of what you are doing and the ecosystem. If one thing makes you annoyed that signifies that one thing BIG is not working proper or it is incorrect”.

Closing Ideas 🔗︎

I normally like to put in writing weblog posts like this after some launch is finished, or some dialogue has come to an finish, when it appears like that story is full and there is some concrete takeaways and solutions to put in writing about. It would be nice if I might say “hey, RTK 2.0 is revealed, and it features a set of packaging updates that I do know are appropriate and work all over the place”.

Sadly, that is not the case right here.

This has been an especially busy summer time for me throughout work, Redux, and private time, and I’ve realized I am coping with some burnout. However, additionally, I began this submit in the beginning of June getting back from React Summit, and have simply now gotten again to it a pair months later to get it wrapped up.

The place Do Issues Stand As we speak? 🔗︎

This is the state of issues as of in the present day:

Classes and Takeaways 🔗︎

I’ve stated many instances that I nonetheless really feel like I barely know something concerning the subjects of module packaging and publishing sorts. In observe, I would guess that I do really know greater than most JS devs, as a result of I’ve needed to spend a lot time attempting to work on this (each over time, and this 12 months particularly). However, given the insane complexity, myriad of conflicting necessities, and quickly altering record of issues to maintain observe of, I believe it is fairly comprehensible that I nonetheless really feel like a clueless imposter about all of those subjects.

So, what have I really discovered?

• I’ve a set of bundle configurations that appear to work in most bundlers and construct instruments proper now, and appear to have legitimate ESM/CJS packaging
• Publishing TS typedefs correctly provides one other degree of complexity on high of that
• Pre-bundling your JS construct artifacts sidesteps potential points round having to have ".js" in file imports, and appears to work okay with tree-shaking…
• However then can result in different issues when one thing like this “cut up out your consumer code for RSC eventualities” factor comes up
• It is virtually inconceivable to maintain up with all of the completely different instruments and constraints which are on the market, and so they preserve altering
• Equally, there simply aren’t sufficient official, complete sources on publishing packages. I discovered some first rate guides ( equivalent to The Trendy Information to Packaging Your JS Library , The React Library Information (draft) , and Publishing Trendy JS ), nevertheless it nonetheless appears like a lot of this information is fragmented, scattered, and conflicting. There’s particularly a necessity for steering on what bundle codecs+artifacts you ought to be transport to cowl completely different sorts of utilization eventualities.
• The ecosystem desperately wants higher tooling to assist automate this course of.
  • tsup appears to be fairly good, and I’ve seen another instruments that declare to assist with transport twin ESM/CJS packages that I have never tried, nevertheless it simply appears like a lot of this could possibly be automated.
  • Equally, it appears like there is a want for some sort of “test-your-lib-against-a-range-of-build-tools As A Service” instrument. I arrange a bunch of pattern Redux initiatives with quite a lot of construct instruments, and I noticed react-aria did one thing related, however this feels just like the sort of setup that could possibly be commoditized in some way and assist lib authors confirm that their packages work accurately throughout the ecosystem.
• React Server Elements are a helpful idea and gear, nevertheless it positive appears like this can be a main disruption that is going to interrupt quite a lot of the ecosystem. I perceive the React crew’s feedback that “nothing about consumer React adjustments, that is all additive”… however on the identical time it is a huge enhance in psychological overhead and use case complexity that each customers and lib authors need to take care of.
  • There’s been successfully no outreach from the React crew to the library ecosystem about the right way to take care of packages and RSCs. To be honest, we did have a name with Dan+Andrew, Dan reviewed Lenz’s RFC, and there is been some dialogue in points. However there was no announcement or warning about Subsequent throwing errors on consumer imports, no desk of “what’s in every canary model?” out there, and no sort of “steering to lib authors for RSC compat” submit or doc revealed.
• The general ecosystem CJS/ESM transition has been an extended and ongoing nightmare, and it exhibits no indicators of ending any time quickly. Only in the near past I noticed a pair of dueling posts that argued “CJS is hurting JavaScript”, and “CommonJS shouldn’t be going away”. Clearly this can be a drawback we will be caught with for years.

Future Steps 🔗︎

We nonetheless want to complete up Redux Toolkit 2.0, however that is going to take some time. Simply attempting to verify all of the packages are totally updated with the packaging adjustments, cross-reference one another’s variations, and that every one the TS sorts work, is time-consuming. We in all probability have extra code-related adjustments to make too, however we do not have a stable record of what else should be on this set of majors.

Moreover, all of us maintainers are coping with attempting to steadiness jobs, motivation, time, and actual life, and meaning Redux work is a decrease precedence for us proper now.

It has been an exhausting 12 months thus far coping with all these points. We nonetheless intend to get out the set of main releases (finally – completely no ETA guarantees right here), and I am hopeful that these new variations can be a profit to the ecosystem and considerably enhance compatibility.

I positive hope that every one this effort and time may have been value it.

Additional Info 🔗︎

• ES Modules Historical past and Specs
• ESM/CJS Debates
• Package deal Publishing Sources
• Instruments