Add experimental support for history.pushState and history.replaceState #58335

timneutkens · 2023-11-11T19:19:40Z

What?

This PR introduces support for manually calling history.pushState and history.replaceState.

It's currently under an experimental flag:

/**
 * @type {import('next').NextConfig}
 */
const nextConfig = {
  experimental: {
    windowHistorySupport: true,
  },
}

module.exports = nextConfig

Going forward I'll refer to history.pushState as replaceState is interchangable.

When the flag is enabled you're able to call the web platform history.pushState in the usual way:

const data = {
	foo: 'bar'
}
const url = '/my-new-url?search=tim'
window.history.pushState(data, '', url)

Let's start by explaining what would happen without the flag:

When a new history entry is pushed outside of the Next.js router any back navigation to that history entry will cause a browser reload as it can no longer be used by Next.js as the required metadata for the router is missing. In practice this makes it so that pushState/replaceState is not feasible to be used. Any pathname / searchParams added can't be observed by usePathname / useSearchParams either.

With the flag enabled the pushState/replaceState calls are instrumented and is synced into the Next.js router. This way the Next.js router's internal metadata is preserved, making back navigations apply still, and pathname / searchParams is synced as well, making sure that you can observe it using usePathname and useSearchParams.

How?

Added a new experimental flag windowHistorySupport
Instruments history.pushState and history.replaceState
- Triggers the same action as popstate (ACTION_RESTORE) to sync the provided url (if provided) into the Next.js router
- Copies the Next.js values kept in history.state so that they are not lost
- Calls the original pushState/replaceState

~~Something to figure out is how we handle additional pushes/replaces in Next.js as that should override the history state that was previously set.~~
Went with this after discussing with @sebmarkbage:

When you open a page it preserves the custom history state
- This is to solve this case: when you manually window.history.pushState / window.history.replaceState and then do an mpa navigation (i.e. <a> or window.location.href) and the navigate backwards the custom history state is preserved
When you navigate back and forward (popstate) it preserves the custom history state
When you navigate client-side (i.e. router.push() / <Link>) the custom history state is not preserved

…pport

timneutkens · 2023-11-11T19:19:51Z

Current dependencies on/for this PR:

canary
- PR Add experimental support for history.pushState and history.replaceState #58335 👈

This stack of pull requests is managed by Graphite.

ijjk · 2023-11-11T19:24:43Z

Stats from current PR

Default Build (Increase detected ⚠️)

General Overall increase ⚠️

	vercel/next.js canary	vercel/next.js 11-11-Add_support_for_history.pushState_and_history.replaceState	Change
buildDuration	10.6s	10.6s	N/A
buildDurationCached	6s	6.1s	N/A
nodeModulesSize	199 MB	199 MB	⚠️ +28.8 kB
nextStartRea..uration (ms)	418ms	419ms	N/A

Client Bundles (main, webpack) Overall increase ⚠️

	vercel/next.js canary	vercel/next.js 11-11-Add_support_for_history.pushState_and_history.replaceState	Change
199-HASH.js gzip	29.1 kB	29.2 kB	⚠️ +138 B
3f784ff6-HASH.js gzip	53.3 kB	53.3 kB	N/A
494.HASH.js gzip	180 B	181 B	N/A
framework-HASH.js gzip	45.2 kB	45.2 kB	✓
main-app-HASH.js gzip	241 B	239 B	N/A
main-HASH.js gzip	31.7 kB	31.8 kB	N/A
webpack-HASH.js gzip	1.7 kB	1.7 kB	✓
Overall change	76 kB	76.1 kB	⚠️ +138 B

Legacy Client Bundles (polyfills)

	vercel/next.js canary	vercel/next.js 11-11-Add_support_for_history.pushState_and_history.replaceState	Change
polyfills-HASH.js gzip	31 kB	31 kB	✓
Overall change	31 kB	31 kB	✓

Client Pages

	vercel/next.js canary	vercel/next.js 11-11-Add_support_for_history.pushState_and_history.replaceState	Change
_app-HASH.js gzip	194 B	195 B	N/A
_error-HASH.js gzip	182 B	181 B	N/A
amp-HASH.js gzip	504 B	506 B	N/A
css-HASH.js gzip	322 B	323 B	N/A
dynamic-HASH.js gzip	2.5 kB	2.5 kB	✓
edge-ssr-HASH.js gzip	253 B	255 B	N/A
head-HASH.js gzip	348 B	347 B	N/A
hooks-HASH.js gzip	369 B	368 B	N/A
image-HASH.js gzip	4.3 kB	4.3 kB	N/A
index-HASH.js gzip	256 B	256 B	✓
link-HASH.js gzip	2.65 kB	2.65 kB	N/A
routerDirect..HASH.js gzip	311 B	311 B	✓
script-HASH.js gzip	384 B	383 B	N/A
withRouter-HASH.js gzip	307 B	308 B	N/A
1afbb74e6ecf..834.css gzip	106 B	106 B	✓
Overall change	3.17 kB	3.17 kB	✓

Client Build Manifests

	vercel/next.js canary	vercel/next.js 11-11-Add_support_for_history.pushState_and_history.replaceState	Change
_buildManifest.js gzip	486 B	484 B	N/A
Overall change	0 B	0 B	✓

Rendered Page Sizes

	vercel/next.js canary	vercel/next.js 11-11-Add_support_for_history.pushState_and_history.replaceState	Change
index.html gzip	528 B	526 B	N/A
link.html gzip	540 B	541 B	N/A
withRouter.html gzip	524 B	521 B	N/A
Overall change	0 B	0 B	✓

Edge SSR bundle Size Overall increase ⚠️

	vercel/next.js canary	vercel/next.js 11-11-Add_support_for_history.pushState_and_history.replaceState	Change
edge-ssr.js gzip	92.5 kB	92.5 kB	N/A
page.js gzip	145 kB	145 kB	⚠️ +114 B
Overall change	145 kB	145 kB	⚠️ +114 B

Middleware size

	vercel/next.js canary	vercel/next.js 11-11-Add_support_for_history.pushState_and_history.replaceState	Change
middleware-b..fest.js gzip	623 B	627 B	N/A
middleware-r..fest.js gzip	150 B	151 B	N/A
middleware.js gzip	24.8 kB	24.8 kB	N/A
edge-runtime..pack.js gzip	1.92 kB	1.92 kB	✓
Overall change	1.92 kB	1.92 kB	✓

Next Runtimes

	vercel/next.js canary	vercel/next.js 11-11-Add_support_for_history.pushState_and_history.replaceState	Change
app-page-exp...dev.js gzip	167 kB	167 kB	✓
app-page-exp..prod.js gzip	93.2 kB	93.2 kB	✓
app-page-tur..prod.js gzip	93.9 kB	93.9 kB	✓
app-page-tur..prod.js gzip	88.5 kB	88.5 kB	✓
app-page.run...dev.js gzip	137 kB	137 kB	✓
app-page.run..prod.js gzip	87.9 kB	87.9 kB	✓
app-route-ex...dev.js gzip	23.8 kB	23.8 kB	✓
app-route-ex..prod.js gzip	16.4 kB	16.4 kB	✓
app-route-tu..prod.js gzip	16.4 kB	16.4 kB	✓
app-route-tu..prod.js gzip	16 kB	16 kB	✓
app-route.ru...dev.js gzip	23.2 kB	23.2 kB	✓
app-route.ru..prod.js gzip	16 kB	16 kB	✓
pages-api-tu..prod.js gzip	9.37 kB	9.37 kB	✓
pages-api.ru...dev.js gzip	9.64 kB	9.64 kB	✓
pages-api.ru..prod.js gzip	9.37 kB	9.37 kB	✓
pages-turbo...prod.js gzip	21.8 kB	21.8 kB	✓
pages.runtim...dev.js gzip	22.5 kB	22.5 kB	✓
pages.runtim..prod.js gzip	21.8 kB	21.8 kB	✓
server.runti..prod.js gzip	48.8 kB	48.8 kB	✓
Overall change	922 kB	922 kB	✓

Diff details

Diff for page.js

Diff too large to display

Diff for edge-ssr.js

Diff too large to display

Diff for 199-HASH.js

Diff too large to display

Commit: 358a380

packages/next/src/client/components/app-router.tsx

packages/next/src/build/webpack/plugins/define-env-plugin.ts

packages/next/src/client/components/app-router.tsx

…lback

ijjk · 2023-11-12T08:21:29Z

Tests Passed

…-Add_support_for_history.pushState_and_history.replaceState

…d_history.replaceState

franky47 · 2023-11-14T04:26:23Z

packages/next/src/client/components/app-router.tsx

+        startTransition(() => {
+          dispatch({
+            type: ACTION_RESTORE,
+            url: new URL(url ?? window.location.href),


issue: This should be based on window.location.href, to allow relative/partial URLs.

One can set search params on the current page (and maintain router state) via history.replaceState(history.state, '', '?foo=bar'), but this line fails as it's not a valid URL.

suggestion: new URL(url ?? '', window.location.href)

I created issue #56636 some time ago. I spotted this experimental feature in the 14.0.3 release notes and thought it could fix it. However, when I try it, I just get TypeError: Failed to construct 'URL': Invalid URL errors. It looks to be related to this review comment.

@cduff see #58438

Thanks @franky47. I just tested against 14.0.4-canary.1 which includes #58438 and it looks like it will now resolve #56636.

dvoytenko · 2023-11-16T17:30:10Z

packages/next/src/client/components/app-router.tsx

+  const currentState = window.history.state
+  const __NA = currentState?.__NA
+  if (__NA) {
+    data.__NA = __NA


@timneutkens I believe data can be very plausible be null or undefined here based on the replaceHistory API.

Follow-up to #58335. Fixes #48110 (reply in thread) As reported in the discussion passing a origin-relative string didn't work as `new URL` will throw an error. This ensures the `origin` parameter is provided to `new URL`. Added tests for the string behavior.  --------- Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com> Co-authored-by: Zack Tanner <[email protected]>

dennation · 2023-11-21T05:41:12Z

@timneutkens thanks for this feature, it's much needed! But did you forget about useParams hook?

doiali · 2023-11-21T14:10:47Z

https://medium.com/@moh.mir36/shallow-routing-with-next-js-v13-app-directory-2d765928c340

I've been using a solution similar to the one mentioned in above article. Unfortunately it no longer works after this pr.

What I did was implementing a useURL hook that returned stateful pathname and search. The result of this hook was always up to date for any kind of navigation (<Link>, router.push, window.history.replace, ...).
In previous nextJS versions, we didn't need such a workaround, because useRouter had everything we needed.

justrealmilk · 2023-11-23T23:51:20Z

Would it be possible to alter this addition so that it amends whatever the set function for pushState is rather than fully replacing it.

As it stands, these changes break anything else that hooks into the history API such as Plausible analytics. Presumably, other analytics function in similar ways to unobtrusively track page change.

timneutkens · 2023-11-24T13:06:18Z

@justrealmilk I'm guessing the gap between the effect call and when we read window.history.pushState / replaceState in the module is when those override? That can be changed yeah.

It would be helpful to know the exact case because then a test can be added.

timneutkens · 2023-11-24T13:21:18Z

Opened a fix here: #58861, this is based on the assumption mentioned above, would be good to get confirmation of the exact case.

manuelseisl · 2023-11-26T10:43:49Z

Now, there is an issue with Google Tag Manager "History change" event trigger:
#58924

alexcarpenter · 2023-11-27T16:17:03Z

Now, there is an issue with Google Tag Manager "History change" event trigger: #58924

Similarly looks to cause an issue with Plausible 4lejandrito/next-plausible#107

joacub · 2023-11-30T00:20:54Z

@timneutkens thanks for this feature, it's much needed! But did you forget about useParams hook?

seems like that, usePArams is not using this

timneutkens · 2023-12-02T09:04:02Z

seems like that, usePArams is not using this

I didn't forget about it, it can't be supported. Client-side you don't know what routes could match. If you want to change parameters you have to use router.push / router.replace so that the request happens that correctly resolved the url.

joacub · 2023-12-02T15:13:20Z

seems like that, usePArams is not using this

I didn't forget about it, it can't be supported. Client-side you don't know what routes could match. If you want to change parameters you have to use router.push / router.replace so that the request happens that correctly resolved the url.

So, I have to use first push and then replace ?

timneutkens · 2023-12-02T19:04:29Z

No I meant you can use either, but in that case you just don't shallow route.

joacub · 2023-12-02T19:07:34Z

No I meant you can use either, but in that case you just don't shallow route.

The problem @timneutkens is that in the case of the server request nextjs is losing the state of all tree, is not yet possible to keep the state alive in between path changes ?

For example /es/pricing to /en/pricing it should keep the state and just hydrate the changes ….,

if you just remove the keying you are using it will just hydrate, the keying you are using lose all things shout the needing

AmirL · 2023-12-02T19:08:20Z

May I ask when it's available in stable NextJs 13.x?

## What? Fixes #58335 (comment). Waiting on a reply for the exact case but my assumption is that the history is overwritten in the user module instead of before the other JS loads.

sunwrobert · 2023-12-07T21:24:42Z

Why wouldnt this be added on the actual router hook? Seems a bit weird that we have to directly use window, no?

timneutkens added 5 commits November 11, 2023 16:51

Add support for history.pushState and history.replaceState

174861b

Add experimental flag for these changes: experimental.windowHistorySu…

db4c408

…pport

Add comment explaining the override

e5a8bd2

Add comment explaining check for canonicalUrl

2486105

Add marker for custom data push

b78d35c

ijjk added area: tests created-by: Next.js team PRs by the Next.js team. type: next labels Nov 11, 2023

timneutkens commented Nov 11, 2023

View reviewed changes

packages/next/src/client/components/app-router.tsx Show resolved Hide resolved

feedthejim reviewed Nov 11, 2023

View reviewed changes

packages/next/src/build/webpack/plugins/define-env-plugin.ts Show resolved Hide resolved

acdlite reviewed Nov 11, 2023

View reviewed changes

packages/next/src/client/components/app-router.tsx Outdated Show resolved Hide resolved

Move onPopState and applyUrlFromHistoryPushReplace into useEffect cal…

af92394

…lback

timneutkens added 11 commits November 12, 2023 15:36

Add updateHistory to pushRef

c5284f2

Merge branch 'canary' of https://github.com/vercel/next.js into 11-11…

8d85bb2

…-Add_support_for_history.pushState_and_history.replaceState

Fix type

1498a8c

Skip updating history when updateHistory is false

2b2ee73

Set mutable.updateHistory in all reducers

d957ef9

Add updateHistory: false

69befda

Add preserveCustomHistoryState

e83926a

Fix tests

c95b795

Handle preserveCustomHistoryState correctly

c37e0fa

Remove __PRIVATE_NEXTJS_INTERNALS_CUSTOM_DATA as it's not used

14f8317

Merge branch 'canary' into 11-11-Add_support_for_history.pushState_an…

358a380

…d_history.replaceState

timneutkens marked this pull request as ready for review November 13, 2023 12:18

timneutkens requested review from ijjk, shuding and huozhi as code owners November 13, 2023 12:18

kodiakhq bot merged commit 797fecb into canary Nov 13, 2023
60 checks passed

kodiakhq bot deleted the 11-11-Add_support_for_history.pushState_and_history.replaceState branch November 13, 2023 13:32

This was referenced Nov 13, 2023

fix: Don't reset shallow URL updates on prefetch #58297

Merged

wrong url after refresh page 47ng/nuqs#392

Closed

Resets to initial state when there is Link with prefetch 47ng/nuqs#388

Closed

franky47 reviewed Nov 14, 2023

View reviewed changes

timneutkens mentioned this pull request Nov 14, 2023

Support passing a relative string to pushState/replaceState #58438

Merged

dvoytenko reviewed Nov 16, 2023

View reviewed changes

TheSGJ mentioned this pull request Nov 18, 2023

NextTopLoader stopped working on Nextjs 14.0.3 TheSGJ/nextjs-toploader#56

Open

justrealmilk mentioned this pull request Nov 22, 2023

Changes as of Next.js 14.0.3 break the Plausible script 4lejandrito/next-plausible#107

Closed

timneutkens mentioned this pull request Nov 24, 2023

Ensure original history is read in effect #58861

Merged

eatyourgreens mentioned this pull request Dec 17, 2023

Subject URLs should update for projects that use subject selection zooniverse/front-end-monorepo#2537

Open

github-actions bot added the locked label Dec 22, 2023

github-actions bot locked as resolved and limited conversation to collaborators Dec 22, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add experimental support for history.pushState and history.replaceState #58335

Add experimental support for history.pushState and history.replaceState #58335

timneutkens commented Nov 11, 2023 •

edited

Loading

timneutkens commented Nov 11, 2023

ijjk commented Nov 11, 2023 •

edited

Loading

ijjk commented Nov 12, 2023 •

edited

Loading

franky47 Nov 14, 2023

cduff Nov 17, 2023

franky47 Nov 17, 2023

cduff Nov 17, 2023

dvoytenko Nov 16, 2023

dennation commented Nov 21, 2023

doiali commented Nov 21, 2023

justrealmilk commented Nov 23, 2023

timneutkens commented Nov 24, 2023 •

edited

Loading

timneutkens commented Nov 24, 2023

manuelseisl commented Nov 26, 2023

alexcarpenter commented Nov 27, 2023

joacub commented Nov 30, 2023

timneutkens commented Dec 2, 2023

joacub commented Dec 2, 2023 •

edited

Loading

timneutkens commented Dec 2, 2023

joacub commented Dec 2, 2023 •

edited

Loading

AmirL commented Dec 2, 2023

sunwrobert commented Dec 7, 2023

Add experimental support for history.pushState and history.replaceState #58335

Add experimental support for history.pushState and history.replaceState #58335

Conversation

timneutkens commented Nov 11, 2023 • edited Loading

What?

How?

timneutkens commented Nov 11, 2023

ijjk commented Nov 11, 2023 • edited Loading

Stats from current PR

ijjk commented Nov 12, 2023 • edited Loading

Tests Passed

franky47 Nov 14, 2023

Choose a reason for hiding this comment

cduff Nov 17, 2023

Choose a reason for hiding this comment

franky47 Nov 17, 2023

Choose a reason for hiding this comment

cduff Nov 17, 2023

Choose a reason for hiding this comment

dvoytenko Nov 16, 2023

Choose a reason for hiding this comment

dennation commented Nov 21, 2023

doiali commented Nov 21, 2023

justrealmilk commented Nov 23, 2023

timneutkens commented Nov 24, 2023 • edited Loading

timneutkens commented Nov 24, 2023

manuelseisl commented Nov 26, 2023

alexcarpenter commented Nov 27, 2023

joacub commented Nov 30, 2023

timneutkens commented Dec 2, 2023

joacub commented Dec 2, 2023 • edited Loading

timneutkens commented Dec 2, 2023

joacub commented Dec 2, 2023 • edited Loading

AmirL commented Dec 2, 2023

sunwrobert commented Dec 7, 2023

timneutkens commented Nov 11, 2023 •

edited

Loading

ijjk commented Nov 11, 2023 •

edited

Loading

ijjk commented Nov 12, 2023 •

edited

Loading

timneutkens commented Nov 24, 2023 •

edited

Loading

joacub commented Dec 2, 2023 •

edited

Loading

joacub commented Dec 2, 2023 •

edited

Loading