Test works in isolation but fails alongside other RTL-based tests

[johncornish]

• @testing-library/react version:
  "@testing-library/jest-dom": "^5.17.0",
  "@testing-library/react": "^14.0.0",
  "@testing-library/user-event": "^14.0.0",
• Testing Framework and version:
  Jest, maybe 29.5.12? Unknown version within Create React App (package-lock says "@types/jest": "^29.5.12",)
• DOM Environment:
  jsdom from jest-environment-jsdom 27.5.1

Relevant code or config:

// This test works on its own but not alongside the other tests shown below
  it('should navigate to customers page', async () => {
    renderWithProviders(<UserApp/>, {path: '/login'});

    await act(() => user.click(screen.getByDisplayValue("Log In")));
    await user.click(await screen.findByText("Customers"));

    expect(await screen.findByText('[email protected]')).toBeInTheDocument();
  });


// It works if I skip the test before it (**not** the failing test itself):

  xit('should not navigate to create-transaction page', async () => {
    renderWithProviders(<UserApp/>, {path: '/login'});

    await act(() => user.click(screen.getByDisplayValue("Log In")));

    expect(screen.queryByText("Create Transaction")).toBeNull();
  });

  it('should navigate to customers page', async () => {
    renderWithProviders(<UserApp/>, {path: '/login'});

    await act(() => user.click(screen.getByDisplayValue("Log In")));
    await user.click(await screen.findByText("Customers"));

    expect(await screen.findByText('[email protected]')).toBeInTheDocument();
  });

// I can get the test after it to fail if I move the test block up to be after `it('should not navigate to create-transaction page', ...)`


  it('should not navigate to create-transaction page', async () => {
    renderWithProviders(<UserApp/>, {path: '/login'});

    await act(() => user.click(screen.getByDisplayValue("Log In")));

    expect(screen.queryByText("Create Transaction")).toBeNull();
  });

  it('should navigate to agents page', async () => { // <== the 5th test tends to fail
    renderWithProviders(<UserApp/>, {path: '/login'});

    await act(()=> user.click(screen.getByDisplayValue("Log In")));
    await user.click(await screen.findByText("Agents"));

    expect(await screen.findByText('TestAgent OnAdminPage')).toBeInTheDocument();
  });

// The full test suite, with mock API using `nock` (I moved from `msw` because I thought it was the problem, but I'm experiencing the exact same problems so I'm pretty sure the problem is with React Testing Library):
import {renderWithProviders} from "./render-utils";
import {act, cleanup, screen} from "@testing-library/react";
import {UserApp} from "../UserApp";
import userEvent from "@testing-library/user-event";
import nock from "nock";

const user = userEvent.setup();

describe('navigation and default pages for Tryula Admin', () => {
  beforeAll(() => {
    const scope = nock('http://localhost:3000')
      .defaultReplyHeaders({
        'access-control-allow-origin': '*',
        'access-control-allow-credentials': 'true'
      });
    scope
      .persist()
      .post('/login/')
      .reply(200, {
          user: {
            user_type: 'TryulaAdmin',
          }
        }
      );
    scope
      .persist()
      .get('/privacy_check/invalid/')
      .reply(200, {privacy_policy_consent_required: true});
    scope
      .persist()
      .get('/any_admin_stats/')
      .reply(200, {});
    scope
      .persist()
      .get('/service_areas/')
      .reply(200, [{name: 'Fakerton'}]);
    scope
      .persist()
      .get('/languages/')
      .reply(200, [{name: 'Engl-ish'}]);
    scope
      .persist()
      .get('/agents/')
      .reply(200, [
        {first_name: 'TestAgent', last_name: 'OnAdminPage', service_areas: []}
      ]);
    scope
      .persist()
      .get('/agents/1/')
      .reply(200, {
        bio: 'I am the real fake agent',
        first_name: 'TestFirstName',
        last_name: 'TestLastName',
        languages: [],
        service_areas: []
      });
    scope
      .persist()
      .get('/transactions/')
      .reply(200, [{
        agent_id: 1,
        agent_name: 'Test Agent',
        customer_name: 'Test Customer'
      }]);
    scope
      .persist()
      .get('/customers/')
      .reply(200, [{
        email: '[email protected]'
      }]);
  });

  afterEach(cleanup);

  it('should go to /dashboard and see stats after login from agent search', async () => {
    renderWithProviders(<UserApp/>, {path: '/login?return_url=/'});

    await act(()=> user.click(screen.getByDisplayValue("Log In")));

    expect(await screen.findByTestId('num-gt45d-transactions')).toBeInTheDocument();
  });

  it('should go to /dashboard and see stats after login from agent profile', async () => {
    renderWithProviders(<UserApp/>, {path: '/login?return_url=/agent/1'});

    await act(()=> user.click(screen.getByDisplayValue("Log In")));

    expect(await screen.findByTestId('num-gt45d-transactions')).toBeInTheDocument();
  });

  it('should navigate to stats after seeing transactions', async () => {
    renderWithProviders(<UserApp/>, {path: '/login'});

    await act(()=> user.click(screen.getByDisplayValue("Log In")));
    await user.click(await screen.findByText("Transactions"));
    expect(await screen.findByText("Test Agent")).toBeInTheDocument();
    await user.click(await screen.findByText("Dashboard"));

    expect(await screen.findByTestId('num-active-transactions')).toBeInTheDocument();
    expect(await screen.findByTestId('num-pending-transactions')).toBeInTheDocument();
    expect(await screen.findByTestId('num-closed-transactions')).toBeInTheDocument();
    expect(await screen.findByTestId('num-gt45d-transactions')).toBeInTheDocument();
  });

  it('should not navigate to create-transaction page', async () => {
    renderWithProviders(<UserApp/>, {path: '/login'});

    await act(() => user.click(screen.getByDisplayValue("Log In")));

    expect(screen.queryByText("Create Transaction")).toBeNull();
  });

  it('should navigate to agents page', async () => {
    renderWithProviders(<UserApp/>, {path: '/login'});

    await act(()=> user.click(screen.getByDisplayValue("Log In")));
    await user.click(await screen.findByText("Agents"));

    expect(await screen.findByText('TestAgent OnAdminPage')).toBeInTheDocument();
  });

  it('should navigate to customers page', async () => {
    renderWithProviders(<UserApp/>, {path: '/login'});

    await act(() => user.click(screen.getByDisplayValue("Log In")));
    await user.click(await screen.findByText("Customers"));

    expect(await screen.findByText('[email protected]')).toBeInTheDocument();
  });

  it('should navigate to create-agent page', async () => {
    renderWithProviders(<UserApp/>, {path: '/login'});

    await act(()=> user.click(screen.getByDisplayValue("Log In")));
    await user.click(await screen.findByText("Create Agent"));

    expect(await screen.findByText('First Name')).toBeInTheDocument();
    expect(await screen.findByText('Last Name')).toBeInTheDocument();
    expect(await screen.findByText('Email')).toBeInTheDocument();
    expect(await screen.findByText('Phone')).toBeInTheDocument();
    expect(await screen.findByText('Powerform URL')).toBeInTheDocument();
    expect(await screen.findByText('Referral Agreement')).toBeInTheDocument();
    expect(await screen.findByText('Powerform ID')).toBeInTheDocument();
  });

  it('should not navigate to tryula-admins page', async () => {
    renderWithProviders(<UserApp/>, {path: '/login'});

    await act(() => user.click(screen.getByDisplayValue("Log In")));

    expect(screen.queryByText("Tryula Admins")).toBeNull();
  });

  it('should not navigate to super-admins page', async () => {
    renderWithProviders(<UserApp/>, {path: '/login'});

    await act(() => user.click(screen.getByDisplayValue("Log In")));

    expect(screen.queryByText("Super Admins")).toBeNull();
  });

  it('should not navigate to Site Settings page', async () => {
    renderWithProviders(<UserApp/>, {path: '/login'});

    await act(() => user.click(screen.getByDisplayValue("Log In")));

    expect(screen.queryByText("Site Settings")).toBeNull();
  });
});

What you did:

I have 10 unit tests that simulate a login with some fake API data (all data is contained in beforeAll), and then assert the default page for that type of user and what else they can navigate to. They each work individually, and they ought to, because I don't think I'm trying to do anything crazy--I'm not even trying to change the endpoints for each test, which I was doing before and which proved to be extremely finicky. I've isolated a stable subset of the fake API for each set of tests because I banged my head against the wall for a day with nonsensical failures when attempting to define different responses to the same endpoints within each it(...) block (for example, in one run there's one transaction, and in the next that endpoint returns [] and the app should display No transactions found.) I have some inkling to watch out for Jest test parallelization, so I've tried --runInBand to no avail. I have been berated by warnings about how I have to wrap state changes in act(), so I've tried doing that; at first it seemed like it helped, but then it just went back to failing. I figured somewhere I should be using waitFor, but RTL documentation says that using the find... queries has that built in automatically. Interestingly, I can get the "problem test" to succeed by skipping the test before it, and I can change which test fails by moving them around, as if just the 5th or 6th test is doomed to fail.

What happened:

The 5th test usually fails if run alongside the other tests. When it's run by itself, it works fine.

Reproduction:

Problem description:

It's seemingly so arbitrary. I've meticulously scoured the documentation and many, many Stack Overflow posts and I just can't reason about why it's behaving so erratically.

Suggested solution:

Document how to do this type of testing because the current learning curve is immensely steep.

[Aerophite]

I also have this issue after updating

[email protected]
[email protected]
@testing-library/[email protected]
@testing-library/[email protected]
@testing-library/[email protected]
@testing-library/[email protected]

---

The only solution I've found is to re-render after all user-event interactions...which is painful and I can't believe would be intended.

Anyone else find a better solution?

[Aerophite]

You can also disable fake timers and that seems to fix the issue for me...but I also don't like this solution. As long as userEvent is setup with { advanceTimers: jest.advanceTimersByTime }, it shouldn't be a problem.

[Aerophite]

Found two more things that work

• if you don't await the userEvent interaction it works..but this can't be intended since all userEvent interactions are promises and should be awaited...
• If you wrap the userEvent interaction in act...which shouldn't be necessary as all userEvent interactions should already be wrapped in act.

[Aerophite]

You can also get around this issue by rendering with { legacyRoot: true } but this is clearly not something that should have to be done either. Plus, doing this option will throw a warning in the console.

[hawkeyes21]

Hey @Aerophite, @johncornish any updates on this? Did you downgrade to a different version?

[MatanBobi]

@johncornish @Aerophite Can you please share a cloneable reproduction? Either a github repo or https://testing-library.com/new-rtl will help us to try and investigate this.