DataTable: stateful functionality is a buggy mess

[jeffreytgilbert]

Describe the bug

https://primereact.org/datatable/#stateful

Hello fellow developers,

I have followed the documentation. I've read through the code. I've been through support tickets. Stateful saving and restoring of filters, sorts, and pages in the DataTable is a buggy poorly documented mess.

Is this a feature request or a bug report? I think PrimeReact thinks this feature works. So this is a bug report. A feature request would be "please give us a setting that allows saving state in multiple ways through a flag, or at the least, a plugin that works.

I've tried getting this working with URLs but it repaints and pukes so hard i have not been able to get it to work. I replaced the code with localStorage per some examples and cobbled together something that works most of the time, until it doesn't and then it is stuck in the "doesn't" state until the user clears their local storage. I get weird, terribly opaque errors that are nearly impossible to triage or reason about.

My only relief from this is 2 options: 1) abandon prime react and rewrite all of my tables on the sites in a better library which makes me mad or 2) abandon stateful saving between page changes and make my users mad.

Who that uses this "feature" thinks it works well? I'd love to know your use case and implementation details. I'm fascinated by the idea of a smoothly running stateful react searchable data table with any level of persistence. I see another ticket from last year around this time which hasn't been addressed also regarding stateful support in this component so I'm guessing this isn't something that's high on anyone's priority list.

Reproducer

I created a standalone example of the data table which causes the behavior. It just takes a few toggles on the dropdown select boxes and it starts freezing up and geeks out. it wont interact and sometimes crashes the page.

https://github.com/jeffreytgilbert/prime-react-datatable-wrapper-example/tree/main

System Information

primereact from 10.5.3 to 10.9.2

    "primereact": "^10.9.2",
    "react": "18.3.1",
    "react-dom": "18.3.1",
    "@tanstack/react-query": "^5.66.9",
    "@tanstack/react-query-devtools": "^5.66.9",
    "@tanstack/react-router": "^1.111.11",
    "@tanstack/router-devtools": "^1.111.11",
    "@tanstack/router-vite-plugin": "^1.111.12",
    "typescript": "^5.7.3",
    "vite": "^6.2.0"

Steps to reproduce the behavior

import { useLocalStorage };

const SFP_VERSION: string | number = 2;

const wrapFilterMeta = (filterData: DataTableStateEvent) => {
  return {
    lastTouched: format(new Date(), "full"),
    sfp: filterData,
  };
};

  const dt = useRef(null);
  const navigate = useNavigate();
  const queryParams = useSearch({ strict: false });
  const match = useMatch({ strict: false });
  const filteredFilters = useMemo(() => {
    return columns
      .filter((column) => column.isFiltered)
      .map((column) => {
        return column.filterConfig;
      });
  }, [columns]);

  const defaultFilters = useMemo(
    () => Object.assign({}, ...filteredFilters),
    [filteredFilters]
  );
  const [filters, setFilters] = useState<DataTableFilterMeta>(defaultFilters);

  const [updatedFilterStateLocalStorage, setUpdatedFilterStateLocalStorage] =
    useLocalStorage<{
      lastTouched: string;
      sfp: DataTableStateEvent | null | undefined;
    } | null>(`sfp:${SFP_VERSION}:${match.routeId}`, null);

  useEffect(() => {
    columns
      .filter((column) => column.customFilter)
      .map((column) => {
        if (
          column.customFilter &&
          column.customFilter.filterName &&
          column.customFilter.filterMethod
        ) {
          FilterService.register(
            column.customFilter.filterName,
            column.customFilter.filterMethod
          );
        }
      });
  }, [columns]);

  const handleFilterChange = useCallback(
    (event: DataTableStateEvent) => {
      const wrappedMeta = wrapFilterMeta(event);
      setUpdatedFilterStateLocalStorage(() => {
        const state = {
          ...wrappedMeta
        };
        return state;
      });
      setFilters(event.filters);
      navigate({
        replace: true,
        search: { ...wrappedMeta },
      } as NavigateOptions);
    },
    [setUpdatedFilterStateLocalStorage]
  );

  const clearFilter = useCallback(() => {
    setFilters({});
    setUpdatedFilterStateLocalStorage({
      lastTouched: format(new Date(), "full"),
      sfp: undefined,
    });
    navigate({
      replace: true,
      search: {},
    } as NavigateOptions);
  }, []);
  }, [setUpdatedFilterStateLocalStorage]);

  const [hasParams] = useState(Object.keys(queryParams).length !== 0);
  useEffect(() => {
    if (hasParams) {
      setFilters(queryParams?.sfp?.filters as DataTableFilterMeta);
      setUpdatedFilterStateLocalStorage((prev) => {
        const data = queryParams?.sfp;
        return wrapFilterMeta({ ...prev, ...data });
      });
    }
    else if (updatedFilterStateLocalStorage?.sfp && !hasParams) {
      setFilters(updatedFilterStateLocalStorage.sfp.filters);
    }
    else {
      setFilters(defaultFilters);
    }
  }, [queryParams, hasParams]);

 return (
    <DataTable
      onFilter={handleFilterChange}
      onSort={handleFilterChange}
      dataKey="id"
      value={rows}
      filters={filters}
      filterDisplay="row"
      ref={dt}
      ...some settings and callbacks...
    >
      ...some columns...
  </DataTable>
);

Expected behavior

Stop re-rendering so much when applying filters, sorts, and pagination.

Have a better way to integrate persistence stores through flags, not custom code.

If you're not going to have a flag, have better documentation on how this should work with examples that go beyond the LEAST useful option of localstorage.

Instead use the MOST useful option of history state so it saves where you're at to the URL and when you go back and forth between pages, it remembers where you are.

Also use history replace so it doesn't ask you to click back as many times as you clicked sort, filter, page buttons and typed in the filter every letter into filter fields.

please!

[jeffreytgilbert]

Here is the full component file. The bits above are just cherry picked code related to filters.

import { Button, Flex, Select, Space } from "@mantine/core";
import { Column } from "primereact/column";
import {
  DataTable,
  DataTableFilterMeta,
  DataTableSelectionMultipleChangeEvent,
  DataTableSortMeta,
  DataTableStateEvent,
  DataTableValue,
  DataTableValueArray,
} from "primereact/datatable";
import { useCallback, useEffect, useMemo, useRef, useState } from "react";
import { IconFilterOff, IconPhoto } from "@tabler/icons-react";
import {
  useLocalStorage,
  useThrottle,
  useWindowSize,
} from "@uidotdev/usehooks";
import { VirtualScrollerProps } from "primereact/virtualscroller";
import ExportToCSVButton from "./ExportToCSVButton";
import { ColumnProps } from "./ColumnOptions";
import { dataTablePaginator } from "./DataTablePaginator";
import { NavigateOptions, useMatch, useNavigate, useSearch } from "@tanstack/react-router";
import { FilterService } from "primereact/api";
import { format } from "@formkit/tempo";
import { useResponsive } from "../../hooks/useResponsive";
import { useSearchThumbnailSize } from '../../context/SearchThumbnailSize/context';

type FilteredSortedSearchResultsProps = {
  rows: object[];
  columns: ColumnProps[];
  tableStyle?: object;
  onSelectionChange?: (
    event: DataTableSelectionMultipleChangeEvent<DataTableValueArray>
  ) => void;
  virtualScrollerOptions?: VirtualScrollerProps;
  bulkActionButtons?: React.ReactNode;
  selection?: DataTableValue[];
  selectionMode?: "multiple" | "checkbox" | null;
  sortMode?: "single" | "multiple" | undefined;
  yOffset?: number;
  xOffset?: number;
  id?: string | number;
};

export type DataTableFilterState = {
  columnOrder: string[];
  filters: DataTableFilterMeta;
  first: number;
  multiSortMeta: DataTableSortMeta[] | null | undefined;
  rows: number;
  selection: DataTableValue[];
};

type RenderHeaderProps = {
  bulkActionButtons: React.ReactNode;
  clearFilter: () => void;
  xOffset: number;
};

const SFP_VERSION: string | number = 1;

const RenderHeader: React.FC<RenderHeaderProps> = ({
  bulkActionButtons,
  clearFilter,
}) => {
  return (
    <div
      style={{
        display: "grid",
        gridTemplateColumns: "max-content 1fr 150px",
        gap: "8px",
        alignItems: "center",
      }}
    >
      {bulkActionButtons ? bulkActionButtons : <Space />}
      <Space />
      <Button
        type="button"
        leftSection={<IconFilterOff size={16} />}
        size="sm"
        style={{ width: "100%" }}
        variant="light"
        onClick={clearFilter}
      >
        Clear Filters
      </Button>
    </div>
  );
};

const wrapFilterMeta = (filterData: DataTableStateEvent) => {
  return {
    lastTouched: format(new Date(), "full"),
    sfp: filterData,
  };
};

export const FilteredSortedSearchResults: React.FC<FilteredSortedSearchResultsProps> = ({
  rows,
  columns,
  tableStyle,
  onSelectionChange = () => {},
  virtualScrollerOptions,
  bulkActionButtons,
  selection = [],
  selectionMode = null,
  sortMode = "multiple",
  yOffset = 0,
  xOffset = 0,
}) => {
  const dt = useRef(null);
  const navigate = useNavigate();
  const queryParams = useSearch({ strict: false });
  const match = useMatch({ strict: false });
  const filteredFilters = useMemo(() => {
    return columns
      .filter((column) => column.isFiltered)
      .map((column) => {
        return column.filterConfig;
      });
  }, [columns]);

  const defaultFilters = useMemo(
    () => Object.assign({}, ...filteredFilters),
    [filteredFilters]
  );
  const [filters, setFilters] = useState<DataTableFilterMeta>(defaultFilters);

  const [updatedFilterStateLocalStorage, setUpdatedFilterStateLocalStorage] =
    useLocalStorage<{
      lastTouched: string;
      sfp: DataTableStateEvent | null | undefined;
    } | null>(`sfp:${SFP_VERSION}:${match.routeId}`, null);

  const [containsImageColumn, setContainsImageColumn] = useState(false);
  const {
    searchThumbnailSize,
    setSearchThumbnailSize,
    allThumbnailSizeOptions,
    setAllThumbnailSizeOptions
  } = useSearchThumbnailSize();

  useEffect(()=>{
    setAllThumbnailSizeOptions([
      {value: '25', label: 'x-small'},
      {value: '50', label: 'small'},
      {value: '75', label: 'medium'},
      {value: '100', label: 'large'},
      {value: '200', label: 'x-large'},
    ]);
  }, [setAllThumbnailSizeOptions]);
  useEffect(() => {
    columns
      .filter((column) => column.customFilter)
      .map((column) => {
        if (
          column.customFilter &&
          column.customFilter.filterName &&
          column.customFilter.filterMethod
        ) {
          FilterService.register(
            column.customFilter.filterName,
            column.customFilter.filterMethod
          );
        }
      });
    const imageColumn = columns.find((column) => {
      if(['event_logo', 'media'].includes(column.field)){
        return column;
      }
    });
    if(imageColumn){
      setContainsImageColumn(true);
    }
    else {
      setContainsImageColumn(false);
    }
  }, [columns]);

  const handleFilterChange = useCallback(
    (event: DataTableStateEvent) => {
      const wrappedMeta = wrapFilterMeta(event);
      setUpdatedFilterStateLocalStorage(() => {
        const state = {
          ...wrappedMeta
        };
        return state;
      });
      setFilters(event.filters);
      navigate({
        replace: true,
        search: { ...wrappedMeta },
      } as NavigateOptions);
    },
    [setUpdatedFilterStateLocalStorage]
  );

  const clearFilter = useCallback(() => {
    setFilters({});
    setUpdatedFilterStateLocalStorage({
      lastTouched: format(new Date(), "full"),
      sfp: undefined,
    });
    navigate({
      replace: true,
      search: {},
    } as NavigateOptions);
  }, [setUpdatedFilterStateLocalStorage]);

  const [hasParams] = useState(Object.keys(queryParams).length !== 0);
  useEffect(() => {
    if (hasParams) {
      setFilters(queryParams?.sfp?.filters as DataTableFilterMeta);
      setUpdatedFilterStateLocalStorage((prev) => {
        const data = queryParams?.sfp;
        return wrapFilterMeta({ ...prev, ...data });
      });
    }
    else if (updatedFilterStateLocalStorage?.sfp && !hasParams) {
      setFilters(updatedFilterStateLocalStorage.sfp.filters);
    }
    else {
      setFilters(defaultFilters);
    }
  }, [queryParams, hasParams]);

  const { height } = useWindowSize();
  const debounceHeight = useThrottle(height, 16);
  const [scrollerOptions, setScrollerOptions] = useState<VirtualScrollerProps | undefined>(virtualScrollerOptions);
  const [overflowHeight, setOverflowHeight] = useState<string | undefined>(undefined);

  useEffect(() => {
    if (debounceHeight && debounceHeight > 800) {
      setScrollerOptions(virtualScrollerOptions);
      setOverflowHeight("calc(" + 100 + "dvh - " + yOffset + "px)");
    }
    else {
      setScrollerOptions(undefined);
      setOverflowHeight(undefined);
    }
  }, [
    debounceHeight,
    virtualScrollerOptions,
    yOffset,
    setScrollerOptions,
    setOverflowHeight,
  ]);

  const rowsPerPageOptions = useMemo(() => [50, 100, 1000, 3000], []);
  const paginatorTemplate = dataTablePaginator({
    rowsPerPageOptions,
  });

  const { matchesMobile } = useResponsive();

  return (
    <DataTable
      onFilter={handleFilterChange}
      onSort={handleFilterChange}
      dataKey="id"
      value={rows}
      tableStyle={tableStyle}
      size="small"
      emptyMessage="No Results Found"
      showGridlines={false}
      reorderableColumns={true}
      reorderableRows={false}
      stripedRows={true}
      removableSort
      sortMode={sortMode}
      scrollable={!matchesMobile}
      scrollHeight={!matchesMobile ? overflowHeight : undefined}
      virtualScrollerOptions={!matchesMobile ? scrollerOptions: undefined}
      header={
        <RenderHeader
          bulkActionButtons={bulkActionButtons}
          clearFilter={clearFilter}
          xOffset={xOffset}
        />
      }
      filters={filters}
      multiSortMeta={updatedFilterStateLocalStorage?.sfp?.multiSortMeta}
      filterDisplay="row"
      onSelectionChange={onSelectionChange}
      selection={selection}
      selectionMode={selectionMode}
      rows={rowsPerPageOptions[0]}
      rowsPerPageOptions={rowsPerPageOptions}
      paginator={true}
      paginatorTemplate={paginatorTemplate}
      currentPageReportTemplate={"{first} to {last} of {totalRecords}"}
      paginatorLeft={
        containsImageColumn ?
        <Flex direction="row" gap="xs">
          <IconPhoto size={35}/>
          <Select
            defaultValue="50"
            value={searchThumbnailSize?.value}
            data={allThumbnailSizeOptions}
            onChange={(selectedValue) => {
              const selectedOption = allThumbnailSizeOptions?.find((option) => option.value === selectedValue);
              setSearchThumbnailSize(selectedOption);
            }}
            maw={100}
          />
        </Flex> :
        <Space w={135} />
      }
      paginatorRight={
        <div
          style={{
            display: "flex",
            alignItems: "center",
            justifyContent: "flex-end",
            gap: "8px",
            width: "135px",
          }}
        >
          <ExportToCSVButton tableRef={dt} />
        </div>
      }
      ref={dt}
    >
      {selectionMode === "multiple" && (
        <Column selectionMode="multiple" style={{ width: "3em" }} />
      )}
      {columns
        .filter((column) => !column.isHidden)
        .map((column) => {
          return column.column;
        })}
    </DataTable>
  );
};

[melloware]

@jeffreytgilbert It sounds like definitely something is not working but a simple stack blitz reproducer showing the bugs or issues is the only way devs can figure it out.

I started one for you: https://stackblitz.com/edit/c6ixxqgn?file=src%2FApp.jsx

I tested the above myself and I can watch in SessionStorage as i sort, page etc the JSON object is changing properly. So there is a good starting point for you to reproduce the bug?

[github-actions]

Please fork the Stackblitz project and create a case demonstrating your bug report. This issue will be closed if no activities in 20 days.

[jeffreytgilbert]

Thanks for replying. I'll dig into it further. The URL storage version of this code I do not have handy. I'll have to go back and see if i ever committed anything or try to rewrite it. The issue I'm seeing on the session storage one is some oddity where It wont allow clicks and it has null references when the data is all there. I validated it in the data. It's also very brittle because if you chance the filters, there's nothing that looks at it and says this version is new, dont use the old copy. I might see if i can change the manually versioned iteration to something that hashes the collection of columns.

I've made a wrapper for this component with a number of custom column types. I'll put it all in a public github project and share that as well.

[jeffreytgilbert]

I created a standalone example of the data table which causes the behavior. It just takes a few toggles on the dropdown select boxes and it starts freezing up and geeks out. it wont interact and sometimes crashes the page.

https://github.com/jeffreytgilbert/prime-react-datatable-wrapper-example/tree/main

[melloware]

Thanks I updated this to bug since you have provided a reproducer.

[melloware]

I just ran your example for a while and it seemed to be OK for me. When you say "dropdowns" do you mean the filters?

[jeffreytgilbert]

Yes, sorry for not being explicit. The filters, specifically the select box filters, appear to hang the system when toggling them. This is the issue in the code I've reverted on my production code so it no longer uses persistence through localstorage and it has resolved the hanging issue.

The provided example does not reproduce the issue with attempting to save the state to the URL with a history replace. Unfortunately I don't have the free time to revisit URL case at this time, but I still feel passionately that this should be the built in default behavior of the data table OR minimally it should be a built in option supported with just a parameter. State saved in the URL for search params is maybe the most common use case across the entirety of internet history for search pages. Please consider proposing this. Thanks for listening. I appreciate you.

[jeffreytgilbert]

PS, if I can get through my backlog of client deliverables for this week, I will try and reproduce the hanging in a JAM session and update the ticket with it. May not help, but maybe better than chasing ghosts.