NFTX subgraph syncing issues
Incident Report for NFTX
Resolved
With a bleeding-edge patch applied to one of our Erigon nodes the NFTX Graph Indexer was able to resync the NFTX subgraph.

The site is now running as expected, and we will continue to closely monitor the service over the weekend.
Posted Sep 16, 2022 - 21:31 UTC
Identified
There is an issue with the way in which Erigon is dealing with `trace_block` since the merge. This is resulting in a number of subgraphs that use trace to fail at the same block.

We have tried to switch our indexer from Erigon to GETH to fix the issue, however GETH does not support trace so the subgraph still fails.

Some updates have been made to the subgraph to remove the reliance on trace and a new subgraph has been deployed to test if this can fix the issue, the sync for this is looking to be 8-12 hours.

It is now a race between Erigon pushing a fix for this we can patch our nodes and resume indexing, or once the new subgraph deployment is fully indexed and tested then we can update the endpoints.

In the meantime we have reverted requests for the holdings to fallback to Web3 calls. This means that the vaults will load initially with the subgraph data before updating to show only items that are currenlty held in the vaults.

NFTX will still not be able to show any recent activity or newly created vaults on the UI.
Posted Sep 16, 2022 - 11:21 UTC
Update
This appears to be an issue relating to both The Graph and Erigon.

We applied https://github.com/ledgerwatch/erigon/commit/15ad43e1000e29e48f11de26aa42ebde72050772 patch however there were not improvements.

There are some indexers which are able to index to the current block now so we are looking at whether those will begin to return the up-to-date data for the apps.
Posted Sep 16, 2022 - 09:32 UTC
Investigating
The NFTX subgraph (along with a number of other subgraphs) is failingin on block `15539509`.

This is unrelated to the protocol being paused and likely linked to the recent merge.

The Graph team is looking into the issue and we're awaiting further updates from them.

You will still be able to interact with the app however data related to the holdings, recent transactions, activity etc may be incorrect (i.e. it will be showing the state of the platform as of block `15539509`).
Posted Sep 15, 2022 - 21:20 UTC
This incident affected: NFTX App.