Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Difficulty Running Osmosis #519

Closed
nkrsic opened this issue Oct 10, 2021 · 14 comments
Closed

Difficulty Running Osmosis #519

nkrsic opened this issue Oct 10, 2021 · 14 comments

Comments

@nkrsic
Copy link

nkrsic commented Oct 10, 2021

I am trying to install the daemon and run a node, using branch v4.x

I am able to install and init the moniker, but I am getting this error when I run osmosisd start

7:30PM INF running initialization for module module=epochs panic: unknown field "current_epoch_ended" in types.EpochInfo

When I look at the actual EpochInfo type in x/epochs/types/genesis.pb.go on line 31 I see that it does not have the field mentioned.

@zaiemv
Copy link

zaiemv commented Oct 11, 2021

I'm seeing the exact same issue when trying to launch the daemon.

@nkrsic
Copy link
Author

nkrsic commented Oct 11, 2021

In the Discord here:

https://discord.com/channels/798583171548840026/877743338760069141/891380625687322715

I was able to find advice here suggesting that the when not using state sync, the previous version 3.1 should be synced until the upgrade block (1314500), then switch to v4. Another comment confirms that v3.1 must be run first:

Indeed!! 😃 Ya, you have to switch versions a a certain block. Otherwise use Cosmovisor(Cosmovisor would automatically do this for you

@nkrsic
Copy link
Author

nkrsic commented Oct 11, 2021

In general this error relates to a common aspect of block sync vs state sync. For anyone who is new to Cosmos / Osmosis but generally comfortable with Linux admin, see these related article from Cosmos / Tendermint:

https://hub.cosmos.network/main/gaia-tutorials/join-mainnet.html

https://docs.tendermint.com/master/tendermint-core/state-sync.html

@ValarDragon
Copy link
Member

ValarDragon commented Oct 12, 2021

Yeah, you can't use the v4 binary against genesis, and you can't use the genesis binaries against latest state :/

Unfortunately this is the confusion with syncing from genesis, hopefully we get more tooling to fix things / help protect against syncing with the wrong version.

You can use the chainlayer snapshots to relatively quickly get a snapshot of a recent block height, and run the mainnet binary off of that to get synced to the network: https://quicksync.io/networks/osmosis.html

I think we'll have to brainstorm in the SDK for a better way to give a user error when using too new of a binary against an older version. Its a bit easier tbh to detect using too old of a binary, which has someone working on an improved error message there: cosmos/cosmos-sdk#10318

@sunnya97
Copy link
Collaborator

Maybe we need to ship a binary with cosmovisor and all the version binaries preinstalled?

@zaiemv
Copy link

zaiemv commented Oct 14, 2021

@ValarDragon - using the quicksync link to download the snapshot worked great and i was able to get osmosisd up and running without issue. Trying to get cosmovisor working after getting osmosisd working still ended up failing and erroring out. I think i'll move forward without cosmovisor now.

@sunnya97 - If it's for the sake of making it easier for newbies to be able to set this stuff up pretty easily, updating the docker image might be a great way to go. I tried using the compose file that was in the repo initially and it ended up failing because the image is no longer available on docker hub.

@ValarDragon
Copy link
Member

ValarDragon commented Oct 15, 2021

Maybe we need to ship a binary with cosmovisor and all the version binaries preinstalled?

I've been thinking of something more along jacob's idea, of one stop setup scripts. I'm thinking the end state here is something like rust-up, where it has you pick from a couple compiler options, with a default.
Default:

  • Pruned full node
  • Use chainlayer to download it
    Option axes:
  • pruned node, store all old events or archive node
  • Chainlayer, statesync, or fast sync
  • If fast sync: start from genesis, or block N

@sunnya97 - If it's for the sake of making it easier for newbies to be able to set this stuff up pretty easily, updating the docker image might be a great way to go. I tried using the compose file that was in the repo initially and it ended up failing because the image is no longer available on docker hub.

Thats a great idea. We have not touched the docker image, would love for some help in doing that

@tvassilarso13
Copy link

I am still having trouble with this issue. Is there an easy for fix for someone who isn't has familiar with this stuff?

@dualsystems
Copy link
Contributor

dualsystems commented Nov 24, 2021

@ValarDragon I get that error because genesis started with version 3 and upgraded to v4 ?

1:21AM INF Added peer module=p2p peer={"Data":{},"Logger":{}}
1:21AM INF Discovered new snapshot format=1 hash="$�\x0fĉx��.k�������N��o��$�T\x1f\x17��X�" height=1098000 module=statesync
1:21AM INF Discovered new snapshot format=1 hash="�L\x0f��\x00���7o���2��\x0f&&�X�\"��fI��d�" height=1096500 module=statesync
1:21AM INF new bucket is full, expiring new book=/data/.osmosisd/config/addrbook.json module=p2p
1:21AM INF new bucket is full, expiring new book=/data/.osmosisd/config/addrbook.json module=p2p
1:21AM INF Applied snapshot chunk to ABCI app chunk=33 format=1 height=2112000 module=statesync total=37
1:21AM INF Applied snapshot chunk to ABCI app chunk=34 format=1 height=2112000 module=statesync total=37
1:21AM INF Applied snapshot chunk to ABCI app chunk=35 format=1 height=2112000 module=statesync total=37
1:21AM INF Applied snapshot chunk to ABCI app chunk=36 format=1 height=2112000 module=statesync total=37
1:21AM INF Verified ABCI app appHash="3���¹�z��*�����ZO��\x04�m*����\\�d�" height=2112000 module=statesync
1:21AM INF Snapshot restored format=1 hash="k}\u007f��u\n�p/h�18\x0e��o[bw����\v.i����" height=2112000 module=statesync
1:21AM INF scheduler: run module=blockchain
1:21AM INF processor: run module=blockchain
panic: failed to process committed block (2112001:959110462FDE439320E8427039ECFF830C60A6F03C785520671515F8D8C30AE9): wrong Block.Header.Version. Expected {11 0}, got {11 1} [recovered]
        panic: failed to process committed block (2112001:959110462FDE439320E8427039ECFF830C60A6F03C785520671515F8D8C30AE9): wrong Block.Header.Version. Expected {11 0}, got {11 1}
last events:
0: scBlockReceived{2112001#959110462FDE439320E8427039ECFF830C60A6F03C785520671515F8D8C30AE9 from 727da24c1934ab28aff3156efac798ff20885d7e}
1: scBlockReceived{2112002#87440F228F7E0ACBDB336FBAEC5245C9B6A6DF8AB8F55F1A5BC929D219AA0D76 from 95621bb4aa63e05a23899bd596a6400a9cae04e3}
2: bcResetState{{{{11 0} 0.34.14} osmosis-1 1 2112000 2C27B002FB1D58980BFBA7BA859EA9C0F65220E7A25F0BCE512910B45EB489E7:6:20EAA4C39AA7 2021-11-23 19:09:53.688890796 +0000 UTC ValidatorSet{
  Proposer: Validator{E23AFCF0035FB01ACD02FE96F680066974D7072B PubKeyEd25519{D601D0FA5338D6BCD586AC6FC0AFE096D20C748880D8ACEB2AEED126B639AFB4} VP:73372 A:-29973792}

@faddat
Copy link
Member

faddat commented Nov 28, 2021

@dualsystems I think it's a question of ConsensusVersion somewhere in the app and I'm not real sure that it is possible/good/okay to simply change it in the code you're using to state sync from. Love where your mind is at, though.

Statesync is really the way.

I now do relaying from a state synced Gaia.

I also changed the name of this issue to difficulty running osmosis, and I'd like to make this an omnibus issue.

Regarding state distribution, I view quicksync as fairly flawed, ipfs would be generally faster and better. That said, I need to put together a distribution architecture that has:

  • Archive nodes x20
  • Normal Nodes x20
  • Pruned nodes x20

then does the sorta cron thing found here:

https://github.com/c29r3/cosmos-snapshots

and once that cron thing is done, it would post IPFS CIDs to a simple web page or github repository, say, once every 12 hours per chain. Then, other validators can follow the cluster. Trusted validators could even provide snapshots to the cluster.

@faddat faddat changed the title omsosisd daemon fails to start - Difficulty Running Osmosis Nov 28, 2021
@faddat faddat mentioned this issue Nov 29, 2021
@daniel-farina daniel-farina added this to the 2021 - December Milestone milestone Dec 2, 2021
@faddat
Copy link
Member

faddat commented Dec 8, 2021

#642

@dualsystems
Copy link
Contributor

dualsystems commented Dec 10, 2021

@faddat I can upload small snapshots from https://bootstrap.dual.systems/ to ipfs.
But I think for download and deploy 1Gb snapshot more simply via http imho. We can make that snapshot for request or very frequently. At current time sync node via statesync without list of trusted snapshot peers is not simple procedure.

@faddat
Copy link
Member

faddat commented Dec 10, 2021

Actually on Osmo right now, it isn't possible to state sync.

Best option is looking like #647 which is lookin 🔥

@dualsystems
Copy link
Contributor

SDK is fixed: tendermint/tendermint#7463

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Archived in project
Development

No branches or pull requests

7 participants