Last week, Spadina genesis took place and was slow to start. The goal of Spadina was to run through genesis a final time, but the testnet suffered from low participation and a lack of finality at launch, along with a handful of operational failures. The confusion that ensued led the Eth2 team to schedule another dress rehearsal for Oct 12, called Zinken.
- Spadina: Eventually finalized, but ‘bumpy’ at onset.
- Slasher Issue: New Prysm release did not include configurations for the slasher feature. We suggested a fix to Prysm.
- Zinken: Spadina re-run. Deposit up until Oct 8, Genesis Oct 12.
Although the testnet eventually finalized (it took around 70 epochs and almost 8 hours), a number of operational missteps led to low initial participation, and a less than smooth overall launch experience for Spadina.
Prysm ran into some problems, which they eventually corrected and made a post mortem about. The bottom line was a lack of preparation for the testnet with the consequence being few Prysm nodes able to participate at launch, and general community confusion. Eventually, Prysm released and notified the community about alpha.27 which included updated configurations. This resolved the majority of the issues and eventually got participation up the point that the chain could finalize.
Our Spadina Experience
At genesis, we launched our nodes and created a few validators. During preparations, things seemed to be going well until we encountered the Prysm critical peering issue. We restarted our nodes, cleared the database in the nodes and updated configurations based on Prysm’s new guidelines. Along with the rest of the network, we were eventually up and running and were pleased to see that our Blox Infra nodes proved themselves in a multi-network implementation and that our validators activated on time.
Prysm Slasher Issue
In addition to the Prysm peering issue, we noticed an additional problem with the optional Prysm ‘slasher.’ Slasher was not working in Spadina and the alpha.27 update did not include a configuration to fix it. Due to this, slasher was recognizing correct attestations as slashable events.
In order to resolve this issue, we made a custom fix for the Prysm slasher and suggested this solution in the Prysm Discord channel. Once we identified and pushed the fix, we were happy to see that everything started to work perfectly.
Resolved KeyVault Issue
We also discovered a minor bug in KeyVault. We had passed on an incorrect genesis time for Spadina in KeyVault, in order for our built-in slashing protection to calculate epochs and slots appropriately. This did not effect attestations but required pushing a new version of Blox in order to resolve.
Zinken: Here We Go Again
In light of the bugs discovered during Spadina, we have another dress rehearsal coming up, Zinken. This time, deposits are due 4 days before genesis (rather than 2 as in Spadina) in order to give everyone a chance to adequately prepare. The Eth2 team also asks that the community treat Zinken as if it were mainnet launch, in order simulate the real thing as closely as possible.
Following the initial hiccups of Spadina, the chain eventually finalized and was notably stable, which is a good sign for the Beacon Chain. We are looking forward to giving genesis another go with Zinken, and are anxiously counting down until mainnet release!
Until then, Blox will keep staking on the Medalla testnet. If you want to get involved, you can download Blox Beta to start staking now. Once the mainnet goes live, we will facilitate an easy transition to real ETH staking on the mainnet for all our Beta users.