Installing the archive migration package
The archive node Berkeley migration package is sufficient for satisfying the migration from Devnet/Mainnet to Berkeley. However, it has some limitations. For example, the migration package does not migrate a non-canonical chain and it skips orphaned blocks that are not part of a canonical chain.
To mitigate these limitations, the archive node maintenance package is available for use by archive node operators who want to maintain a copy of their Devnet and Mainnet databases for historical reasons.
Install with Google Cloud SDK
The Google Cloud SDK installer does not always register a google-cloud-sdk
apt package. The best way to install gsutil is using the apt repostory:
curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo gpg --dearmor -o /usr/share/keyrings/cloud.google.gpg
echo "deb [signed-by=/usr/share/keyrings/cloud.google.gpg] https://packages.cloud.google.com/apt cloud-sdk main" | sudo tee -a /etc/apt/sources.list.d/google-cloud-sdk.list
sudo apt-get update && sudo apt-get install google-cloud-sdk
Download the o1labs Mainnet archive database
We strongly encourage you to perform the migration on your own data to preserve the benefits of decentralization. However, if you want to use the archive data that o1labs runs (for example, to bootstrap a new archive from SQL without waiting all day for the chain to download and replay), you can use the following steps:
Download the Devnet/Mainnet archive data using cURL or gsutil:
cURL:
For Devnet:
curl https://storage.googleapis.com/mina-archive-dumps/devnet-archive-dump-{date}_0000.sql.tar.gz
For Mainnet:
curl https://storage.googleapis.com/mina-archive-dumps/mainnet-archive-dump-{date}_0000.sql.tar.gz
To filter the dumps by date, replace
{date}
using the requiredyyyy-dd-mm
format. For example, for March 15, 2024, use2024-03-15
.⚠️ The majority of backups have the
0000
suffix. If a download with that name suffix is not available, try incrementing it. For example,0001
,0002
, and so on.gsutil:
gsutil cp gs://mina-archive-dumps/mainnet-archive-dump-2024-01-15* .
Extract the tar package.
tar -xvzf {network}-archive-dump-{date}_0000.sql.tar.gz {network}-archive-dump-{date}_0000.sql
Import the Devnet/Mainnet archive dump into the Berkeley database.
Run this command at the database server:
psql -U {user} -f {network}-archive-dump-{date}_0000.sql
The database in the dump archive_balances_migrated is created with the Devnet/Mainnet archive schema.
Note: This database does not have any Berkeley changes.
Ensure the location of Google Cloud bucket with the Devnet/Mainnet precomputed blocks
The recommended method is to perform migration on your own data to preserve the benefits of decentralization.
gsutil cp gs://mina_network_block_data/{network}-*.json .
⚠️ Precomputed blocks for the Mainnet network take ~800 GB of disk space. Plan for adequate time to download these blocks. The Berkeley migration app downloads them incrementally only when needed.
Validate the Devnet/Mainnet database
The correct Devnet/Mainnet database state is crucial for a successful migration.
Missing blocks is one the most frequent issues when dealing with the Devnet/Mainnet archive. Although this step is optional, it is strongly recommended that you verify the archive condition before you start the migration process.
To learn how to maintain archive data, see Devnet/Mainnet database maintenance.
Download the migration applications
Migration applications are distributed as part of the archive migration Docker and Debian packages.
Choose the packages that are appropriate for your environment.
Debian packages
To get the Debian packages:
CODENAME=bullseye
CHANNEL=stable
VERSION=3.0.1-e848ecb
echo "deb [trusted=yes] http://packages.o1test.net $CODENAME $CHANNEL" | tee /etc/apt/sources.list.d/mina.list
apt-get update
apt-get install --allow-downgrades -y "mina-archive-migration=$VERSION"
Docker image
To get the Docker image:
docker pull gcr.io/o1labs-192920/mina-archive-migration:3.0.1-e848ecb-{codename}
Where supported codenames are:
- bullseye
- focal
- buster
Devnet/Mainnet genesis ledger
The Mina Devnet/Mainnet genesis ledger is stored in GitHub in the mina
repository under the genesis_ledgers
subfolder. However, if you are already running a daemon that is connected to the Mina Mainnet or the Devnet network, you already have the genesis ledger locally.
Berkeley database schema files
You can get the Berkeley schema files from different locations:
GitHub repository from the
berkeley
branch.Note: The
berkeley
branch can contain new updates regarding schema files, so always get the latest schema files instead of using an already downloaded schema.Archive/Rosetta Docker from
berkeley
version
Example: Downloading schema sources from GitHub
wget https://raw.githubusercontent.com/MinaProtocol/mina/berkeley/src/app/archive/zkapp_tables.sql
wget https://raw.githubusercontent.com/MinaProtocol/mina/berkeley/src/app/archive/create_schema.sql
Next steps
Congratulations on completing the essential preparation and verification steps. You are now ready to perform the migration steps in Migrating Devnet/Mainnet Archive to Berkeley Archive.