# Migrate the Leaderboard Module After Production
In this section, you will:
- Add the leaderboard module via migration.
- Populate the module's genesis with a proper leaderboard.
In previous sections:
- You added, and added a migration for, a player info structure that tallies wins and losses per player. You called it v1.1.
- You added a leaderboard module, which exists only if you start a new blockchain from scratch. You called it v2.
Here you will reuse some learnings from the v1.1 migration, and adjust them for the special case of a new module.
# High level considerations
Here the decision is to start your v2's leaderboard as if all played past games had been counted for the leaderboard. You only need to go through all player information and add a leaderboard including the information. Migration is a good method to tackle the initial leaderboard.
For the avoidance of doubt, v1.1 and v2 refer to the overall versions of the application, and not to the consensus versions of individual modules, which may change or not.
# What you will do
To prepare for your v1.1-to-v2 migration, you will:
- Add helper functions to process large amounts of data from the latest chain state of type v1.1.
- Add a function to migrate your state from v1.1 to v2.
- Make sure you can handle large amounts of data.
- Put callbacks if necessary.
Why do you need to make sure you can handle large amounts of data? The full state at the point of migration may well have millions of players. You do not want your process to grind to a halt because of a lack of memory or I/O capacity.
# v1.1 to v2 leaderboard migration helper
In the migration, there are two time-consuming parts:
- Fetching the stored player info records in a paginated way, consuming mostly database resources.
- Sorting each intermediate leaderboard, consuming mostly computation resources.
It looks beneficial to use Go routines in this case too, and to use a player info channel to pass along arrays of player info records.
In practice, repeatedly building the intermediate leaderboard means adding k new
Winners to the sorted array, sorting it, clipping it to
Params.Length, and repeating. What constitutes a good k value should be dictated by testing and performance measurements. However, you can start with your best guess in a new file created for this purpose.
# Types and interfaces
Where do you put this new file? The leaderboard module's consensus version starts at
2. The application will go from no leaderboard to leaderboard at version 2. So it makes sense to create a new folder to encapsulate this knowledge:
Put your target k length in:
With a view to reusing the module's
Candidate types, you can add a convenience method to convert an array of
PlayerInfo to an array of
To extract the
PlayerInfo, you need access to a checkers keeper. More precisely, you only need access to its paginated
As usual, describe this dependency in
Then put the migration-specific elements in a dedicated folder:
Create the routine that fetches player info from the checkers storage:
- This passes along the channel a tuple
PlayerInfosChunkthat may contain an error. This is to obtain a result similar to when a function returns an optional error.
- It uses the paginated query so as to not overwhelm the memory if there are millions of infos.
- It closes the channel upon exit whether there is an error or not via the use of
This routine populates the player info channel. What about the routine that consumes it?
# Leaderboard computation
Create the routine function that builds the leaderboard in memory and returns it when complete:
- The winners are initialized at a
0size but with a capacity of
Params.Length + chunk, which is the expected maximum intermediate size it will reach. This initialization should ensure that the slice does not need to have its capacity increased mid-process.
- It also passes along a tuple with an optional error.
- It closes the channel it populates upon exit whether there is an error or not via the use of
This routine populates the leaderboard channel. Where is it consumed?
# Routines orchestration
Now you can declare the main function that creates the channels and routines, and collects the leaderboard when done:
- This returns the leaderboard instead of saving it in the keeper. That is because, when introducing a module, you have to initialize it with a genesis, and this computed leaderboard will be part of the module's genesis.
- It delegates the closing of channels to the routines.
- It starts the second routine first to reduce the likelihood of channel clogging.
# Advertising the function to use
The migration proper needs to introduce the new module and then populate the genesis with the result of the
MapPlayerInfosReduceToLeaderboard function. You can encapsulate and advertise this knowledge in two functions:
To further limit the dependency of the leaderboard module on the checkers module, you could consider:
- Having the new expected interface be based on
- And keeping the transformation between one and the other in a file whose name is suffixed with
# v1.1 to v2 migration proper
You now have in place the functions that will handle the store migration. Next you have to set up the chain of command for these functions to be called by the node at the right point in time.
# Consensus version and name
upgrade module keeps in its store the different module versions (opens new window) that are currently running. To signal an upgrade, your module needs to return a different value when queried by the
upgrade module. As it stands, your leaderboard consensus version is
2 and that will be its first value when added to the application. To make this explicit, and consistent with the pattern used in the checkers module, you can keep this information in a constant like you did for the checkers module:
It can be used by
The consensus version number has no connection to v1.1 or v2. The consensus version number is for the module, whereas v1.1 or v2 is for the whole application.
You also have to pick a name for the upgrade you have prepared. This name will identify your specific upgrade when it is mentioned in a
Plan (i.e. an upgrade governance proposal). Use a name that is relevant at the application level:
In this you save:
You have to inform your app about:
- The module being introduced.
- The genesis to use for it.
Prepare these in turn.
# Callback in leaderboard module
Although the module does not upgrade per se, you need to make sure that it does not fail when presented with an upgrade. Call
You could run a proper migration here, if you added a checkers keeper (or rather a
types.PlayerInfoKeeper) in your module.
The downside is that this keeper would be used only at the time of migration and therefore is a massive overkill, if not a security risk.
# Callback in
In a previous section you already prepared
app.go to handle a migration, namely
v1tov1_1. Here you add to this
setupUpgradeHandlers function in two places:
First, after the
v1tov1_1upgrade method, introduce the way to get the new leaderboard's module genesis:
- The version map is populated with the current consensus version of the leaderboard. This stops the leaderboard from being upgraded further.
app.CheckersKeeperis used. This is the only time the leaderboard module has access to the checkers module.
- The genesis needs to be marshalled before being passed to
Second, inform it that as part of the
v1_1tov2upgrade a new store key is introduced:
This is where the genesis will be saved.
With this, the app is configured to handle the module upgrade.
# Unit tests
After all these changes it is worthwhile adding tests, at least on the helpers.
# New mock types
You introduced a new expected keeper. If you want to unit test your migration helpers properly, you have to mock this new expected interface:
Add to the relevant
Now run it:
# Create the mock instance
In a new file, add a function to prepare your new mock:
In your test, prepare the mock:
# Configure the mock
To use your mock, this time it takes a bit of effort:
- To test that it was called correctly, use
.EXPECT()with defined values.
- To have it return player infos in chunks, use
.Return()and make sure the pagination's
NextKeyis populated when necessary.
- To confirm that one call happens before the other, use
Imagine that the keeper has:
Also imagine you want to have it return the info paginated in chunks of size
2. This means two calls:
The first call:
The second call:
Now specify the desired order:
# Rest of the test
From there, the test is as usual:
# Run it
You can confirm that the test passes by running:
Given the configuration difficulty of the mock, only this test will do.
It is not possible to add integration tests on the migration proper, because when the app is created it is already at v2.
# Interact via the CLI
You can execute a live upgrade from the command line. The following upgrade process takes inspiration from this one (opens new window) based on Gaia. You will:
- Check out the checkers v1.1 code.
- Build the v1.1 checkers executable.
- Initialize a local blockchain and network.
- Run v1.1 checkers.
- Add one or more incomplete games.
- Add one or more complete games with the help of a CosmJS integration test.
- Create a governance proposal to upgrade with the right plan name at an appropriate block height.
- Make the proposal pass.
- Wait for v1.1 checkers to halt on its own at the upgrade height.
- Check out the checkers v2 code.
- Build the v2 checkers executable.
- Run v2 checkers.
- Confirm that you now have a correct leaderboard.
Start your engines!
# Launch v1.1
After committing your changes, in a shell checkout v1.1 of checkers with the content of the CosmJS client work:
Build the v1.1 executable for your platform:
release/v1_1/checkersd executable ready, you can initialize the network.
Because this is an exercise, to avoid messing with your keyring you must always specify
Add two players:
Create a new genesis:
Give your players the same token amounts that were added by Ignite, as found in
To be able to run a quick test, you need to change the voting period of a proposal. This is found in the genesis:
This returns something like:
That is two days, which is too long to wait for CLI tests. Choose another value, perhaps 10 minutes (i.e.
"600s"). Update it in place in the genesis:
You can confirm that the value is in using the earlier command.
Make Alice the chain's validator too by creating a genesis transaction modeled on that done by Ignite, as found in
Now you can start the chain proper:
# Add games
From another shell, create a few un-played games with:
--broadcast-mode block flag means that you can fire up many such games by just copying the command without facing any sequence errors.
To get a few complete games, you are going to run the integration tests (opens new window) against it. These tests call a faucet if the accounts do not have enough. Because you do not have a faucet here, you need to credit your test accounts with standard
bank send transactions. You can use the same values as found in the
With the test accounts sufficiently credited, you can now run the integration tests. Run them three times in a row to create three complete games:
You can confirm that you have computed player info:
With enough games in the system, you can move to the software upgrade governance proposal.
# Governance proposal
For the software upgrade governance proposal, you want to make sure that it stops the chain not too far in the future but still after the voting period. With a voting period of 10 minutes, take 15 minutes. How many seconds does a block take?
This returns something like:
blocks_per_year computes down to 5 seconds per block. At this rate, 15 minutes mean 180 blocks.
What is the current block height? Check:
This returns something like:
That means you will use:
What is the minimum deposit for a proposal? Check:
This returns something like:
This is the minimum amount that Alice has to deposit when submitting the proposal. This will do:
Now submit your governance proposal upgrade:
This returns something like:
1 is the proposal ID you reuse. Have Alice and Bob vote yes on it:
Confirm that it has collected the votes:
It should print:
See how long you have to wait for the chain to reach the end of the voting period:
In the end this prints:
Wait for this period. Afterward, with the same command you should see:
Now wait for the chain to reach the desired block height, which should take five more minutes as per your parameters. When it has reached that height, the shell with the running
checkersd should show something like:
At this point, run in another shell:
You should always get the same value, no matter how many times you try. That is because the chain has stopped. For instance:
checkersd with CTRL-C. It has saved a new file:
With your node (and therefore your whole blockchain) down, you are ready to move to v2.
# Launch v2
With v1_1 stopped and its state saved, it is time to move to v2. Checkout v2 of checkers:
Back in the first shell, build the v2 executable:
It should start and display something like:
After it has started, you can confirm in another shell that you have the expected leaderboard with:
This should print something like:
Note how it took the time of the block when v1_1 stopped.
Congratulations, you have upgraded your blockchain almost as if in production!
You can stop Ignite CLI. If you used Docker that would be:
Your checkers blockchain is done! It has a leaderboard, which was introduced later in production thanks to migrations.
You no doubt have many ideas about how to improve it. In particular, you could implement the missing draw mechanism, which in effect has to be accepted by both players.
To summarize, this section has explored:
- How to add a leaderboard to an existing blockchain.
- How to upgrade a blockchain in production, by migrating from v1_1 of the blockchain to v2, and the new store structure that will be introduced by the upgrade.
- How to handle the data migrations and logic upgrades implicit during migration, such as with the use of helper functions.
- Worthwhile unit tests with regard to leaderboard handling.
- A complete procedure for how to conduct the update via the CLI.