# Add a Leaderboard Module
Make sure you have all you need before proceeding:
- You understand the concepts of Protobuf, modules, and migrations.
- Go is installed.
- You have the checkers blockchain codebase up to the Tally Player Info After Production. If not, follow the previous steps or check out the relevant version (opens new window).
In this section, you will:
- Add a new module.
- Add a leaderboard storage type.
- Add hooks for loose coupling of modules.
- Use the transient store.
In the previous section you added a player info structure that tallies wins and losses per player. On its own, this information could be collected outside of the blockchain via a dedicated server.
It was in fact done on-chain so as to make this new step more relevant. If you want an on-chain leaderboard that is provably correct, then you need its information to come from the chain too. As a result of this choice, you have the necessary information on-chain in the form of PlayerInfo. You now need to organize it into a leaderboard.
# High level considerations
Your blockchain is now at v1.1. In this section, you will introduce v2 of your blockchain with leaderboard support. A good leaderboard fulfills these conditions:
- Any player who has ever played should have a tally of games won, lost, and forfeited. You already have that.
- The leaderboard should list the players with the most wins up to a pre-determined number of players. For example, the leaderboard might only include the top 100 scores.
- To avoid squatting and increase engagement, when scores are equal in value the most recent score takes precedence over an older one: the player with the more recent score is listed higher on the leaderboard.
When you introduce the leaderboard in production, you also have to consider migration. This concern is covered in the next section.
The leaderboard is not strictly the concern of the game of checkers. It is a side concern. The concept of a leaderboard is also very generic, you could easily imagine it being used for other types of game. Therefore, it makes sense to introduce it as a separate module next to the checkers module.
The checkers and leaderboard modules will exchange information. More specifically, the leaderboard needs to know when a player's total wins change, as this may warrant entering the leaderboard. If you have the checkers module call the leaderboard module (just as it calls the bank when handling wagers), the checkers module needs to know the details of the leaderboard module. It is best to avoid such tight coupling.
Fortunately, you can reuse a hooks pattern already used in the Cosmos SDK. With this future addition, the leaderboard module adds a listener to the hook interface of the checkers module. With this the checkers module informs any listeners, whether there are none, one, or many.
The leaderboard module will work by listening to results from the checkers module. It will not have messages of it own.
Thinking about early performance optimization, you have to decide what operations the module does when it receives one candidate from the checkers module. The first idea is to:
- Read the leaderboard from storage, which includes all 100 members.
- Conditionally add the candidate to the leaderboard.
- If added, sort and clip the list.
- Put the leaderboard back in storage.
These are a lot of expensive operations for a single candidate.
Fortunately, there is a better way. The leaderboard needs to be computed and saved when the block is prepared, but it does not need to be up to date after each (checkers) transaction. You can imagine keeping the leaderboard (or something approximating it) in memory for the whole length of the block.
In the section about expiring games, you learned about EndBlock. There is also a BeginBlock callback. It is conceivable to prepare the leaderboard in BeginBlock and keep it in the context or a memory or transient storage. Then it would be recalled with each candidate, and finally (in EndBlock, and only there) it would be sorted and clipped before being saved in storage proper.
Better still, though, you do not need to prepare the leaderboard in BeginBlock. You can just keep candidates in the transient storage as they come. Then only in EndBlock is the leaderboard loaded, updated, and saved.
# What you will do
Several things need to be addressed to build your v2 blockchain:
- Add the leaderboard module.
- Define your new data types.
- Add helper functions to encapsulate clearly defined actions, like leaderboard sorting.
- Prepare keys to store candidates in a transient store.
- Adjust the existing code to make use of and update the new data types.
- Add the hooks pattern elements.
- Handle the leaderboard properly.
- Configure the app for it.
# New v2 module
As discussed, you will introduce a new leaderboard module. This is conveniently done with Ignite CLI.
Ignite also offers the possibilty to add new Params to the module. These are module-wide parameters:
- Whose original value is defined in the genesis
- That can be modified via governance proposal
It could be interesting to have the length of the leaderboard be defined like that.
With that, Ignite has created a new x/leaderboard (opens new window) folder next to x/checkers. It has also put a length field inside Params:
The genesis defines a starting value (opens new window) of 0 for this length. You ought to change it now to something adequate:
You also make it const as this is the case and becomes helpful later on.
# New v2 information
It is time to take a closer look at the new data structures being introduced with the new module.
If you feel unsure about creating new data structures with Ignite CLI, look at the previous sections of the exercise again.
To give the new v2 information a data structure, you need the following:
Add a structure for the leaderboard: you want a single stored leaderboard for the whole module. Let Ignite CLI help you implement a structure:
This creates a Protobuf file with
string winners. This is not very useful. So you declare by hand another Protobuf message inleaderboard.protofor use as a leaderboard rung:
Take note of the key features:
addressindicates the player. This will be the same address as the one that comes inPlayerInfo.index.wonCountdetermines the ranking on the leaderboard - the higher the count, the closer to the0index in the array. This should exactly match the value found in the corresponding player stats. This duplication of data is a lesser evil, because ifwonCountwas missing you would have to access the player stats to sort the leaderboard.addedAtis a timestamp that indicates when the player'swonCountwas last updated and determines the ranking when there is a tie inwonCount- the more recent, the closer to the0index in the array.
You make the
Leaderboardmessage usemessage Winneras an array. Add that each element in the map is not nullable. This will compile eachWinningPlayerto a Go object instead of a pointer:The v2 genesis was also updated with the leaderboard. Tell it that the leaderboard should always be there (even if empty):
At this point, you should run
ignite generate proto-goso that the corresponding Go objects are re-created.Remember to make sure the initial value stored for the leaderboard is not
nilbut instead is an empty list. Ingenesis.goadjust:This function returns a default genesis. This step is important if you start fresh. In your case, you do not begin with an "empty" genesis but with one resulting from the upcoming genesis migration in this exercise.
In particular, add a test on the initial genesis:
Fix the compilation error in the same file:
And add a test case that catches a duplicated winner address:
Also adjust other compilation errors:
On
genesis.go:And:
On
genesis_test.go:On
client/cli/query_leaderboard_test.go:Now that the leaderboard will always be in the store, you may as well change the
GetLeaderboardfunction so that it panics instead of returning an error when it cannot find it:This requires further easy fixing of new compilation errors. A more complex fix is the one that checks what happens when the leaderboard is removed:
The test case you added will fail unless you update the
Validate()method of the genesis to not allow duplicate player addresses. This is inspired bytypes/genesis.go, and is best kept in a separate and newtypes/leaderboard.go:After this, you can adjust the
types/genesis.gofile:
You can confirm that the existing unit tests pass.
# Transient object
You will use objects when storing candidates in a transient KVStore between BeginBlock and EndBlock. You want them to be small.
In leaderboard.proto, add:
Where bytes address is the player's undecoded address.
Remember that sdk.AccAddress's underlying type is byte[].
After another round of Go compilation, you can add a helper function to get a Candidate's address as a Bech32 string:
Where sdk.AccAddress(candidate.Address) is casting the byte[] into sdk.AccAddress.
Also add a function to convert it into a leaderboard rung at a given time:
With the structure set up, it is time to add the code using these new elements in normal (non-migration) operations.
# Leaderboard helpers
Continue working on your v2 before tackling the migration. In both the migration and regular operations, the leaderboard helpers have to:
- Add a number of new candidates to your array of winners.
- Sort the array according to the rules.
- Clip the array to the chosen length and save the result.
You can reuse your types/leaderboard.go to encapsulate all your leaderboard helpers:
Add functions to sort a slice of winners in place:
It tests in descending order, first for scores and then for the timestamps.
It is possible to write a one-liner inside this function, but at the expense of readability.
When it comes to adding or updating candidates to the array of winners, your goal is to make these operations as efficient as possible. To avoid having to find duplicate player addresses in an array, it is better to use a map. Add a function to convert an array of winners into a map:
The timestamp used when a winner is added to the leaderboard will be the block's time. In other words, it will be the same time for all candidates added in
EndBlock. Prepare a function to do that:Note how, when creating the map, it initializes with a capacity equal to the sum of both winners and candidates' lengths. This is an approximative way of increasing memory performance.
# Candidate Lifecycle
You have prepared helper functions that will update a list of winners with a list of candidates. The candidates will come from the transient store – transient in the sense that it will be discarded after EndBlock. That is good for this usage, as you do not want to carry candidates from one block to the next.
Your leaderboard module does not have access to a transient store by default, so you will have to prepare that first.
Additionally, you want to reduce the number of marshalling / unmarshalling taking place repeatedly. It would not make sense to unmarshall a whole array of candidates every time you want to add a single candidate to the array. Instead, it makes sense to keep each candidate as a single entry in the store, and separately keep the information on how many n are being stored. Later, you can retrieve them with [k] where 0 <= k < n.
You will:
- Prepare your leaderboard module with access to a transient store.
- Define keys of the candidates transient store.
- Add a function to prepare the candidates transient store in
BeginBlock. - Add a function to add a single candidate to the store.
- Add a function to retrieve all the candidates from the transient store.
# Prepare transient store
By default, Ignite CLI does not prepare your module to have access to a transient store like it prepares it to have access to the proper store. The preparation works the same way as a normal store.
Add a transient store key in your keeper:
Update the constructor accordingly:
This key will be identified by a new string in app.go's list of transient store keys. Add such a distinct key:
Adjust app.go so that it gives the keeper a valid key. Also take this opportunity to fix an Ignite bug on memKeys:
Do not forget to ensure that there is indeed a store key at the string(s) you asked:
# Prepare candidate store keys
Your keeper has access to a transient store. Define the keys by which elements will be accessed in it. Taking inspiration from checkers' stored games use of prefixes and their use in the GetAllStoredGame function, prepare prefix keys for the values in a new types/key_candidate.go file:
# Use the candidate store
Now you can add the functions that will use the transient store at each update and on EndBlock. Add a new keeper/candidate.go file with:
This function saves the candidate at its address. Already having []byte Address in the Candidate object proves useful. This also means that if there are two updates in one block for a single player only the second update is recorded. In the case of a game that has only increasing scores, this is okay.
Next, taking inspiration from StoredGame again, add a function to get all candidates with an iterator:
This gets all candidates. There may be many, but not so many that it grinds the application. After all, it only gets all that was put during the block itself.
# Leaderboard handling
You have created the leaderboard helper functions and the function to get all candidates. You can now update the leaderboard. This takes place in EndBlock.
First, in a separate file, add one function to the keeper:
This function gets the candidates from the transient store and the leaderboard from the regular store, adds the candidates, clips the array to the maximum length found in Params, and saves the updated leaderboard back in storage.
This means that the leaderboard will be unmarshalled and marshalled only once per block.
Next, make sure it is called from EndBlock. In module.go:
If Ignite did it right, app.go has already set up (opens new window) the leaderboard module to be called on EndBlock.
Your leaderboard will now be updated and saved on an on-going basis as part of your v2 blockchain. However, so far, you have put nothing in the transient store.
# Hook infrastructure for candidates
To populate candidate winners in your transient store, you are going to listen to PlayerInfo updates emitted from the checkers module:
- This will avoid tight coupling between the modules.
- The checkers module will not care whether there is a listener or not.
- It will be the duty of
app.goto hook the leaderboard's listener to the checkers emitter. - To reduce the dependency of the leaderboard module on elements of the checkers module, you are going to restrict to a single file.
With the Cosmos SDK, hooks are a design pattern so you have to code them.
# On the checkers module
Add the hooks interface to the checkers module. First as an expected interface:
Here you can imagine you could add functions for all sorts of updates coming from checkers. But for the sake of the exercise keep it simple.
Then, taking inspiration from the governance module's hooks (opens new window), define a convenience multi hook that can accommodate multiple listeners:
Expose this hooks interface via the checkers keeper:
And a function to set it:
Having a function to set the hooks is advised, as that allows you to collect the listeners you need without worrying about the order of creation of other keepers.
With the hooks structure in place, you must have your checkers code call it. The best place for that is precisely where it is updated and saved:
Remember that the hook is named AfterPlayerInfoChanged, not for use by the leaderboard. Therefore you should also emit when there is a change that you know is going to be discarded by the leaderboard.
It verifies != nil to make sure it does not panic if there are no listeners, which is a legitimate situation.
The checkers module is now ready with regards to the hooks.
# On the leaderboard module
In your keeper, define a generic checkers hook listener. In a new keeper/hooks.go file, put a simple:
Then, so as to keep the dependency on checkers' types in as few files as possible, encapsulate the conversion knowledge in a new types/leaderboard_checkers.go:
Now encapsulate the handling in a new keeper/hooks_checkers.go file:
As you can see, this takes the new information and puts it into the transient store only if it is worth doing so.
If your leaderboard hooks listener was set to listen from more than one module, you would add a new hooks_othermodule.go file that only concerns itself with that other module.
The leaderboard handling is now complete.
# On app.go
All app.go has to do is call checkers' SetHooks with the leaderboard's listener after all keepers have been created:
Note how app.CheckersKeeper is replaced. This means that you need to move the checkers module line below:
If you forgot to do so, the module would be created with the hook-less keeper.
# Unit tests
After all these changes, it is worthwhile adding tests.
Just like you did for the checkers module, you can add valid addresses to be reused elsewhere in a new file:
# Candidate unit tests
You added a new Candidate type and helper functions on it. You can test that they work as expected. Add a new leaderboard_test.go file. No need to overdo it:
# Leaderboard helper unit tests
Start by adding tests that confirm that the sorting of the leaderboard's winners works as expected. Here an array of test cases is a good choice:
With that done, you can confirm that the updating or addition of new player info to the leaderboard works as expected, again with an array of test cases:
# Candidate lifecycle unit tests
You added functions to set and get candidates from the transient store. You ought to add unit tests to confirm this works as expected.
First, you need to make sure that your test keeper has a valid transient store:
With this preparation, you can add simple tests. For example, that you get back one candidate (opens new window) when there is one, or three when there are three:
Note the small hack where the received candidates are sorted by WonCount. The GetAllCandidates function does not ensure an order, so to be able to easily use require.Equal an ordering was used.
# Leaderboard handling unit tests
You can verify that the leaderboard is updated when the keeper.CollectSortAndClipLeaderboard function is called.
To change the context time, you can use the SDK context's WithBlockTime function. For instance, test when a single candidate is added between two (opens new window) existing winners. Or when one candidate replaces its lower score and another enters the leaderboard for the first time:
Where:
- You put a leaderboard in storage.
- Put candidates in the transient storage.
- Call the collection of candidates.
- Confirm the new leaderboard order and values.
You can also add a test that confirms the leaderboard is clipped at the maximum length:
Where carol kicked bob out of the leaderboard since its length was enforced at 2:
# Hook unit tests on leaderboard
Moving to the hooks on the leaderboard module's side, you want to confirm that candidates are added to the transient store when the keeper receives a new update:
Confirm also that it overwrites when it receives an update for the same address (opens new window), or adds a second candidate alongside an existing one (opens new window).
# Hook unit tests on checkers
You introduced a new type, the MultiHook. You should test that it indeed distributes calls to the elements of the list. This calls for a mock of the CheckersHooks expected interface.
Run again your existing script that rebuilds all the mocks.
With that, you can add a test that confirms a multihook with two hooks calls both in order:
Your existing checkers keeper tests should still be passing.
However, there is a small difficulty that would not surface immediately: when you set the hooks after the msgServer has been created, because it takes a keeper instance and not a pointer, the msgServer is created with the old keeper (the one before the hooks were set).
Therefore, add a setup function that encapsulates the knowledge to circumvent this difficulty:
Note that it does not care about what happens on the mocked escrow. You can now add a test that confirms that a game just played does not trigger a call to the hooks:
A more interesting addition is the confirmation that a listener is being called when a game is forfeited (opens new window) or won:
# Integration tests
To further confirm that your code is working correctly you can add integration tests. Since it starts an app, the hooks are already set up.
You could decide to piggy-back on the existing "checkers" integration tests. However, for the sake of clarity, create a separate folder:
Copy the integration test suite from the checkers integration tests, with adjusted imports and others, minus all the balances and denoms. Keep the msgServer, as that is the one that receives messages:
You can confirm the leaderboard is called when a game is won:
Or when a game is expired:
Note how you have to call both end blockers because there are actually no blocks being produced. This recalls what you did previously when integration-testing the game forfeit.
This completes your checkers v2 chain. If you were to start it anew as is, it would work. If you want to see how you would migrate your blockchain if it were running v1.1, jump straight to the next section.
# Interact via the CLI
Your v2 blockchain is fully functioning. It will work as long as you start it from scratch (i.e. you should not try to migrate).
You should already know your way around testing this way. The simplest way is to use Ignite:
Use your CosmJS integration tests to run a full game:
After that, you can query your leaderboard:
It should turn something like:
Congratulations, your leaderboard is functional!
If you used Docker, you can stop the container and remove the network:
To summarize, this section has explored:
- How to add a leaderboard as a module to an existing blockchain, and the characteristics that a good leaderboard should boast.
- How to keep modules loosely coupled, when possible, with the use of hooks.
- How to leverage the transient store to save data for use in
EndBlock. - How to reduce computations and overall blockchain burden by ordering the leaderboard only once per block, in
EndBlock. - Worthwhile unit tests, including recreating the mocks, and integration tests.