# Multistore and Keepers
It is time to address multistore and keeper. Before you begin, take a look at the following sections:
You can find a code example for your checkers blockchain at the end of the section to explore dealing with storage elements, message handling, and gas costs.
A Cosmos SDK application on an application-specific blockchain usually consists of several modules attending to separate concerns. Each module has a state that is a subset of the entire application state.
The Cosmos SDK adopts an object-capabilities-based approach to help developers better protect their application from unwanted inter-module interactions. Keepers are at the core of this approach.
A keeper is a Cosmos SDK abstraction that manages access to the subset of the state defined by a module. All access to the module's data must go through the module's keeper.
A keeper can be thought of quite literally as the gatekeeper of a module's store(s). Each store, typically an IAVL store, defined within a module comes with a storeKey. The storeKey grants unlimited access to it. The module's keeper holds this storeKey, which should otherwise remain unexposed, and defines methods for reading and writing to the store(s).
When a module needs to interact with the state defined in another module, it does so by interacting with the methods of the other module’s keeper. Developers control the interactions their module can have with other modules by defining methods and controlling access.
# Format
Keepers are generally defined in a /keeper/keeper.go file located in the module’s folder. The type keeper of a module is named simply keeper.go by convention. It usually follows the following structure:
# Parameters
The following parameters are of importance concerning the type definitions of keepers in modules:
- An expected
keeperis a keeper external to a module that is required by the internal keeper of said module. External keepers are listed in the internal keeper's type definition as interfaces. These interfaces are themselves defined in anexpected_keepers.gofile in the root of the module's folder. Interfaces are used to reduce the number of dependencies and to facilitate the maintenance of the module itself in this context. storeKeysgrant access to the store(s) of the multistore managed by the module. They should always remain unexposed to external modules.cdcis the codec used to marshal and unmarshal structs to/from []byte. Thecdccan becodec.BinaryCodec,codec.JSONCodec, orcodec.Codecbased on your requirements. Note thatcode.Codecincludes the other interfaces. It can be either a proto or amino codec as long as they implement these interfaces.
Each keeper has its own constructor function, which is called from the application's constructor function. This is where keepers are instantiated and where developers make sure to pass correct instances of the module's keepers to other modules that require them.
# Scope and best practices
Keepers primarily expose getter and setter methods for the store(s) managed by their module. Methods should remain simple and strictly limited to getting or setting the requested value. Validity checks should already have been done with the ValidateBasic() method of the message and the Msg server before the keeper's methods are called.
The getter method will typically have the following signature:
The setter method will typically have the following signature:
Keepers also should implement an iterator method with the following signature when appropriate:
# Store types
The Cosmos SDK offers different store types to work with. It is important to gain a good overview of different store types available for development.
# KVStore and Multistore in Cosmos
Each Cosmos SDK application contains a state at its root, the Multistore. It is subdivided into separate compartments managed by each module in the application. The Multistore is a store of KVStores that follows the Multistore interface (opens new window).
The base KVStore and Multistore implementations are wrapped in extensions that offer specialized behavior. A CommitMultistore (opens new window) is a Multistore with a committer. This is the main type of multistore used in the Cosmos SDK. The underlying KVStore is used primarily to restrict access to the committer.
The rootMulti.Store (opens new window) is the go-to implementation of the CommitMultiStore interface. It is a base-layer multistore built around a database on top of which multiple KVStores can be mounted. It is the default multistore store used in BaseApp.
# CacheMultistore
A cachemulti.Store (opens new window) is used whenever the rootMulti.Store needs to be branched. cachemulti.Store branches all substores, creates a virtual store for each substore, in its constructor and holds them in Store.stores. This is used primarily to create an isolated store, typically when it is necessary to mutate the state but it might be reverted later.
CasheMultistore caches all read queries. Store.GetKVStore() returns the store from Store.stores, and Store.Write() recursively calls CacheWrap.Write() on all substores.
# Transient store
As the name suggests, Transient.Store is a KVStore that is discarded automatically at the end of each block. Transient.Store is a dbadapter.Store with a dbm.NewMemDB(). All KVStore methods are reused. A new dbadapter.Store is assigned when Store.Commit() is called, discarding the previous reference. Garbage collection is attended to automatically.
Why not take a closer look at the IAVL spec (opens new window) when working with the IAVL store?
The default implementation of KVStore and CommitKVStore is the IAVL.Store. The IAVL.Store is a self-balancing binary search tree that ensures get and set operations are O(log n), where n is the number of elements in the tree.
# Additional KVStore wrappers
Beside the above store types, there are a few others with more specific usage.
# GasKv store
Cosmos SDK applications use gas to track resources usage and prevent spam. The GasKv.Store is a KVStore wrapper that enables automatic gas consumption each time a read or write to the store is made. It is the solution of choice to track storage usage in Cosmos SDK applications.
GasKv.Store automatically consumes the appropriate amount of gas depending on the Store.gasConfig when methods of the parent KVStore are called. All KVStores are wrapped in GasKv.Stores by default when retrieved. This is done in the KVStore() method of the context. The default gas configuration is used in this case.
# TraceKv store
tracekv.Store is a wrapper KVStore, which provides operation tracing functionalities over the underlying KVStore. It is applied automatically by the Cosmos SDK on all KVStores if tracing is enabled on the parent MultiStore.
When each of the KVStore methods are called, tracekv.Store automatically logs traceOperation to the Store.writer. traceOperation.Metadata is filled with Store.context when it is not nil. TraceContext is a map[string]interface{}.
# Prefix store
prefix.Store is a wrapper KVStore which provides automatic key-prefixing functionalities over the underlying KVStore:
- When
Store.{Get, Set}()is called, the store forwards the call to its parent with the key prefixed with theStore.prefix. - When
Store.Iterator()is called, it does not simply prefix theStore.prefixsince it does not work as intended. Some of the elements are traversed even when they are not starting with the prefix in this case.
# AnteHandler
The AnteHandler is a special handler that implements the AnteHandler interface. It is used to authenticate the transaction before the transaction's internal messages are processed.
The AnteHandler is theoretically optional but still a very important component of public blockchain networks. It serves three primary purposes:
- It is a primary line of defense against spam and the second line of defense - the first one being the mempool - against transaction replay with fees deduction and sequence checking.
- Perform preliminary stateful validity checks like ensuring signatures are valid or that a sender has enough funds to pay for fees.
- Play a role in the incentivization of stakeholders via the collection of transaction fees.
BaseApp holds an AnteHandler as a parameter that is initialized in the application's constructor. The most widely used AnteHandler is the auth module.
For more information on the subject, a closer look at the following resources could prove worth it:
# Next up
In the next section, you will find more information on BaseApp and its role in the Cosmos SDK.
In the Accounts section, you were introduced to the elements of the stored game but were left in the dark about where this game is stored. In light of what you learned above, fix that.
# Game object in storage
You need to decide under what structure you want to store a game in the storage. The Cosmos SDK partitions the global storage per module with checkers being its own module. You need to take care of how to store games in the checkers module's corner of the key/value pair storage.
The first idea would be to attribute a unique ID to a game and to store the game value at that ID. For the sake of clarity and to be able to differentiate with other stored elements in the future, you ought to add a prefix to that ID. The storage structure would look like this:
It is pseudo-code because:
- Prefixes have to be
byte[]instead ofstring. This is easily handled with a cast such as[]byte("games-"). - The Cosmos SDK prevents you from directly accessing any random module's store, such that
globalStore.getAtPrefix("checkers-")is not allowed and instead has to be accessed via a secret key.
Define the ID of the StoredGame. To return a single object, we include StoredGame in the object's value:
With most of the action handled by the Cosmos SDK, you are left with defining the required prefixes in your corner of the storage:
Which assists you with accessing a game:
If you want to save a game:
If you want to delete a stored game, you would call gamesStore.Delete(byte[](storedGame.Index)).
The KVStore allows you to obtain an iterator on a given prefix. You can list all stored games because all stored games share the same prefix, which you would do with:
Notice the MustMarshalBinaryBare and MustUnmarshalBinaryBare functions in the codec above. They need to be instructed on how to proceed with the marshaling. Protobuf took care of this for you.
See the previous section on Protobuf to explore how Protobuf deals with the marshaling.
# Boilerplate, boilerplate everywhere!
Also notice how the Set, Get, Remove, and GetAll functions above look like boilerplate too. Do you have to redo these functions for every type? You do not. It was all created with this Starport command:
Where map is the command that tells Starport to add an Index and store all elements under a map-like structure.
To create the above boilerplate in your module, you can use Starport. For Starport and if you want to go beyond these out-of-context code samples to see more in detail how to define all this, head to My Own Chain.
# Other storage elements
How do we create this storedGame.Index? A viable idea is to keep a counter in storage for the next game. Unlike StoredGame, which is saved as a map, this NextGame object has to be at a unique location in the storage.
We define the object:
Plus, the key where it resides:
Then the functions to get and set:
Not to forget that it needs an initial value, which is the role of the genesis block definition:
And its initialization:
# What about message handling
You defined the MsgCreateGame in an earlier section on messages.
You go from the message to the game in storage with the MsgCreateGame. That is also the role of the keeper. You should define a handling function such as:
It looks like you now have all the pieces to replace the TODO, which turns out to be straightforward.
Get the next game ID:
Extract and sanitize the message elements:
Create the stored game object:
Save it in storage:
Prepare for the next created game:
Return the game ID for reference:
You would do the same for MsgPlayMoveResponse and MsgRejectGame. Why not try it out as an exercise?
# More on game theory
Time to introduce a game deadline:
Set its initial value on creation:
Update its value after a move:
Extract and verify its value when necessary:
# How to expire games
When do you verify that a game has expired? An interesting feature of an ABCI application is that you can have it perform some actions at the end of each block. Should you load all games and filter for those that have expired? That would be extremely expensive. Better keep a First-In-First-Out (FIFO) where fresh games are pushed back to the tail so that the head contains the next games to expire.
In the context of the Cosmos SDK, you need to keep track of where the FIFO starts and stops by saving the corresponding game IDs:
And to have each game know its relative position and the number of moves done to assist the refunding logic on games with zero, one, or more than two moves:
Next, you need to code a regular FIFO, whereby:
- Games are sent to the back when created or played on.
- Games are removed from the FIFO when they are finished or time out.
# When to expire games
Since you want to expire the games that have timed out at the end of a block, you need to hook your keeper to the right call. The Cosmos SDK will call at various points into each module when building the whole application. The function it calls at each block's end looks like this:
This is where you write the necessary code, preferably in the keeper. For instance with:
Those among you with a well-placed paranoia must be asking whether you can ensure that the execution of this EndBlock does not become prohibitively expensive. The number of games to potentially expire is unbounded after all, which is a recipe for disaster in the blockchain world. Is there a situation or an attack vector that would make this a possibility? And what can you do to prevent it?
The timeout duration is fixed and is the same for all games. This means that the n games that expire in a given block have all been created or updated at roughly the same time or roughly the same block height h, give or take a margin of error h-1 and h+1. These created and updated games are limited in number because as per one of the chain consensus parameters every block has a maximum size and a limited number of transactions it can include. If by any chance all games in blocks h-1, h, and h+1 expire now, then the EndBlock function would have to expire three times as many games as a block can handle. This is the worst-case scenario but it still sounds manageable.
You should be careful about letting the game creator pick a timeout duration. It could open an avenue for a malicious actor to stagger game creations over a large number of blocks with decreasing timeouts so that they all expire at the same time.
# Gas costs
The keeper also makes it easy for you to charge the gas to the players as you see fit. This gas fee comes on top of the configured standard fee for transactions on your chain. Propose some ratios, which would have to be adjusted so it makes sense compared to the base transaction costs:
- Create a game: costs 10. It should include the costs of closing a game. If that was not the case, the losing player would be incentivized to let the game hit its timeout.
- Play move: costs one. You could make it cost zero when the player loses the game to incentivize the player to conclude the game instead of letting it hit the timeout.
- Reject: costs zero, because you want to incentivize cleaning up the state. This transaction would still cost what your chain is configured to charge for basic transactions.
So you define the cost:
Then, you add the line in your MsgCreateGame handler: