# Project Report - M2 Delivery

https://forum.moonbeam.foundation/t/proposal-xx-stakeglmr-com-and-stakemovr-com-v2-treasury-proposal/801

## Code Stats Summary

|Component|Language|Lines of Code|
| --- | --- | --- |
|SvelteKit App|Typescript|14,740|
|SvelteKit App|Svelte|9,912|
|Chat Server|Typescript|4,346|
|Staking Indexer|Typescript|11,730|
|Deep State Indexer|Typescript|8,888|

**Total Typescript Lines: 39,704**

Initial Schedule: 2-6 months Actual Time: 1 year and 2 months

## Milestone Status

### Milestone 1

* ✅ SvelteKit frontend app, with mock db calls
* ✅ Integration of app with wallets, chain queries (TODO: extrinsic calls)
* ✅ Indexing backend and ETL services

### Milestone 2

* ✅ Additional microservices (pending only the collator management services)
* ✅ Full frontend + backend integration (pending only the integration of the above)
* ✅ Backend Testing
* TODO: Frontend Testing
* TODO: Deployment to production

## Features planned and delivered
* New dashboards with more staking info for collators
* New personal account dashboards for delegators
* Faster load time - V2 loads all data straight from the database on a single http call. Collator details data load on separate call, when requested.
* We ended up not using Subsquid and built our own custom indexing pipeline using Temporal for cloud-level process durability on our own hardware.
* Wallet Connect was used to support multiple wallets.
* The new setup allows infinite resolution times, should an error occur during the ETL process. Of course, this does cause the indexer to fall behind, but there is no deadline by which data will start getting lost.
* Cost-efficiency: the new setup requires one 48-core Hetzner server to house the database and workflow/activity programs ($200 per month), one vercel app ($30), and one S3-compatible data bucket. It also requires Temporal cloud services (we don't run our own temporal system) which is a flat $50 (pro-rata, as i also run other services on Temporal) + $10 variable.

## Features not planned and delivered
# Deep State Substrate Indexer.
We have built an indexing pipeline that goes well beyond indexing what is readily available, i.e. extrinsic, event, and block data, to indexing the values returned by pallet methods, for any valid arguments. The indexer is built around substrate and the Polkadot.js library. In a nutshell, it works by reading the chain metadata, deconstructing every method input and output values to basic types, creating new basic and complicated inputs, and feeding them back into the methods, and repeating the process (outputs create new inputs).
* Chat interface for the staking data and deep state indexer. This is a ChatGPT assistant based interface that uses ~35 methods to query the database. I plan to add more methods in the future that will provide some interesting graph-based abilities to the bot.

## Features planned and not delivered yet
* Multiple collator staking functionality. This includes the backend logic for recommending staking allocations, and the evm calls on the frontend.
* Full integration with insurance contracts. The contract code is there but i have to connect it to the frontend.
* Push updates of dashboard information via websockets. This is supported out of the box by surrealdb but the functionality is still a bit buggy. Waiting for it to mature. Then, it should be as simple as rewriting the current db queries to include a LIVE keyword and opening surrealdb's websocket port.

## Features planned but will not be delivered
We will not use a two-layer indexing architecture due to the high computing cost required by the deep state indexer. Instead, we will backup the data extracted files (large jsons, initially cached in s3) and take snapshots of rocksdb.

## Next Steps

### Production Indexing

Catching up to the chain from block 0 will be a challenge. Using a single 48-core machine would take around 7 months to get to block 6M, and by that time the chain would be well ahead especially given the 6-second block periods. Fortunately, the process can be parallelized across multiple servers. I would like to wait for surrealdb 2 to hit the production version before starting syncing from block 1. We are currently using surrealdb 1.5

### Finishing Up Pending Functionality

I will finish up all remaining tasks, but I can't work full-time on this anymore. Indexing will take some time anyways, so I can probably finish everything over part-time weekends.

## M2 Payment

| Total   | M2 Payment + 2K | Token    | 30D MA  | Quantity | Unit  |
|---------|-----------------|----------|---------|----------|-------|
| $44,800 | $24,400         | Moonbeam | 0.1958  | 99,694   | GLMR  |
|         |                 | Moonriver| 9.3326  | 523      | MOVR  |



- **Problem Statement:** Delivering the M2 project within the proposed timeframe and budget to enhance staking information for collators and delegators.
- **Significance:** Enhancing dashboards, load times, and data indexing for improved user experience and efficiency.
- **Proposed Solution:** Utilizing a new setup with custom indexing pipelines and cloud services to support staking functionalities and data management.
- **Expected Outcomes:** Improved staking information dashboards, faster load times, and cost-efficient infrastructure for better user engagement and data management.

#26 StakeGlmr & StakeMovr V2 - M2 Delivery