MWEB Progress Update Thread

July Progress:

The focus this month was on the Initial Block Download. Before I can detail the progress made, I need to give some background info for those not intimately familiar with mimblewimble.

The biggest innovation behind mimblewimble is that, in order to verify the chain, you just need to know all of the unspent coins/outputs, and a small part of each transaction called the “Kernel.” These 2 things together are called the “chain state.”

In bitcoin/litecoin, each block header uses a merkle tree to commit to only the transactions in that block. Since we don’t want to require everyone to download all old mimblewimble blocks, or to know about all old, spent outputs, we use a different structure to commit to the transactions. Each mimblewimble header commits to the root of 2 different Merkle Mountain Ranges(MMRs). One represents all historical kernels up to that block, and the other represents all historical outputs/coins. Merkle Mountain Ranges are a different sort of tree that supports “pruning”, which means we can verify the root of the structure without knowing all of its members (called leaves). For an in-depth look at how this works, I recommend reading grin/mmr.md at master · mimblewimble/grin · GitHub

Since old blocks don’t need to be downloaded, Initial State Download (ISD) is a more accurate term for the initial sync process that I worked on this month. To facilitate the ISD I ended up defining the following P2P messages:

  • GETMWSTATE - This simply consists of the hash of a header sometime in the recent past. We’ll call this the “horizon” header. A new node, after syncing most of the canonical (non-mimblewimble) blocks in the chain, sends this message to a single peer, requesting the mimblewimble chain state at the time of that block.

  • MWSTATE - The response to a GETMWSTATE message. This contains the entire chain state, which is everything you need to validate the extension chain up to the point of the horizon. The chain state consists of:

    • kernels - All of the kernels in the chain, in order.
    • utxos - All of the outputs that had not yet been spent at the time of the horizon, including their associated rangeproofs, in order.
    • unspent_bitmap - A compact structure that indicates the MMR leaf indices of the provided utxos.
    • parent_hashes - Some additional metadata that allows you to verify the MMR root without knowing all of the past, spent outputs.

Quick sidenote - This design relies on one peer providing quite a bit of data at once. After launch, it would be ideal to parallelize the sync process using something like grin-rfcs/0000-parallel_ibd.md at pibd · jaspervdm/grin-rfcs · GitHub or UTXO set, horizons and cut through · BeamMW/beam Wiki · GitHub

All of the logic for serializing and deserializing those messages, as well as choosing when to send them has been implemented. I also implemented the logic for verifying the kernel signatures, and the kernel MMR roots.

Unfortunately, I was short on time this month due to a Grin hardfork that ended up requiring an enormous amount of effort to support in Grin++. As a result, I was unable to finish up the logic for validating and processing MWSTATE messages - in particular, the output roots are not yet verified, and I still need to validate that the kernels match up with the pegins and pegouts for each block. Grin++ is now in maintenance mode for the remainder of the year, so this should be the last time it interferes with litecoin progress.

Updated Plan:

The plan was to spend August focusing on reorg logic, but I’m going to delay that until September. Instead, I’m just going to focus on wrapping up the MWSTATE message processing, and focus on cleaning up the code, adding more tests, and maybe even some documentation if I’m feeling really wild. When developing projects this large, things rarely go exactly as planned, so the design changes over time. Having a chance like this to go back through the code and clean things up, re-evaluate decisions made, etc. will really pay off in terms of code quality, hopefully allowing for a smooth first testnet launch at the end of September.

13 Likes

Thanks for the update David. We appreciate all of your hard work. You are making litecoin fungible & this is the last feature needed for a global currency.

We look forward to testing launch next month.

Thank you @masterbtcltc

3 Likes

Keep trucking David, thanks for sharing. I hope all is well on the Grin side. Definitely doing it right and spending extra time to clean things up is the way to go about, it’s the Łitecoin way lol. We appreciate your hard work and honored to have you apart of the ŁTC community :metal::owl::black_heart::boom::boom:. I hope all is well with you and yours on a personal level as we continue life in this altered state as we know it, and look forward to August updates. Cheers.

4 Likes

David, thanks for keeping good work up!

2 Likes

I apologize in advance for the much-briefer-than-usual update. I’ve been busy with tying up loose ends, which are important & necessary, but not very exciting to write about.

August Progress:

As planned, I was able to wrap up the Initial State Download, in particular the validation of the MWSTATE messages. There are still a number of ways where malicious peers can use these new messages to DoS nodes. I’m not yet guarding against all of those attacks, but there will be plenty of time to focus on that after the testnet is released.

I also took advantage of the opportunity to go back and clean up some code & strengthen some of the more fragile logic, particularly around management of the UTXO set (the collection of unspent coins).

Remaining work:

Everything is on track for a testnet release at the end of this month. I’m still figuring out a few final details, but I’ll share more information later this month about what the testnet will look like, and how everyone can get involved. I’m hoping to have a basic CLI or JSON-RPC option for advanced users to experiment with while I transition to focusing on adding non-interactive transaction support.

Sidenote:

For those interested, I recently gave a talk about future directions in Mimblewimble which highlighted the weaknesses of vanilla Mimblewimble, and what we can do/are doing to try to solve those. Here’s a recording of the talk: David Burkett Grin ++ |Crypto Chat - YouTube

13 Likes

Hi David. Looking forward to watching the YT video.

We appreciate all your hard work on adding scalable fungibility to litecoin.

We look forward to the testnet launch & ultimately the mainnet launch.

3 Likes

can’t wait to try out the cli/rpc

3 Likes

Top notch bro! As usual :metal::owl::black_heart:Ł

Hey @DavidBurkett38

I’ve had this idea. Transferring crypto via wallet is dangerous, by mistaking wallet you lose everything. It is the worst nightmare. Why not use an optional send / receive password which, if incorrect/untyped, sends funds back? #Litecoin

5 Likes

This would be awesome, don’t know if it’s possible tho

Dear David
Any updates about the testnet?

3 Likes

Im waiting for the testnet since long days but when the date is near they wont give us any update as usual.

1 Like

I’m still wrapping up the final details. Building something like this is far from easy, but I’m doing my absolute best to make it a success. I will loop everyone in as soon as I can.

16 Likes

the deadline is near still no update :rofl: :joy:

2 Likes

Please wait patiently, trust me in david, trust mimblewinble

5 Likes

This cat right here :clown_face::clown_face::clown_face::clown_face::clown_face::clown_face::clown_face::clown_face::clown_face::clown_face:

4 Likes

:clown_face::clown_face::clown_face::clown_face::clown_face::clown_face::clown_face::clown_face::clown_face:

2 Likes

Interesting idea.

1 Like

Correct me if I’m wrong, but wasn’t the MW testnet going to launch at end September? Are we waiting for the last second? Just asking, no hard feelings.

3 Likes

Of course we are! :laughing:

It’s going to launch this evening EST time. For developers and other technical folks who want to participate in initial testing, please join the telegram group at Telegram: Contact @MWEB_testnet and please wait patiently for instructions.

For non-technical users, it may be a bit difficult to get involved now, but I’m working on making it easier for everyone to test. This could take several days though.

Thanks everyone for your patience. I will have a detailed update about the launch late tonight.

10 Likes