Good luck
2 Likes
April Progress:
I’ve built out a functional testing framework that builds valid headers, blocks, and transactions. I’ve now got some (mostly) complete end-to-end block validation tests.
I also began integrating with the litecoin codebase, starting with the ConnectBlock logic (See: Bitcoin Core 0.11 (ch 6): The Blockchain - Bitcoin Wiki). This is the part where blocks are validated to ensure valid UTXOs are being spent, no double spends, etc, and is also responsible for actually adding the blocks to the chain.
I haven’t created the repo for this portion yet, but will try to get that out on github sometime in the next few weeks, once I’ve had the chance to clean the code up a bit.
There’s a few different directions I could go once I finish testing the ConnectBlock logic, but I’m still deciding what makes sense to tackle next. All I can give at this time is a high level plan for May, which involves continued integration with the LTC codebase, and lots more testing.
It’s been a few months since I gave an update on the Grin++ side, so for those interested how that’s going, there’s been some big news recently. David Tavarez (davidtavarez (David Tavarez) · GitHub) has completely taken over the UI development, so I can focus solely on the C++ node & wallet logic. He actually rewrote the entire UI (https://github.com/GrinPlusPlus/GrinPlusPlusUI/tree/1.0) and it’s much nicer than it was before. Things are so stable now that we actually have a release candidate for v1.0.0, what we’re calling the first non-beta version of Grin++. That release has been tested with the help of several Grin community members, and will be pushed to all Grin++ users this weekend.
Happy Friday Everyone!
7 Likes
Hi David, thanks for the update. We appreciate your hard work & we I’ll continue to raise litecoin and bitcoin to fund your work.
You are building the future!
4 Likes
Hi David,
Is there any way we could contribute with our time? I’m sure there are a lot of programmers out there that are willing to chip in especially for the mundane bits.
4 Likes
Absolutely! That would be amazing. I’ve had a few manual testers and 1 or 2 UI devs message me to offer help, but it is a bit too early to pull them in. I can use c++ devs at anytime though, and if I take a few days to add RPC APIs, I could even use python devs interested in writing some functional tests.
I encourage any would-be contributors to message me on Telegram (Telegram: Contact @DavidBurkett) or on Keybase (dburkett (David Burkett) | Keybase)
3 Likes
May Progress:
I’ve finished implementing the AcceptBlock and ConnectBlock integration logic, meaning that when a mimblewimble block is received, it will be fully validated, added to the chain, and the UTXO set will be updated. This ended up requiring a lot more rework than I hoped, since the approach I originally took on the mimblewimble side for adding, validating, and removing blocks didn’t line up well with the existing code’s approach to handling the UTXO set using “views” and “caches” (See “Data Access Layer and Caching” here: Bitcoin Core 0.11 (ch 2): Data Storage - Bitcoin Wiki).
The logical next step was to work on DisconnectBlock, which will remove a block from the chain in the case of chain reorgs. However, I chose instead to start working on the mempool, with the hopes that we can soon start mining Mimblewimble blocks. Being able to mine valid mw blocks will allow us to test the end-to-end flow of including mw transactions in a block, validating those blocks, removing the transactions from the mempool, adding the blocks to the chain, updating the UTXO set, and flushing everything to disk. There’s a lot that could go wrong there, so the sooner we can get everything working together, the smoother the rest of development should go.
This is all subject to change, but for now, here’s the plan between now and the end of Summer testnet launch:
June: Wrap up mempool logic, and start mining valid chains with mw extension blocks, making sure all pieces work well together.
July: Initial block download. The way initial block download is handled in mimblewimble is quite a bit different than how litecoin does it. In LTC, every single block from the genesis block is downloaded and validated in order. With mimblewimble, we don’t need the whole history, but instead only parts of each transaction, and all of the unspent outputs. I’ll need to find a way to make those two very different syncing approaches work smoothly together.
August: DisconnectBlock logic (for chain reorgs).
September: Testnet activation logic and launch of the first mimblewimble testnet. Then start fixing everything I broke 
8 Likes
Thanks for the update David 
1 Like
Courage David! It’s almost done
Nice! Great work. Thank you
This is great. I really appreciate your ability to discuss the details of what you’re working on, and your plan to tackle it so clearly and concisely. That’s a rare skill.
Keep trucking David appreciate everything your doing:raised_hands:
June Progress:
The existing LTC mempool logic ended up being quite a bit more complex than I predicted, so we’ll need to revisit this area after the testnet launches. For now though, a very minimal implementation supporting MW transactions has been written.
In addition to the mempool changes, code was written to support mining extension blocks, although there are a few edge cases left to handle, and much more testing is needed.
I’ve had concerns about the way we were storing mimblewimble block data in a separate database. It was originally designed this way to be a clean separation from the existing code, in order to facilitate merging future bitcoin commits. Having separate databases is generally a bad idea though, because we lose the ability to make atomic updates, which means the 2 databases could become out of sync. This can lead to a whole host of problems, potentially even ones that are exploitable by remote attackers.
Because of these concerns, I decided to spend some time modifying the code for serializing and deserializing MW blocks & transactions to disk. I was able to take advantage of the groundwork that was laid down as part of the segwit enhancement, to cleanly support serializing the additional data without making major changes to the existing block storage format. As a result, upgraded nodes can successfully save extension block data to disk the same place they’ve always saved blocks, without having to introduce an additional database. A side effect of these changes is that it was relatively straightforward to add support for sharing mimblewimble transactions over the p2p network, which is the first step toward July’s goal of handling MW data as part of the Initial Block Download.
The high-level plan for the rest of the summer remains the same:
July: Initial Block Download
August: Chain reorg logic
September: Activation logic and testnet launch
15 Likes
Outstanding work David, always a treat to here your monthly updates
1 Like
Eagerly awaiting the next update !!
2 Likes
Same here 
! I am checking this website several times a day 
2 Likes
July Progress:
The focus this month was on the Initial Block Download. Before I can detail the progress made, I need to give some background info for those not intimately familiar with mimblewimble.
The biggest innovation behind mimblewimble is that, in order to verify the chain, you just need to know all of the unspent coins/outputs, and a small part of each transaction called the “Kernel.” These 2 things together are called the “chain state.”
In bitcoin/litecoin, each block header uses a merkle tree to commit to only the transactions in that block. Since we don’t want to require everyone to download all old mimblewimble blocks, or to know about all old, spent outputs, we use a different structure to commit to the transactions. Each mimblewimble header commits to the root of 2 different Merkle Mountain Ranges(MMRs). One represents all historical kernels up to that block, and the other represents all historical outputs/coins. Merkle Mountain Ranges are a different sort of tree that supports “pruning”, which means we can verify the root of the structure without knowing all of its members (called leaves). For an in-depth look at how this works, I recommend reading grin/mmr.md at master · mimblewimble/grin · GitHub
Since old blocks don’t need to be downloaded, Initial State Download (ISD) is a more accurate term for the initial sync process that I worked on this month. To facilitate the ISD I ended up defining the following P2P messages:
-
GETMWSTATE - This simply consists of the hash of a header sometime in the recent past. We’ll call this the “horizon” header. A new node, after syncing most of the canonical (non-mimblewimble) blocks in the chain, sends this message to a single peer, requesting the mimblewimble chain state at the time of that block.
-
MWSTATE - The response to a GETMWSTATE message. This contains the entire chain state, which is everything you need to validate the extension chain up to the point of the horizon. The chain state consists of:
- kernels - All of the kernels in the chain, in order.
- utxos - All of the outputs that had not yet been spent at the time of the horizon, including their associated rangeproofs, in order.
- unspent_bitmap - A compact structure that indicates the MMR leaf indices of the provided utxos.
- parent_hashes - Some additional metadata that allows you to verify the MMR root without knowing all of the past, spent outputs.
Quick sidenote - This design relies on one peer providing quite a bit of data at once. After launch, it would be ideal to parallelize the sync process using something like grin-rfcs/0000-parallel_ibd.md at pibd · jaspervdm/grin-rfcs · GitHub or UTXO set, horizons and cut through · BeamMW/beam Wiki · GitHub
All of the logic for serializing and deserializing those messages, as well as choosing when to send them has been implemented. I also implemented the logic for verifying the kernel signatures, and the kernel MMR roots.
Unfortunately, I was short on time this month due to a Grin hardfork that ended up requiring an enormous amount of effort to support in Grin++. As a result, I was unable to finish up the logic for validating and processing MWSTATE messages - in particular, the output roots are not yet verified, and I still need to validate that the kernels match up with the pegins and pegouts for each block. Grin++ is now in maintenance mode for the remainder of the year, so this should be the last time it interferes with litecoin progress.
Updated Plan:
The plan was to spend August focusing on reorg logic, but I’m going to delay that until September. Instead, I’m just going to focus on wrapping up the MWSTATE message processing, and focus on cleaning up the code, adding more tests, and maybe even some documentation if I’m feeling really wild. When developing projects this large, things rarely go exactly as planned, so the design changes over time. Having a chance like this to go back through the code and clean things up, re-evaluate decisions made, etc. will really pay off in terms of code quality, hopefully allowing for a smooth first testnet launch at the end of September.
13 Likes
Thanks for the update David. We appreciate all of your hard work. You are making litecoin fungible & this is the last feature needed for a global currency.
We look forward to testing launch next month.
Thank you @masterbtcltc
3 Likes
Keep trucking David, thanks for sharing. I hope all is well on the Grin side. Definitely doing it right and spending extra time to clean things up is the way to go about, it’s the Łitecoin way lol. We appreciate your hard work and honored to have you apart of the ŁTC community 
. I hope all is well with you and yours on a personal level as we continue life in this altered state as we know it, and look forward to August updates. Cheers.
4 Likes
David, thanks for keeping good work up!
2 Likes