MerkleMe API

Welcome to another what's going to be a great DevNTell. If you didn't know what DevNTell is, it's a 30-minute window for members of the DAO to showcase something they're passionate about or have been working on. This can be an awesome project you've been working on, demonstrating unit testing best practices, automation goodies, smart contracts, how to structure a project, etc. Basically, if you've got a passion for something, this is your opportunity to share it with the community. And I am happy to pass it over to team MerkleMe, who is going to show off their Web3Con hackathon project, the MerkleMe API. Over to you, gang.

Thank you, Narb. So yeah, thanks for having us again. We're, like Narb said, we're just going to give a quick informal introduction to our Web3Con hackathon project, MerkleMe. One thing that Jose was talking about before we started was he was wondering how much people knew about Merkle trees in general. So is there anyone who has no idea what a Merkle tree is or what Merkle verification is? Don't be afraid. I think it'd be worthwhile if you give like a brief overview of it.

Okay, I'm going to leave that to Jose. I'm just here to give the introduction and just give a little background. So MerkleMe, the project, was me, Jose, Orlando, and Kenny. I'm not sure if Kenny is in the chat right now; doesn't look like it, but yeah. Jose came to us with this idea, this really good idea, to create kind of a backend API to abstract away Merkle tree verification and proof process. We just got organized pretty quickly and we went straight at it for the Web3Con hackathon and we created a small prototype, a very simple prototype, that we're quite proud of and yeah. So I'll give it over to Jose and he can give the Merkle tree background.

Awesome. Thanks, Vinny. So I'm going to share my screen real quick here and I've got a little presentation, so I'll start with that. I guess I'll just quickly mention that Jose was kind of responsible for the original idea. Kenny and Orlando took care of UI and frontend, and I kind of did the most of the backend implementation. Awesome. Thanks, Vinny. So yes, so this is MerkleMe and the idea behind MerkleMe is that we want to abstract away the process of using and implementing Merkle trees for different applications. The primary application that we're kind of focused on right now is just using it for whitelists. And so the idea is you generate, you publish, you verify.

I want to give a quick primer on what a Merkle tree is for folks that may not understand what they are. So a Merkle tree is just a hash-based data structure. It's in the form of a binary tree and where you have at most two children per node. So for example, in the tree you see here, you'll start at the very bottom where you have, say, a list of four addresses, and each of those is re-hashed recursively up to the top where you get to your root hash. And so that's what a Merkle tree looks like. The cool property of Merkle trees is that because of the recursive hashing, for any given list of addresses, for example, once you get to the top, the root hash, you're pretty much guaranteed to always have the same root hash if you have the same base set of data. And so it's easy to verify the underlying data that it hasn't changed and you can do that without needing to see the entire set of data. And so this is used for data verification on distributed systems.

So one interesting thing to know is that GitHub uses Merkle trees for commits. I didn't know that until I did a little deep dive into Merkle trees. As far as how you verify data, so for example, if I have a list of four addresses down here at the bottom and I want to create a Merkle tree from this list of addresses, I'll get up to the root hash and now all I need is the root hash. So for example, if I wanted to verify the second address to calculate the root hash, I wouldn't need my entire list of addresses to do that. I would need to generate a proof. And I'm going through this quickly and I apologize if this... I just don't want to spend... you can spend hours talking about Merkle trees. So I'm just trying to do this very quickly.

So if I just wanted to verify, for example, address two, I could do it by generating a proof. And you do that by basically hashing all of these till you get to the root hash. And I wouldn't need all four addresses to do that. I would just need to know the minimum pieces of information I would need to know are the address itself, the neighboring address, and then this hash here. So the way it would work is I would have this address so I can hash it. I would need this address so I can hash it and then I can get this hash. And then with this hash and this hash, I can get the root hash. So there's an algorithm in place—we didn't do this, these are built-in libraries that we're integrating—that will basically do this. It'll tell you the minimum pieces of information you need to get to the root hash. So it's an efficient way to calculate the root hash without having to know all of the underlying data.

So here's an example where we're generating a proof where we want to get the proof for this address. And we're just basically getting the minimum pieces that we need, which is this and this, to get to the root hash. And then what a proof is, is it's an array of nodes that you can use to generate the root hash without needing to know the entire data set. So each leaf will have its own proof. All right. So that was the quick primer on that and I'll give a demonstration of what that looks like. But use case for Merkle trees is using NFT whitelists where a project owner may want to open up their minting contract to a specific list of folks or addresses. And so right now the implementation is fairly complicated and time-consuming and usually requires running a node backend to generate the trees and the proofs. So we want to abstract that away and make it easier.

So here's a quick overview of how that works in MerkleMe is you bring your whitelist, you upload your whitelist to MerkleMe, the API will then generate a Merkle tree for you, and then it'll publish to IPFS your whitelist, your root hash, and something else I call tree summary. Not important right now, I need to improve the documentation, but generally, the two pieces you need to know are your whitelist and your root hash. So MerkleMe will do that. All you do is copy and paste your whitelist and then it will generate this for you. So once you get there, this is the generate and publish aspect of MerkleMe. Then you get to the verify aspect. And this is where we take the proof for a specific leaf and then we generate it for that leaf. So the frontend, for example, will take an address and then we'll request a proof from the MerkleMe API and then it will provide that proof back and then with this proof you can interact with your minting contract to verify if they are indeed on the whitelist and they can mint or not.

So I'm going to stop there and give a quick demonstration of sort of how this works. So let's say I've got an NFT project and I want to create a whitelist for my project. So I'll come to MerkleMe and I'll get started. I'll just call this anything: Test. I can put an email in here; it's optional. If I do put an email, it'll send me the IPFS links to my whitelist and my proof. I'll just leave that out for now. I've got a little set of addresses here that I can use. These are basically the addresses you get provided to you in Remix plus two of my dev wallets. So I'm just going to copy and paste that in there. Click on that. Okay, I've got my Merkle tree. So now that I've got that, it's pushed my whitelist out to IPFS. This is on Pinata. So now my whitelist is out there. I don't have to share this with anybody. Some folks may want to keep this private and that's okay, but at least you have it available to you. And then I also have my root hash, which is saved on IPFS as well.

So with this information that I've been provided, now I can go and generate a proof for any address that wants to try and mint. So for example, in this list of addresses, I want to verify this address and make sure that it's a valid... I want to generate a proof for this. And so what I'll do is in your frontend you'll send a POST request to our API. So to mimic that, I've got this example here where I'm going to hit the API to get a proof. I'm going to provide the IPFS link to my whitelist and then I'm going to provide the address that I want to generate a proof for. And when I do that, it'll hit the API and it'll come back and provide you this proof. So now that you have this proof for a specific address, then you can go and interact with your minting contract and see if this is a valid proof that they can mint.

And we have a sample Remix instance. This is a very simple minting contract just provided as an example. And so it's just called MyToken. And down here we're importing this OpenZeppelin library called MerkleProof. And what that is, is it's just a simple function that requires three arguments: you give it a proof, the root hash, and then the leaf that you want to verify against. So we're bringing that into our minting contract here and we're calling it. So for brevity, I'm not going to go into too much detail because I don't want to run out of time. But we're going to go ahead, I'm going to deploy this on Rinkeby with my address here. To deploy this I need the root hash, so I'll just grab the root hash real quick, which I can just get here that's on IPFS. Deploy that. Once that deploys, I'll have access to the minting function.

Okay, so once I've got that, I've got the minting function here. So to mint the token, I need to provide the proof. And so I just did that here, but I'm going to do it again. I'm going to grab the whitelist address, put that into my request body here, and the leaf is the same. So I'm going to call that again. Now I've got my proof. So I'm going to grab this and I'm going to try and mint. And it's already pulling in my address, so it's using the address that I provided a proof for. So I'm going to throw that in there for mint the token, approve this, wait for that to go through, and we should get a success. And there we go. So if I look here, there's my proof and I was able to mint an example ERC-721 with this address. So I know I'm kind of going through this very quickly. One of the things that potential use cases we see for this is, you know, other types of off-chain data validation, so it doesn't necessarily have to be a whitelist. We want to look at explore other potential use cases where you use Merkle trees.

One that I think would be really cool is for POAP whitelists. So imagine you just finished up an event, you get everybody's addresses, and you can basically generate on the fly a whitelist for those folks that were in that event so that they can go and then get those POAPs without having to rely like on a public facing or a public Google sheet that, you know, we typically see right now. So there's other areas to explore as well. And then roadmap items, some things that we want to work on are improved documentation and then do some improvements on the API. So for example, allowing a user to integrate their own IPFS gateway or own accounts. So there's not just Pinata, there's other services that interact with IPFS. So it would be nice for them to be able to use their own accounts. Provide easier API access to like do this programmatically without having to go to the frontend and do it. And then also create additional API endpoints for like custom Merkle tree applications other than the standard whitelist.

And I just did that in what, 10 minutes?

Nice one; that was really good, Jose. I just wanted to interject and say like, before Jose came to me with this, he was exploring Merkle trees pretty in-depth and I wasn't aware of how they worked. And yeah, he kind of turned me into a Merkle tree fanboy in the end because the cool thing is like, for example, why GitHub would use it, you take a hash of all the data or the set of data and that's all you need to compare. Two people can trustlessly compare the hashes and be like, okay, we're talking about the same set of data here. And you don't need to compare line by line your code or the files, you know? So something, one thing changes in those files, the hash completely changes and that's what's so powerful about Merkle tree verification and Merkle tree proofs is that, especially for decentralized web, all you need are those hashes and then yeah, it gives a lot of power to users to, you know, compare data in a trustless manner. So that's the big takeaway I took from all this which I really liked about Merkle trees. And you don't have to save tons of data on on-chain either.

Yeah, you only have to store the root hash. I skipped over that, but basically in your minting contract, all you need is one piece of data, which is your root hash. And so our vision as a team, what we've talked about is that we want to see this as a, you know, piece of the Web3 tooling that people can use for using Merkle trees without having to, you know, mess with all the implementation and other stuff that goes along with it.

Yeah, this is fantastic, gang. Thank you so much for sharing. Do you guys see any opportunities for us to perhaps use MerkleMe in the DAO or is it too early yet to know concrete use cases?

Vinny, I'll let you touch on that. To be honest, no, like, I kind of wanted to open up questions and we're all kind of new at this. This was, like I said, I didn't know about it until like I learned about it while working on the project pretty much based on Jose's initiative. So I was kind of looking for feedback from the community to see if there are other... I can imagine people might have in mind other use cases that we've not even thought about or other potential use cases. Or maybe someone has a good criticism that we didn't think about where they say, 'Ah, this is useless' or 'This will be useless for X reason.' So if anyone has any input, feedback, criticism, we would love to hear it.

I would have a question. So I looked into Merkle proofs myself already and so we're planning to do like a distribute ERC-20 tokens to like certain amounts for certain addresses. Is that possible with Merkle trees? So basically instead of just one address, you would have an address and an amount.

Yeah, I mean like we said, it's any set of data. So if you can pack in any other data in there with the address, if something changes then you'll have a different hash and yeah, it won't be verifiable in that sense. So totally, I can't say 100%, but sounds like a decent use case, yeah.

Okay, so instead of just the address you would have like address plus, I don't know, amount or some some combination.

Exactly. Exactly. You can even... like that's essentially what GitHub is doing, right? You have within a branch, you have a set of files, and anything changes within that one of those files within that branch, you're changing the hash. So from what I understand, definitely would work. That's a good idea.

And the files that you just showed, are they available somewhere on the internet?

MerkleMe.io. Like yeah, exactly. But the code, how to integrate in the frontend and the contract that you showed in the Remix, is something that's on GitHub too, right?

Yeah, it's all there. Can you say that again? All there, okay great. It's on the GitHub and on the documentation of MerkleMe.io, right? Yes, everything we have, everything is open source, it's on our GitHub, the API and the frontend, yes. Awesome, awesome. Let me get those links for you. Thank you very much. It's great, by the way. Oh thanks, thank you.

Any other questions? I'm to be honest, I was waiting for someone to just knock our idea down and be like, 'This isn't gonna work.' I'm still waiting for someone to tell us why it's not a good idea. Yeah. I see Jose. Jose wanted me to discuss the frontend. So yeah, go ahead.

No, somebody was asking about some specifics about the frontend, so I figured if you wanted to talk about it.

Yeah, well the frontend was Next.js and literally vanilla CSS. That's what we used on it. One second, let me pull up a stream. So I did the design on it—I'm not a designer, but I just was like me and Kenny, like we just got on a call one day and just kind of like jammed out to like doing something on Figma, but doing something with Figma and yeah, we just quickly put it together. Right now it's all of these are kind of messy right now because it's undergoing like a different look for the frontend. I got an actual designer and we're probably going to have a redesign. But yeah, we designed it on Figma and then we just spun up Next.js and then got the frontend started. Honestly, it was just like nights of just like being up all night just kind of fixing it, tweaking it, as much as we could until it was what we wanted it to be. But of course, I still want it to be a little bit more pristine, so we're working on that. But if Kenny want to talk a little bit more about the frontend as well, because he was... I was more like Padawan and he was more like Master Jedi on it. So yeah, if you want to pitch in and just talk about connecting the API and so forth, he can do that.

Okay. So the frontend is built with Next.js and yeah, we used the normal CSS and CSS module for the styling and also for state management, we actually, we decided... I said we should go with... we used Zustand, just a very simple state management because we did some simple state management transferring the data from a particular page to another page. So we used Zustand, a library for the state management, but we can probably in the future we are going to change it to the just simple context API for managing the state. And for the API interaction, we interacted with API, we consumed the API with Axios and the rest. Also for the rest, the yeah, it's just a very simple frontend and if you check the code is something you can understand. But the brain, everything is actually going on in the backend, which is what Jose has done and Vinny also. And also we almost, I think all of us, Jose and Vinny worked on the backend and Orlando and I, we worked on the frontend, but I would say all of us worked on the frontend because some of the stuff, you know, when we push, when we make a PR, there are changes to work on and most, all of us, we did work on the frontend and it is a very great experience for most of us. Yeah, it's the frontend is just a very simple Next.js application which you can, when you open it, you can something you can definitely follow up and I don't also have so many anything else to say.

Yeah, and also Peter Pan with like ethers or Web3. We didn't use it because I see Jose answering the question. We didn't use it because we didn't need to connect a wallet or anything like that, so most of it is just like backend API work so the frontend was pretty much very simple. It's just me designing and then we just go and build out the quick pages that we need to. So yeah, no need for it. I definitely thought we were going to need it as well but then I realized like, oh, we're not connecting a wallet, we're just like taking in data and sending back data, so. Excellent, excellent. We have about sixish minutes left. Does anybody have any final questions for the gang?

I was just going to mention, so for Pixel Devs, we were considering Merkle trees but then sort of went with a real-time server solution. The only problem we ran into consideration was having to continuously regenerate the snapshots, but obviously a tool such as yours you could I guess you could automate that a lot more easily too if you wanted to regularly update snapshots.

Yeah, absolutely, because all you have to do is just regenerate it and you just need to give your contract a new root hash. Right, right. Very quick and easy. It's a great project, thank you.

We appreciate you guys and Narb, we will now accept our winnings for Web3Con. You'll know soon enough, soon enough, friend. You'll know soon enough. But yeah, for real, this was a really fantastic project, gang. I personally was nerding out over here over Merkle trees. So thank you so much for sharing. And if there are no final questions here, I guess we can call it. Going once. I do want to say real quick, I mean, we are looking for like people to test it out. So if you want to try it out, definitely like feel free. MerkleMe.io, I'll just put it in the chat and then if you have any questions, just reach out to one of us. Jose is usually the best when it comes to like troubleshooting and answering the questions of like issues that may occur, but reach out to any one of us, me, Vinny, Jose, Kenny. We'll help you guys out.

Great stuff, great stuff. Yeah, for sure. All right, if there are no final questions, going once, going twice, sold. All right, thank you again, gang, team MerkleMe. Like I said, thank you so much for sharing. So everybody, stay tuned, I will be posting a link to claim POAPs for this event. And then team MerkleMe, I will DM you separately for your special speaker POAPs. But I want to wish everybody a very happy Friday or Saturday, wherever you may be. Happy weekend. And we will see you back here next week for another fantastic installment of DevNTell. All right, gang. Sweet, thanks guys. All right, cheers. Thank you.